RPi not booting after Update

[Hypernia]

I made a script to update and - if necessary - reboot my RPi automatically. This script invokes apt-get update, apt-get upgrade and rpi-update (by hexxeh). It worked several times and rpi-update was run several times succesfully also. My log files report everything went fine and a reboot is needed.

Now I've got the problem that my RPi doesn't start anymore. The red LED is on and the green light flashes 3 times, which means that loader.bin was not found. According to my research loader.bin should not be needed anymore. The SD card works fine when inserted in my laptop and all existing files on the boot partition look fine.

Does anyone have any ideas what the problem might be and how to fix it?

[RaTTuS]

... apt-get update, apt-get upgrade

Good

and rpi-update
....

not so good
as that will be bleeding edge and may not work .....

[malakai]

Your getting pretty aggressive on your updates. Probably should reboot after the initial upgrade then go for the experimental rpi-update. But if something went wrong and is corrupted in the boot just copying the boot from another card could work. Assuming you have one that is. I don't know how updated the light sequence is but it is showing something is wrong. And last update was:

3 flashes: start.elf not found
4 flashes: start.elf not launched
7 flashes: kernel.img not found

So start.elf would be the place to start replacing.

I still sit on the side if you don't know what's being updated don't update so frequently and definitely not so aggressively that a script does it. It would be horrible to read a post saying there is a big known problem and you pi kicks off the update while your reading the post. Murphy's Law.

[Hypernia]

Thank you Rattus for your information and thank you malakai for your detailed information

I downloaded the RPi Firmware from Github and compared md5sums to the files in my boot partition. There were no differences, all values were the same.

Reinserted the SD card in my RPi and still green LED is flashing 3 times

In case that helps anyone to help me:
fdisk -l output

Code: Select all

Disk /dev/sdc: 31.9 GB, 31925993472 bytes
64 heads, 32 sectors/track, 30447 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00017b69

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               5          60       57344    c  W95 FAT32 (LBA)
Partition 1 does not end on cylinder boundary.
/dev/sdc2              61       30447    31116288   83  Linux
Partition 2 does not end on cylinder boundary.

[Luutifa]

Yup, same problem here.
Pretty much exactly the same problem.
(I also had a script too )

What do?

Will I copy the boot partition from a fresh image to my card?

Or just backup what I can from the card and start a new one from a fresh download?

My script has "sync" at the end.

[WallaceIT]

same problem here using rpi-update, solved in this way from my linux laptop:

- downloaded previous firmware revision: https://github.com/Hexxeh/rpi-firmware/ ... 8ee909.zip

- extracted all "boot files" in the boot partition of the sd card (but this is not enough)

- extracted the vc folder in /opt/vc (on the sd card) replacing the existent folder

- extracted the modules/3.6.11+ folder in /lib/modules/ (on the sd card) replacing the existent folder

please note that BEFORE extracting files and folders you must (should?) annotate permissions on existent files and folders ( i used ls -al ) and AFTER verify/change the new permissions accordingly

Wallace

[Hypernia]

Thank you WallaceIT! That was it. I followed your instructions (did not have to chmod anything btw) and now my RPi lights and flashes in all colors again

[nommo]

I too made the mistake of running an rpi-update following an apt-get update and apt-get upgrade and got the three green blinkenlights.

I downloaded the previous firmware from the link WallaceIT posted, and on my windows laptop deleted start.elf from the SD card, and pasted the start.elf from the older firmware. Now it boots...

Note to self - stop running rpi-update!

[KTWS]

Same problem here. I thought it would have been fixed with the flurry of updates on the 26th but obviously not

[fixitnow]

I believe there must be something wrong with this specific update. I have been regularly updating through scripts and it always worked.
I played around for a few days and i can confirm that the issue is repeatable. I restored the card from an image taken a few days ago and I run into the same issue multiple times.
The interesting part is that it does not seem to happen with a plain vanilla raspbian installation.

[fixitnow]

I believe there must be something wrong with this specific update. I have been regularly updating through scripts and it always worked.
I played around for a few days and i can confirm that the issue is repeatable. I restored the card from an image taken a few days ago and I run into the same issue multiple times.
The interesting part is that it does not seem to happen with a plain vanilla raspbian installation.

[grahamed]

Hi

It seems that rpi-update is still broken as of 2013/03/29 2200. I wish I had read this post before updating.

[dom]

There do seem to be more update problems than usual over last week or two.
Nothing that seems likely to have caused this has changed.

This commit:
https://github.com/raspberrypi/linux/co ... b6b8ae7347

did seem to be at the start of the problems, and rbej felt it introduced update problems in xbmc.
This commit is an official upstream kernel patch, and appears correct and harmless. However I've reverted it to see if it helps.

The failure of the updates seem to be corruption to the FAT partion of the sdcard (usually resulting in 3-flash error on boot).
Because of this, if you are having problems updating, then copy off the files from the /boot partition and format the FAT partition (you can do this in windows and it won't affect the contents of the linux partition, although backing up the whole sdcard may be worthwhile).

Download fresh files from:
https://github.com/Hexxeh/rpi-firmware

You will want as a minimum bootcode.bin, start.elf, fixup.dat and kernel.img. Put them on /boot partition and copy back your backed up cmdline.txt and config.txt.

If you want to help with testing, then try running an extra rpi-update after booting, and check you can boot again. I've just done this twice, and it has been successful.
If rpi-update says you are up to date then
sudo rm /boot/.firmware_revision
followed by rpi-update will force it to update.

[fixitnow]

The positive side of this event is that it highlighted that my previous backup strategy via daily rsync to my NAS is not effective. Restore seems impossible either via PI itself seems or another box.
The only one that worked is in fact restoring from an older full image.
Does anyone have a better approach?

[grahamed]

Hi

Repaired as advised by Dom. rpi-pdate now OK, deleted version file, run again, still OK.. Thanks.

[fixitnow]

i had a somewhat different experience. I successfully restored my SD a few times, either from a previous image or by recreating the fat partition. However, running rpi-update would always corrupt it again. However I did not try to delete the version file. Not sure if i want to try now and invest again several hours later...

[fixitnow]

Just tried again. I forced fsck before, just to be sure that the card was in good shape.
rpi-update completed correctly but PI would not boot afterwards, with the usual 3 blinks.
The new thing is that the card was totally gone, i had to reformat it with my camera as both windoes and linux would not detect it anymore... this is getting weird, really.

[MaxK1]

Last night I tried an rpi-update (Just like last weekend) and the FAT partition looked ok. Copied off all the files just to be safe and sure enough, 3 blinks. fsck complained about bad short file names and other "issues", so I just used gparted in another Pi to re-format the FAT32 partition. Restored the files
and all was good. On the next system I upgraded, the FAT partition looked good (no errors) but 3 blinks later... All the required files seemed to be in place and fsck did not complain. Copied off my config.txt and cmdline.txt files, reformatted and copied the bare minimum from my working upgraded system+config.txt cmdline.txt rebooted and all was good. Same thing with my 3rd Pi (Although I had to do it twice - I'm sure I either performed steps out of sequence or somehow screwed up) It's easy to fix _if_ you have multiple Pi's, running off HDD's and at least one USB card reader. Not so easy if all you have is 1 Pi that just sits and blink at you...

[dallasjeff]

The same thing happens to me every time now and I've tried with 2 different SD cards. I have used rpi-update in the past so don't think it's anything that I've introduced. I'll try several of the different options presented here as a workaround. From what I can tell, it's happening to a number of other people as well.

[MaxK1]

Seems to be happening to cards that have been updated several times. 'start.elf' was on the card, but in one case, I had to let fsck rename a butchered version of the file name and then delete that file before it would boot. Almost as if boot.bin saw the deleted/mangled file as the correct file and tried to load that instead of the updated start.elf which was present.

[huminaFerk]

To supplement WallaceIT's tip, copy over the boot files and the vc folder from the link ; nothing else is needed. This removed the brick where dd'ing and copying other files did not.

[MrEngman]

I've been experimenting to try and figure out why rpi-update is causing the boot problems with the green LED flashing three times and the Pi not booting as it happened for me a couple of times last week. It appears that two of the rpi-update commits are causing the problem. I guess some code loaded by either of these two commits is then causing problems for the NEXT rpi-update.

The rpi-update commits I've found causing the problem appear to be from the 18 Mar (aea9d8e529baf2c11e0282706b71ae8cfc8ee909) and 24 Mar (61f7aa5742bc8f07be2378d727ac8dc14268e848) - details here - https://github.com/Hexxeh/rpi-firmware/commits/master. An rpi-update following either of these updates will fail to boot and show the green LED flashing 3 times repeatedly.

So if you've used rpi-update and its loaded the 18 Mar code then the next rpi-update that loads new code will fail to reboot with the flashing green LED. This will also happen if you used rpi-update and loaded the 24 Mar code, the next rpi-update will fail to boot.

From info in other posts it looks like the problem is down to some issue with writing the directory data, viewable in Linux with fsck so I believe, although viewing a broken image with Windows shows no errors, and a disk test in Windows shows no errors.

I have been able to fix the problem very easily however. Viewing the SD card with my Windows laptop, copy the files from the SD card /boot directory to a temp directory, format the SD card boot directory (quick format works OK) and then copy the files read from the SD card back to the SD card. No need to download any files from github. The SD card then boots OK - well they have for me - every time.

I have been able to generate the boot error multiple times and it is very consistent, and the fix has also worked every time. Really curious to figure out what's causing it to happen.

Looking at the directory images the faulty commits appear to be treating some filenames as long filenames when they are not and are normal 8.3 filenames. The file entries for START.ELF, START_X.ELF and BOOTCODE.BIN are being written in the directory listing as long filenames for some reason. Windows doesn't seem to find this a problem but Pi does as it cannot boot.

Example of what the directory entries look like for the faulty commits and what they look like when written correctly.

Faulty directory entries

Code: Select all

0000033792   41 73 00 74 00 61 00 72 - 00 74 00 0F 00 8F 2E 00   As.t.a.r.t.....
0000033808   65 00 6C 00 66 00 00 00 - FF FF 00 00 FF FF FF FF   e.l.f...ÿÿ..ÿÿÿÿ
0000033824   53 54 41 52 54 12 C0 20 - 45 4C 46 20 00 64 FA 15   START.À ELF .dú.
0000033840   84 42 84 42 00 00 FA 15 - 84 42 53 00 34 1F 29 00   „B„B..ú.„BS.4.).
0000033856   41 73 00 74 00 61 00 72 - 00 74 00 0F 00 DE 5F 00   As.t.a.r.t...Þ_.
0000033872   78 00 2E 00 65 00 6C 00 - 66 00 00 00 00 00 FF FF   x...e.l.f.....ÿÿ
0000033888   53 54 41 52 54 5F 58 00 - 45 4C 46 20 00 64 FA 15   START_X.ELF .dú.
0000033904   84 42 84 42 00 00 FA 15 - 84 42 9C 01 64 D3 37 00   „B„B..ú.„Bœ.dÓ7.
0000033920   41 62 00 6F 00 6F 00 74 - 00 63 00 0F 00 05 6F 00   Ab.o.o.t.c....o.
0000033936   64 00 65 00 2E 00 62 00 - 69 00 00 00 6E 00 00 00   d.e...b.i...n...
0000033952   42 4F 4F 54 43 4F 44 45 - 42 49 4E 20 00 64 FA 15   BOOTCODEBIN .dú.
0000033968   84 42 84 42 00 00 FA 15 - 84 42 5B 03 64 45 00 00   „B„B..ú.„B[.dE..
0000033984   00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00   ................

Good directory entries

Code: Select all

0000033088   53 54 41 52 54 20 20 20 - 45 4C 46 20 18 B8 4E 29   START   ELF .¸N)
0000033104   84 42 84 42 00 00 FA 15 - 84 42 FC 05 34 1F 29 00   „B„B..ú.„Bü.4.).

0000033152   53 54 41 52 54 5F 58 20 - 45 4C 46 20 18 74 4F 29   START_X ELF .tO)
0000033168   84 42 84 42 00 00 FA 15 - 84 42 8B 07 64 D3 37 00   „B„B..ú.„B‹.dÓ7.

0000033664   42 4F 4F 54 43 4F 44 45 - 42 49 4E 20 18 C0 51 29   BOOTCODEBIN .ÀQ)
0000033680   84 42 84 42 00 00 FA 15 - 84 42 57 09 64 45 00 00   „B„B..ú.„BW.dE..

MrEngman

[dom]

I've been experimenting to try and figure out why rpi-update is causing the boot problems with the green LED flashing three times and the Pi not booting as it happened for me a couple of times last week. It appears that two of the rpi-update commits are causing the problem. I guess some code loaded by either of these two commits is then causing problems for the NEXT rpi-update.

That ties in with my understanding. This appears to be the commit that caused the problem. It was added on March 11, and reverted on March 29.

This was the upstream commit:
https://git.kernel.org/cgit/linux/kerne ... cb469942b5

Ah! That had a bug in it. There was an upstream bug fix:
https://git.kernel.org/cgit/linux/kerne ... c64d4ed177

Currently this is reverted, but I guess it may be safe to pick up both those commits.

[haihaino]

I have been able to fix the problem very easily however. Viewing the SD card with my Windows laptop, copy the files from the SD card /boot directory to a temp directory, format the SD card boot directory (quick format works OK) and then copy the files read from the SD card back to the SD card. No need to download any files from github. The SD card then boots OK - well they have for me - every time.

Thank you very much. That was all that was needed to get my PI booting again.

[oboka]

I have been able to fix the problem very easily however. Viewing the SD card with my Windows laptop, copy the files from the SD card /boot directory to a temp directory, format the SD card boot directory (quick format works OK) and then copy the files read from the SD card back to the SD card. No need to download any files from github. The SD card then boots OK - well they have for me - every time.

Thanks, thanks, thanks!! It works for my SD card!