STICKY: USB boot now working on Pi3

[pws]

Update/Upgrade and reboot were all successful.

I did do both steps which ultimately worked - meaning that when I just did step 1, it stated that firmware was up to date. I then deleted (renamed) ".firmware_revision" then re-executed "sudo BRANCH=next rpi-update". This time, it did update everything.

Thanks!

Use sudo apt-get update && sudo apt-get -y dist-upgrade && sudo BRANCH=next rpi-update

You may need to remove /boot/.firmware_revision to encourage sudo BRANCH=next rpi-update to re-install the USB MSD & PXE bootcode.

Thx a lot, those to posts solved the upgrade problem for me.

[peterlite]

Has anyone tested netboot with power off to the building?
Based on my Pi and my router, the Pi will boot before the router is ready to connect the Pi to a net boot source. The source could also take a long while to boot.

[ermethic]

Can anyone please advise me on making a Libreelec bootable USB ?

[dgoadby]

The addition of the feature to be able to boot from USB devices is very welcome. However, the implementation seems to have created a firestorm of blog entries by Rpi3 users trying to make it work. Some blogs make it look like a black art to get a RPi3 to boot from USB devices.

I have been trying to get a couple of RPi3's to boot from USB devices for over a week and I think my findings might add some clarity. My goal was to boot from a 60GB SSD attached to a USB2 adaptor.

First, the lack of clear documentation on the correct use and real affect of parameters in the cmdline.txt is not helping. Where in the boot process does the boot_delay and boot_delay_ms value take effect? How does adding a file "timeout" help the boot process?

Anyway, this is what I manged to do and what I have found.... Hopefully, it might help others.

1) I created a 16GB USB memory stick that booted with no problems. The USB stick was a Sandisk Cruzer Fit. If the USB stick is fasdt enough then this does seem to work well. I didn't want to use a USB stick in my system as EXT4 is not recommend for them.

2) I created the half-way-house USB boot solution using the Adafruit script. The script isn't perfect but, after a bit of tweaking, I made it work reliably. The item that may catch some out is usbmount. If that is enabled then, after the script parted creates the paritions, the USB is mounted and the mkfs command fails due to the drive being mounted. The fix here is to set ENABLED=0 in /etc/usbmount/usbmount.conf. If you need usbmount then re-enable it after installation. The other subtle changes required relate to the sed editing of fstab which assumes a basic clean system. In the end the system booted 100% reliably.


3) I created a bootable SSD on a USB adaptor. I followed the instructions to the letter (many times) but the USB/SSD only booted once per 20 power-ups. Real hit and miss.

On a working system I checked the mount time (tail -f /var/log/messages) of the USB/SSD. The boot partition mounts in less then a second but the ext4 partition took between 5 and 6 seconds. This is probably the reason the boot process barely worked. It was the luck of the draw and clearly no use for a production system.

I tried at least four other USB/SSD adaptors that I had. They were all USB2 and the chipset shows as "generic" with an is code of 6116. I did, however, have a couple of USB3 adaptors. As USB3 is downward compatible with USB2 then I tried one -what's to lose? Voila! Immediate success with 25 power-ups booting perfectly every time.

Now, I tried the other "so-called" USB3 adapter. This did not work.

The USB3 adaptors were clearly not the same. The good USB3 adaptor has an ASMEDIA chipset (AS2115). The other one has an unknown vendor code:1f75 and product id:0621.

The only other information I can offer at this time is the USB3 adaptor that works perfectly has a single cable (no extra power) and the SATA connnector is also coloured blue. The other USB3 adapter has two cables, one is USB3 (blue) and the other USB2. The SATA connector is black. I am suspicious that is may be a USB2 adapter with a USB3 plug fitted. Until I have a PC with a real USB3 port I cannot easily tell.

The USB3 adaptor that works came from here:().

In summary, the boot code doesn't seem to wait long enough for an EXT4 filesystem to be mounted. A USB3 adapter seems to mount faster than USB2 and the boot process is successful - at least with one of my adapters. It may be worth trying a EXT2 system which is not journaled and this may mount faster. At least one brand of USB3 adapter does seem to be fast enough to work reliably so I have ordered more from the same supplier to see if they are consistent.

Until I have more ASMEDIA adapters to try I cannot 100% say that there is a solution. As the SD/USB hybrid boot set-up works 100% with all USB adapters then that is my fallback position. The boot fail problem is clearly a complex hardware/software one.

And finally, I used a USB power analyser to check the SSD current (0.1A peek 0.06 average). This showed that the USB voltage is not always present during the boot process. I do not know why this is but the power-up delays on the SSD must add to the delay issue. Tip: always purchase USB adaptors that have in-built LEDs as you can see both the power status and activity.

When I get more data on this I will update this blog.

[gsh]

Did you try using an SD card containing only the bootcode.bin file (from the next branch)

This has some fixes in it that should help with delays...

[KaySee]

Hi,

I run 'sudo apt-get update && sudo apt-get upgrade' on my Pi3 that was booting off an external USB drive, and rebooted.

Obviously I didn't do the 'sudo BRANCH=next rpi-update" so it now doesn't boot from USB drive.

I do not have a recent backup and do not want to overwrite my current config, or worse do a complete reinstall,

I have created a new install on an SSD card and updated it with both the previous commands, what I'm not sure about is what files to copy to the USB drive, and the easiest way to accomplish this, to get the drive booting again.

Any help would be appreciated.

KaySee

[pws]

I have been running an AMD SSD as a boot drive very reliably for a couple of weeks. Updates are performed regularly without issue. I also have a WD PiDrive set up as a boot drive and it has also been without issue.

If you follow the document published (msd.md), and update as instructed, it should work fine.

Also, remember that this is all still in testing.

[gsh]

I run 'sudo apt-get update && sudo apt-get upgrade' on my Pi3 that was booting off an external USB drive, and rebooted.

Oops...

Obviously I didn't do the 'sudo BRANCH=next rpi-update" so it now doesn't boot from USB drive.

I do not have a recent backup and do not want to overwrite my current config, or worse do a complete reinstall,

I have created a new install on an SSD card and updated it with both the previous commands, what I'm not sure about is what files to copy to the USB drive, and the easiest way to accomplish this, to get the drive booting again.

Just copy fixup* start* and bootcode.bin to the drive:

Other option would have been to go here and copy them over
https://github.com/raspberrypi/firmware/tree/next/boot

Gordon

[KaySee]

Just copy fixup* start* and bootcode.bin to the drive:

Other option would have been to go here and copy them over
https://github.com/raspberrypi/firmware/tree/next/boot

Gordon

Great that worked, thanks.

For some reason I couldn't replace the bootcode.bin on the drive after booting from the SD, and when I plug the drive in a Windows PC it showed up as being only about 17 KB, the one on GitHub is 49.6 KB, so maybe this was corrupted somehow.

In the end I copied the files from GitHub and the RPi booted up fine.

Just out of curiosity, could I just copy the whole of the 'firmware-next/boot' folder to my USB's boot partition?
Also could I make any USB MSD bootable just by copying this to a bootable fat32 partition, with a suitably edited cmdlines.txt, config.txt and fstab?

Finally what does the command 'dpkg-reconfigure openssh-server' do and why is it necessary?

KaySee

[KaySee]

and update as instructed

Where is this? I couldn't find any update instructions, unless you mean from this forum thread.

KaySee

[pws]

In this thread:

DougieLawson wrote:
You may need to remove /boot/.firmware_revision to encourage sudo BRANCH=next rpi-update to re-install the USB MSD & PXE bootcode.

DougieLawson wrote:
Use sudo apt-get update && sudo apt-get -y dist-upgrade && sudo BRANCH=next rpi-update

and update as instructed

Where is this? I couldn't find any update instructions, unless you mean from this forum thread.

KaySee

[DougieLawson]

You shouldn't need to do anything special other than sudo BRANCH=next rpi-update there was a new build with a 4.4.27 kernel this week.

[dgoadby]

Update: I bought another USB3 adapter which has the ASMEDIA chipset. And, the good news is it works too. I booted the same SSD and no problems at all.

In case I was seeing things I tried all of the other adaptors again and none worked. So, it looks like the ASMEDIA/USB3/SSD solution works every time.

I’m ordering some more for my project while stocks are available.

I hope this information is helpful to others.

[gsh]

What would be really useful is to know whether the one that doesn't work now works with a bootcode.bin on the SD card... This would tell us whether the problem has been fixed or whether this is a massive failure waiting to happen...

Thanks

Gordon

[polychaeta]

Hi,
assuming I have a bootable USB mass storage device AND a bootable sd card inserted, is there a setting which device will be preferred for booting?
Thanks

[gsh]

It'll always default to SD booting, then fall back to mass storage if it fails to find a bootcode.bin (or if bootcode.bin is the only file on the SD card it'll look for all the other files on the MSD or ethernet)

[datajerk]

Hi,
assuming I have a bootable USB mass storage device AND a bootable sd card inserted, is there a setting which device will be preferred for booting?
Thanks

Search back a few posts in the thread. I have the same question. Here's what I know so far:

This document implies that with the correct OTP settings you can then use GPIO pins to select what to boot from: https://www.raspberrypi.org/documentati ... ootflow.md

However it is not clear how to enable this mode. I've submitted a gitlab issue asking for clarity on this.

The other option (still in this thread), is to short CMD and Vcc (not recommended, look back at this thread) on the SD card for a virtual eject. This works for me about 99% of the time (yes I have measured it). So, I've gone forward with this approach.

My requirement is for fully automated testing of an SD flash image without having to rely on access to that image in the event of a failure. So hacks like dual partitions, or editing /boot, etc... are not an option.

My autotesting rig has a sparkfun microsd breakout board with the microsd card in it. I have the SD card CMD and Vcc connected to a relay. The RPI3 is connected to an Odroid C2 emulating USB mass storage from its OTG port. This way I can remotely control the contents of the USB drive. Lastly I use a Yepkit hub to power the RPI3 and also power it on/off. Oh, and I use a "null modem cable" to get to the serial console by connecting the GPIO serial pins of the Odroid (ttyS1) to the GPIO serial pins of the RPI3 for console access (the Odroid has 2 serial ports, S0 for console, and S1 for whatever you like).

With this setup from the Odroid C2 I can see the RPI3 console, power it on/off, and via the relay insert/eject the SD card. The later controls booting from SD or USB. The USB images are delivered by OTG from the odroid and managed there as well.

About 1 out of 100 the CMD/Vcc short does not work. To fix I just power cycle (power off, sleep 5, power on). My USB RPI3 image has sshd and I use that to test if the boot to USB worked (ssh hostname some verification command). Once booted from USB, its safe to turn the relay off to remove the short. In < 1 sec the SD card shows up to the USB OS (from /dev/sda) as /dev/mmcblk0 and I can then dd a test image and just type reboot to boot it.

My current application takes 35 min to go from dd, to boot, to completing all tests. I have had this running continuously for about a week without issue and have completed 100s of tests.

Again, the threads above do not recommend this approach, but it works for me now.

That all said my preference would be:

1. Network boot first, then fail to USB, then to SD. With this I could control network vs SD via DHCP or TFTP control. Untested but I assume that if the TFTP payload is missing for that host then it would fail through to the next boot method (like x86 does with PXE configure to boot before HD).
2. The GPIO control based on the docs above.

Let me know if you figure out something more elegant.

[polychaeta]

datajerk, I remeber your posts. Interesting approach you got there!
For me, it would be nice if I could set the USB MSD to the first boot device, and on failue look on the sd card. Assume there is a production system on the sd. When you wish to update it, you attach a specially prepared usb key, boot from it, flash the sd, unplug and reboot into the updated production system. In this scenario you could update the firmware without opening the device and replacing the sd card or so. This could of cause work with network boot as well. This would be such a nice feature...
Poly.

[rhester72]

A few (unanswered, I believe) questions, if I may (hi, Gordon! =):

- Given that program_usb_boot_timeout (which is incorrectly documented in bootflow.md and host.md as program_usb_timeout, BTW - caused me no end of confusion at first presumably blows an OTP fuse, is it reasonable to assume that means you can never remove the +5s timeout once set?

- In a similar vein, once program_gpio_bootmode is properly documented (so we know what to set it to to default to/force USB boot instead of having to leave an empty microSD card inserted), if we do set it for USB-only boot, and that blows an OTP fuse, does that mean we can never set the RPi to boot from SD (only) again?

- Is there any further information on why the load average goes up to 0.25 (i.e. one core fully utilized) when booting from USB? In Raspbian, it appears related to a kworker thread, but I'm at quite the loss as to what it's actually doing (other than raising my core temp 11C and processing 2x the interrupts and 8x the context switches, as shown by munin).

Much thanks for any insight,

Rodney

[with ice cream]

Cold USB booting (from no power) works, rebooting fails. Reproducible. I already activated the additional delay ('program_usb_boot_timeout=1' in config.txt) but that doesn't help. What else can I try? (Using an SD card with a cmdline that points to /dev/sda2 is my current workaround.)

[ravustaja]

Has anyone managed to boot from an USB stick/drive while they have more than 1 device connected to the Pi?
I managed to get my Pi to boot from an USB stick, but after I connect another USB drive to it, it stops booting. Nothing happens when I turn the power on.

[Paul Webster]

Perhaps you are trying to draw too much power from RPi USB.

[ravustaja]

Perhaps you are trying to draw too much power from RPi USB.

Nope, that's not the case. Actually the Pi is connected to a powered USB hub which is then connected to the external HDD.

[PeterO]

Since there is not a way to indicate which of the two drives should be "/dev/sda" and which should be "/dev/sdb" it will come down to the order that usb discovery finds them, so my guess is that it is trying to boot from the wrong drive.

Can you try swapping over the USB connections between the two drives ?

Or it could be that the "experimental" firmware has not been tested with two or more drives connected.

PeterO

[ravustaja]

I have set PARTUUID in cmdline.txt and disk-by-uuid in fstab, so that shouldn't be the problem either. It seems like after connecting the hub, it isn't even trying to boot. Nothing just happens when I turn the power on. Everything works fine when I unplug the hub. Switching them around doesn't work either.