Question about Network Booting on the Raspberry Pi 3

[rbutani]

I apologize if this is in the wrong section (I'm new to the forums), but I've been toying with Network Booting for the Pi 3 and I have a question: Is it possible to run the DHCP Server and the TFTP Server needed for network booting on separate computers?

The language in the documentation about network booting seems to imply that it is (emphasis mine):

...use tcpdump on the tftpboot server (or DHCP server if they are different.)

..but the Pi (in the captures from tcpdump) seems to always request bootcode.bin from the device running the DHCP Server, regardless of what the pxe-service entry in dnsmasq specifies.

For reference the line I added to my dnsmasq.conf is:

Code: Select all

pxe-service=0,"Raspberry Pi Boot ",192.168.1.4

..where 192.168.1.4 is the computer running the TFTP Server. I also tried adding a filename ("bootcode.bin") as the third argument and the IP as the fourth, which also did not work.

I know that the above is the standard way to direct PXE booting devices to a different computer for TFTP; is there a different way to do this for the Pi?

Thanks in advance.

[patrikg]

Have you read this from https://www.raspberrypi.org/documentati ... des/net.md
What i can see you just enter one space at the end.



The important part of the reply is the Vendor-Option Option 43. This needs to contain the string "Raspberry Pi Boot", although, due to a bug in the boot ROM, you may need to add three spaces to the end of the string.

Code: Select all

pxe-service=0,"Raspberry Pi Boot   ",192.168.1.4

[FM81]

I've also running DHCP and TFTP on different machines. My DHCP is an Raspberry B1 and all the other services like TFTP, NFS and so on are running on an X86-PC.
The relevant part of my dnsmasq-config is:

Code: Select all

dhcp-mac=rpi-boot,b8:27:eb:XX:XX:XX
dhcp-mac=rpi-boot,b8:27:eb:YY:YY:YY
dhcp-mac=rpi-boot,b8:27:eb:ZZ:ZZ:ZZ
dhcp-option-force=net:rpi-boot,43,"Raspberry Pi Boot"
dhcp-option-force=net:rpi-boot,66,192.168.0.100
# following not needed, RPi always requests 'bootcode.bin'
# dhcp-option-force=net:rpi-boot,67,"bootcode.bin"
dhcp-option-force=net:rpi-boot,encap:175,176,1b

rpi-boot is the tag, to give the options below only to the raspberrys with the corresponding MAC-addresses. 192.168.0.100 is my TFTP.

But the internal boot-ROM of an RPi3 seems to be very tricky?
1. NetBooting any X86-machine works at 100% with my setup. Of course they refer not to 'bootcode.bin', they use 'pxelinux.0' or such.
2. NetBooting a RPi2 with an nearly empty SD-card (only bootcode.bin from NEXT-branch on it) works at 100% with my setup.
3. NetBooting a RPi3 with an nearly empty SD-card (only bootcode.bin from NEXT-branch on it) works at 100% with my setup.
4. NetBooting a RPi3 without SD-card (by using the internal boot-ROM) works at ~50% with my setup.
I cannot determine before if it works or not in this last case. Sometimes no DHCPDISCOVER is seen, sometimes I see this, but nothing is pulled via TFTP, sometime it works. I've also tried several switches, but no relation here ... But I think, there is no chance, to reflash the boot-ROM with better firmware at home? (These last sentences only refer to point 4.)

Greetings, FM_81

[milhouse]

Interesting. I just added the following line to the dnsmasq config on my Raspberry Pi3 "server" that handles DHCP and DNS, in order to successfully netboot RPi3 clients over TFTP (and NFS) running on 192.168.0.9 (x86 Ubuntu PC running dnsmasq - tftp only - and nfs-kernel-server respectively):

Code: Select all

dhcp-boot=pxelinux,192.168.0.9

The above is 100% reliably netbooting the RPi3's without any SD card.

[FM81]

It makes me wonder more and more! How can a Raspberry 3 handle 'pxelinux' as bootfile? Everybody says, 'bootcode.bin' is needed? (No matter if via network or from a SD-card.)

Some bugs are described here:
https://www.raspberrypi.org/blog/pi-3-b ... e-awesome/

Greetings, FM_81

[milhouse]

It makes me wonder more and more! How can a Raspberry 3 handle 'pxelinux' as bootfile? Everybody says, 'bootcode.bin' is needed? (No matter if via network or from a SD-card.)

I don't think it matters, the RPi3 just requests "bootcode.bin" regardless of how dnsmasq is configured. You could try putting "bootcode.bin" in place of "pxelinux" and I'm sure that would work too - I left it as pxelinux in case I decide to boot something other than an RPi3, eg. an x86 device, which might respect the filename property.

I've tested booting LibreELEC and not had any problems (bootcode.bin/start.elf/fixup.date from branch=next dated 11 August, testing with recent LibreELEC master and a custom commit) - this is the shared RPi3/cmdline.txt:

Code: Select all

boot=NFS=192.168.0.9:/nfs/System/@UID@ disk=NFS=192.168.0.9:/nfs/Storage/@UID@ quiet ssh ip=dhcp

where @UID@ is automatically replaced by LibreELEC with the serial number of the booting client eg. ff449f90.

The TFTP/NFS directories:

Code: Select all

$ ls -la /home/neil/pxeboot/nfs/System
total 68
drwxr-xr-x 4 root root  4096 Aug 11 10:39 .
drwxrwxr-x 6 root root  4096 Aug 11 10:00 ..
lrwxrwxrwx 1 root root     4 Aug 11 10:39 3b952c03 -> RPi3
-rw-r--r-- 1 root root 50820 Aug 11 10:03 bootcode.bin
lrwxrwxrwx 1 root root     4 Aug 11 10:39 fe129e7b -> RPi1
lrwxrwxrwx 1 root root     4 Aug 11 10:02 ff449f90 -> RPi3
drwxr-xr-x 3 root root  4096 Aug 11 10:02 RPi1
drwxr-xr-x 5 root root  4096 Aug 15 21:17 RPi3
$ ls -la /home/neil/pxeboot/nfs/Storage/
total 12
drwxrwxr-x  3 root root 4096 Aug  9 04:02 .
drwxrwxr-x  6 root root 4096 Aug 11 10:00 ..
drwxr-xr-x 42 root root 4096 Aug  9 05:15 3b952c03
drwxr-xr-x 42 root root 4096 Aug  9 05:15 ff449f90
drwxr-xr-x 42 root root 4096 Aug  9 05:15 fe129e7b 

[FM81]

OK, thanks.
But all these different variants didn't explain, why, under same "race conditions", sometimes no DHCPDISCOVER is sent, or if it is sent, no Bootfile will be requested? And why all this only happens, when internal boot-ROM is used?

No DHCPDISCOVER is like "a showstopper", in this case it doesn't matter, what you have inside your config-files ...

Best Regards, FM_81

UPDATE @milhouse: At one point you are right, the line with 'bootcode.bin' isn't needed, it seems "hardcoded in boot-ROM" and I've got same behaviour without it!

[hazeii]

Empirically the Pi requests 'bootcode.bin' (and totally ignores the usual DHCP bootfile offer). Normally both DHCP and TFTP are running on the same server, so all DHCP needs to do is return the server address (it is possible to return a different server address, and the Pi will then grab bootcode.bin and other files from the other server -tested this).

Not clear why the Pi suggests PXE in its initial request (it's not an X86, and can't see an architecture in the tcpdump output).

The most common problem appears to be the rather-too-quick DHCP request in the current implementation - an issue that;s clearly noted in the post about network booting. Seems to work more often on gigabit than 100Mb (on a rather haphazard home network).

Netboot fans may find it a bit annoying having to dump all the files the Pi needs into the tftpboot directory (simplistically, everything in /boot). Better solution welcome! Local workaround is to put the files into /tftpboot/boot.pi/, then create a link to that directory for each disklesss Pi based on its serial number, e.g. 'cd /tftpbooot && ln -s boot.pi 81724fcf' (why this works is explained in the docs linked from the blog post @ https://www.raspberrypi.org/blog/pi-3-b ... e-awesome/).

[baeckerman83]

where @UID@ is automatically replaced by LibreELEC with the serial number of the booting client eg. ff449f90.

So this is only working with LibreELEC? Is there a solution to boot from different folders If I want to user raspbian? I'Ve two RPI3 but only get one booting from network because the second one would boot from the same folder.

[milhouse]

where @UID@ is automatically replaced by LibreELEC with the serial number of the booting client eg. ff449f90.

So this is only working with LibreELEC? Is there a solution to boot from different folders If I want to user raspbian? I'Ve two RPI3 but only get one booting from network because the second one would boot from the same folder.

Correct, this is only working for LibreELEC as we use our own custom boot loader. I don't know of a similar solution for Raspbian when it comes to the root partition (the boot partition automatically uses a folder based on serial number, if available).

[baeckerman83]

Correct, this is only working for LibreELEC as we use our own custom boot loader. I don't know of a similar solution for Raspbian when it comes to the root partition (the boot partition automatically uses a folder based on serial number, if available).

Please can you explain me that?
Do I've to change the cmdline.txt? Or online the path?
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/nfs nfsroot=192.168.2.1:/ospath/nfs/serialnumber1 rw ip=dhcp rootwait elevator=deadline

or

dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/nfs nfsroot=192.168.2.1:/ospath/nfs/ rw ip=dhcp rootwait elevator=deadline

but the path had to be 192.168.2.1:/ospath/nfs/serialnumber1 on the root device?

[milhouse]

Please can you explain me that?

As I said, I don't know of a Raspbian solution for the root partition. However for the boot partition, it's explained in the documentation:

From this point the bootcode.bin code continues to load the system. The first file it will try to access is [serial_number]/start.elf. If this does not result in a error then any other files to be read will be pre-pended with the serial_number.

[baeckerman83]

Thank you! This helped me. Because I looked up in the wrong documentation.

[spsarolkar]

@rbutani:

I am trying to network boot my Raspberry PI 3 from my x86 Linux box. I am facing some issues with it. I have documented the steps I have taken https://raspberrypi.stackexchange.com/q ... -x86-linux. I have also attached the tcpdump and dhcpdump for the same.

Can you please let me know if I am missing something, can you please help me identify the issue.

Thanks,
Sunil Sarolkar