How fast a Raspberry Pi can boot?

[markokrajnc]

Hello!

Today I measured boot loader times in different configurations. Because this may be interested to somebody else I am posting my results here...

The idea behind this test is to write bare metal programs for Internet of Things which can boot very fast and can be used in different cases - for example for a lawn mower which can not wait 30 seconds to boot before it can be used...

What did I measure?
I measured the time the Raspberry Pi needs to boot: from power on until it can send the first byte over serial port.

Tested computer:
- Raspberry Pi Model A (Revision 2)
- Latest Raspberry Pi firmware (downloaded today, August 17, 2014 from https://github.com/raspberrypi/firmware ... aster/boot)
- Default settings (config.txt has only start_file and fixup_file settings to choose the desired firmware ELF file and gpu_mem=16 setting in case of start_cd.elf)
- tested with SD Cards:
1) GoodRAM MicroSDHC, Class 10, 8 GB
2) Samsung MicroSD, 2 GB (older card, can't find class information, part number is: MMAGR02GUECA-MB)

How did I measure it?
I used another Raspberry Pi which turns the tested RPi on with Sainsmart Relay and then waits until the first byte is received over UART serial port (both RPis were cross connected over UART).

What configurations did I use?

As you know the Raspberry Pi boots in several stages:
1) First stage bootloader: stored in ROM and runned by GPU (you can not change this part)
2) Second stage bootloader (part of RPi firmware): bootcode.bin file on SD card, runned by GPU from L2 cache
3) Third stage bootloader (part of RPi firmware): start.elf or start_cd.elf or start_x.elf file on SD card, runned by GPU from RAM
4) Kernel: kernel.img file on SD card, runned by ARM CPU from RAM (this is usually operating system kernel)

Boot configurations:

A) Herman H Hermitage uart01 second stage bootloader (GPU bootloader bootcode.bin, see: https://github.com/hermanhermitage/vide ... ter/uart01)
This configuration involves only stages 1 and 2 and completely bypasses RPi firmware...

B) Raspberry Pi firmware start_cd.elf + dwelch67 uart02 kernel.img (ARM, see: https://github.com/dwelch67/raspberrypi ... ter/uart02)
(Stages 1, 2, 3 and 4)
Contents of config.txt:
gpu_mem=16
start_file=start_cd.elf
fixup_file=fixup_cd.dat

C) Raspberry Pi firmware start.elf + dwelch67 uart02 kernel.img (no config.txt file, stages 1, 2, 3 and 4)

D) Raspberry Pi firmware start.elf + dwelch67 uart02 kernel.img + minimal config.txt file (stages 1, 2, 3 and 4)
Contents of config.txt:
start_file=start.elf
fixup_file=fixup.dat

E) Raspberry Pi firmware start_x.elf + dwelch67 uart02 kernel.img + minimal config.txt file (stages 1, 2, 3 and 4)
Contents of config.txt:
start_file=start_x.elf
fixup_file=fixup_x.dat

(Regarding dwelch67 uart02: I wanted to use uart01, but uart02 was already precompiled and it outputs 12345678 on UART before everything else... I spent too much time with everything else, so I skiped setting up and compiling uart01... )

I measured everything 10 times and runned garbage collection before every measurement so it didn't affect the result times.

How fast you can boot your Raspberry Pi?

Raspberry Pi Boot Times for different configurations (SD Card: GoodRAM MicroSDHC, Class 10, 8 GB):

Configuration A ... 234 ms
Configuration B ... 1011 ms
Configuration C ... 1409 ms
Configuration D ... 1407 ms
Configuration E ... 1534 ms

Raspberry Pi Boot Times for different configurations (SD Card: older Samsung MicroSD, 2 GB):

Configuration A ... 404 ms
Configuration B ... 1046 ms
Configuration C ... 1479 ms
Configuration D ... 1480 ms
Configuration E ... 1545 ms

Yes, configuration D may even be faster than configuration C: if you provide minimal config.txt than booting may be slightly faster but not always...

If you want to boot in less than 250 ms, then you have to write your own bootloader.bin which will run on GPU in L2 cache. If you are OK with 1 second boot time, then you don't have to bother with GPU and write only ARM code...

Question for the Raspberry Pi Foundation: Would it be possible to get "start_min.elf" (or even "bootloader_min.bin") which would only initialize RAM and ARM CPU, load kernel.img and start running ARM? This could get us bellow 400 ms boot time which would be great for Internet of Things devices and competition with Arduino community... And because this code wouldn't contain any GPU stuff you could also make it open source for us... Or maybe you can regard it as an "GPU example bootloader" or "bootloader tutorial"... Thank you!!!!!

So much for now...

Marko Krajnc

[mimi123]

For you, there is a minimal start.elf which work GPU side but with C instead of ASM .

Disclaimer: It is my project.

https://github.com/freeblob/freeblob

The C compiler can be compiled with: git clone git://github.com/freeblob/ack-vc4 -b master;cd ack-vc4;make -i; sudo make -i install.

It also needs binutils-vc4. git clone git://github.com/mm120/binutils-vc4 -b vc4; cd binutils-vc4; ./configure --target=vc4-elf ;make ; make install
And you copy videocoreiv.arch to /home/marmar01/src/rpi/videocoreiv/

[mimi123]

For Freeblob help, look @ #freedom-videocore on Freenode.

[markokrajnc]

Hello mimi123!

Is freeblob just a "Hello World" in C for GPU or is it a complete bootloader?

What I would expect from a "complete bootloader":
- RAM initialization (including split between ARM/GPU)
- ARM CPU initialization (including CPU frequency)
- loading of kernel.img
- starting kernel.img code

I've seen freeblob before, but wondered if this is just a "Hello World" example with an attempt to make a bootloader later...

Marko

[mimi123]

Hello mimi123!

Is freeblob just a "Hello World" in C for GPU or is it a complete bootloader?

What I would expect from a "complete bootloader":
- RAM initialization (including split between ARM/GPU)
- ARM CPU initialization (including CPU frequency)
- loading of kernel.img
- starting kernel.img code

I've seen freeblob before, but wondered if this is just a "Hello World" example with an attempt to make a bootloader later...

Marko

The RAM init is in the bootcode, Freeblob can access to all system memory. The freqs are also setted up by bootcode.bin.
There is a ARM bootstrap version but it doesn't work currently: git://github.com/freeblob/freeblob-arm (bad linker script, anyone which knows binutils well can help)

200th post

[flakeshake]

-loading of kernel.img

Don't you need an SD card driver for that ? And filesystem support ?

[mimi123]

@flakeshake: Currently the kernel.img in Freeblob is part of the start.elf

[markokrajnc]

Ideally I would like to have a bootcode.bin source code (in GPU assembler or C) which does the following:
- RAM initialization (including split between ARM/GPU)
- ARM CPU initialization (including CPU frequency)
- loading of kernel.img
- starting kernel.img code

Nothing more (no graphics or minimal framebuffer graphics).

If all this can be packed into bootcode.bin (or "bootcode_min.bin"), this would be great! If not, additional "start_min.elf" would be OK (which would do the things, which bootcode.bin can not, because it would be too big for the L2 cache).

Marko

[fuffkin]

Marko,

Those boot times are incredible! What is the functionality of these minimal loads once booted? Have you done any tests to measure the time to take an image from the camera?

Jon

[ghans]

Note that he never seems to boot Linux , which would take
lots of extra time in the standard configuration.


ghans

[PiGraham]

The basic 'Testdrive' of buildroot boots impressively fast. A few seconds from power on to login prompt.

I downloaded the Test Drive sd card image, extracted it and burned it to SD card using win32diskimager. First boot is a bit slower, then it flies.

Baremetal can get you near instant start.

[markokrajnc]

Those boot times are incredible! What is the functionality of these minimal loads once booted? Have you done any tests to measure the time to take an image from the camera?

Configuration A (Herman H Hermitage uart01 second stage bootloader) does nothing more, than:
- initializing UART01 serial port and
- sending characters to UART01 serial port
right after power-up (from VideoCore 4 GPU processor).

Configurations B, C, D and E are:
- loading different versions of firmware (start_cd.elf, start.elf or start_x.elf)
- initializing RAM, ARM CPU and other firmware stuff,
- loading kernel.img which does nothing more, than:
- initializing UART01 serial port and
- sending characters to UART01 serial port.
(from ARM processor).

Like ghans has said, it never boots Linux or any other OS...

If anything else is needed (like taking image from the camera), you would need to initialize camera by yourself and take images by yourself (this means writing your own drivers or extracting them from Raspbian or other OS source code).

Marko

[mimi123]

The HHH loader runs on the L2 cache (128K of RAM) unlike Freeblob. (full memory)
I would like to see your stats about running the normal Freeblob ( printf used only , not ARM).

[ghent360]

Have you tries BOOT_DELAY=0, allegedly here is default delay in start*.elf and this should remove it?

[mrvn]

For measuring boot times I used the system timer and it's free running 64bit count register. Could you print that out in your boot test and compare it with the times you measured externally? I wonder if the clock starts automatically on power on or if the VC has to boot up first and configure it. The later would make the clock report less boot time than there really is.

Thanks in advance.

[eriktheitalian]

I'm tried porteus distro. I'm remebering fastest usb boot from this.

[DougieLawson]

Is this stuff important, when on average it takes 16 seconds to shutdown AND boot my RPis.

[shuckle]

How did you reach 16 seconds?
My raspi reboots no less than in 32 seconds. 16 needs very heavy tuning?

My last lines in dmesg:

Code: Select all

[   22.055065] smsc95xx 1-1.1:1.0 eth0: hardware isn't capable of remote wakeup
[   23.707617] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xC5E1

Can you show yours?

[kriss]

16 second is very slow
sadly i lost my µsd adapter so i didn't used my PI2 yet
this horrible timing should make the foundation to dev something more usable than a kernel

[DougieLawson]

How did you reach 16 seconds?
My raspi reboots no less than in 32 seconds. 16 needs very heavy tuning?

I determined it by setting all of my RPis to reboot with sudo shutdown -h 12:30 and then on reboot looking at the btime value from /proc/stat

Code: Select all

btime 1428838216

that's the timestamp when the kernel passes control to the init program.

[shuckle]

We can of course discuss what boot ime is, but I believe most people mean that it is something visible. Like desktop or login prompt or sshd deamon ready to accept connections.

[markokrajnc]

We can of course discuss what boot ime is, but I believe most people mean that it is something visible. Like desktop or login prompt or sshd deamon ready to accept connections.

When I measured the boot times, the characters send from RPi over UART01 serial port were visible on my desktop PC... The main reason for these measurements was: "Can I use the RPi for building devices which must work almost instantly after powering up?"