Pi4 64-bit raspbian kernel for testing - Focus on Pi4

[dom]

As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.
This is for bleeding edge testers. Things may not work. Backing up, or using a fresh sdcard is a wise precaution.

To get the kernel run the usual

Code: Select all

sudo rpi-update

to get the latest kernel. If you want to switch to 64-bit kernel add to config.txt

Code: Select all

arm_64bit=1

You can still the usual 32-bit raspbian buster image just fine, but the kernel will be 64-bit.
Installing and running 64-bit packages should now be possible.

The 64-bit kernel (even with 32-bit raspbian) should have some performance advantages with kernel heavy loads (e.g. usb/ethernet) as we avoid some of the page mapping costs of LPAE. There may also be some (lesser) performance boost from the 64-bit kernel code.

Feel free to try it. Report any tests that run faster (or slower) and anything that does or doesn't work.

Note: We're focusing on Pi4 here. In theory the kernel does run on Pi3, but that's not our priority.
The lack of FIQ on 64-bit kernel will reduce USB performance on Pi3 (Pi4 doesn't require the FIQ).

Code: Select all

pi@pi4:~ $ uname -a
Linux domnfs 4.19.69-v8+ #1261 SMP PREEMPT Tue Sep 3 20:27:16 BST 2019 aarch64 GNU/Linux

Known issues:
vcsm driver doesn't work with 64-bit (only the cma version) which will break software decode with kodi
Mathematica and Wolfram fail to run (detection of raspberry pi os fails) Workaround here
Java 8 doesn't work. OpenJDK 11 does work.

[jdonald]

This is awesome.

performance advantages with kernel heavy loads (e.g. usb/ethernet)

Is the FIQ issue with dwc_otg fixed? Not sure which GitHub tickets track that issue but last I saw detailed mention of it is in rst's post.

If this is unresolved, my impression is that it breaks plenty of USB webcams and other devices transferring in bulk are slower than their 32-bit kernel behavior? If so that would make it hard to properly benchmark kernel-heavy USB loads.

First thought for anyone doing ethernet benchmarking: you must use unencrypted rsync/ftp/rsh/http or else the cipher will bottleneck on the CPU in userland.

[6by9]

This is awesome.

performance advantages with kernel heavy loads (e.g. usb/ethernet)

Is the FIQ issue with dwc_otg fixed? Not sure which GitHub tickets track that issue but last I saw detailed mention of it is in rst's post.

Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).
As noted in the title, this is mainly aimed at pi4.

[epoch1970]

As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.

Great news. Well done!

[jamesh]

As the 64-bit kernel is getting more functional, I've decided to build it as part of rpi-update kernel to allow more users to test it under raspbian.

Great news. Well done!

A lot of work has been done by the community and upstream devs, so very much a combined effort.

[DougieLawson]

Can you remind me, which Raspberries does 64 bit work on? I know 3B+ and 4Bs are OK, as I've run Sakaki-san's Gentoo on my 3B+ (back in February).

What's the revision numbers for the 64-bit capable boards?

[DougieLawson]

This doesn't work on a 3A+ with

Code: Select all

arm_64bit=1

but does work OK with

Code: Select all

kernel=kernel8.img

.

[epoch1970]

Can you remind me, which Raspberries does 64 bit work on? I know 3B+ and 4Bs are OK, as I've run Sakaki-san's Gentoo on my 3B+ (back in February).

Pi 3B is ok, I suppose Pi 2 rev. 1.2 (with the same SOC as 3B) runs in 64bit mode as well.
A good year (or is it more...) ago I had a 64b buildroot OS working just fine on my Pi 3Bs (linux tree off RPF's Github repo), but I switched back to 32 because goodies like vcgen (?) or some GPIO libs were not available. The full Pi "experience" was not there yet.

[jamesh]

As Dom said, this focuses on Pi4, although MAY work on other Pi's. I've changed the title to reflect that.

For revision codes that say which processor you have, see here...https://www.raspberrypi.org/documentati ... /README.md

[geev03]

....
You can still the usual 32-bit raspbian buster image just fine, but the kernel will be 64-bit.
Installing and running 64-bit packages should now be possible.
......
Feel free to try it. Report any tests that run faster (or slower) and anything that does or doesn't work.

.....

Code: Select all

pi@pi4:~ $ uname -a
Linux domnfs 4.19.69-v8+ #1261 SMP PREEMPT Tue Sep 3 20:27:16 BST 2019 aarch64 GNU/Linux

Known issues:
vcsm driver doesn't work with 64-bit (only the cma version) which will break software decode with kodi

The 64 bit kernel shows up and remote access is working.
- Web browser starts and works
- Mathematica and Wolfram are not showing up

Code: Select all

pi@raspberrypi:/boot $ sudo rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** We're running for the first time
 *** Backing up files (this will take a few minutes)
 *** Remove old firmware backup
 *** Backing up firmware
 *** Remove old modules backup
 *** Backing up modules 4.19.66-v7l+
PARTSIZE:268435968
#############################################################
WARNING: 'rpi-update' updates to pre-releases of the linux
kernel tree and Videocore firmware.

'rpi-update' should only be used if there is a specific
reason to do so - for example, a request by a Raspberry Pi
engineer.

DO NOT use 'rpi-update' as part of a regular update process.

##############################################################
Would you like to proceed? (y/N)
 *** Downloading specific firmware revision (this will take a few minutes)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   168    0   168    0     0    800      0 --:--:-- --:--:-- --:--:--   800
100  113M  100  113M    0     0  1292k      0  0:01:29  0:01:29 --:--:-- 2859k
 *** Updating firmware
 *** Updating kernel modules
 *** depmod 4.19.69-v8+
 *** depmod 4.19.69-v7+
 *** depmod 4.19.69-v7l+
 *** depmod 4.19.69+
 *** Updating VideoCore libraries
 *** Using HardFP libraries
 *** Updating SDK
 *** Running ldconfig
 *** Storing current firmware revision
 *** Deleting downloaded files
 *** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to 1b797509b8c28cba7834684986b97feb1a265e9c
 *** A reboot is needed to activate the new firmware
pi@raspberrypi:/boot $

[DougieLawson]

For revision codes that say which processor you have, see here...https://www.raspberrypi.org/documentati ... /README.md

I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.

I still don't have a RPi4B 4GB because the household finance director doesn't leave for her three months in Australia until Oct 7th.

[Gavinmc42]

Just threw some free pascal opengl at it, works fine
Lazarus and Ultibo IDEs working.

I still don't have a RPi4B 4GB because the household finance director doesn't leave for her three months in Australia until Oct 7th.

Oh, thanks for the reminder, Pi4B4's are due in Sep here in Oz
Oh no, that first pre order batch must have sold out, next lot 14th Oct

Ok, now for the serious stuff, see if it works with the stuff I threw at Gentoo64.
Overnight compiles etc

[dom]

This doesn't work on a 3A+ with

Code: Select all

arm_64bit=1

but does work OK with

Code: Select all

kernel=kernel8.img

.

The requirement for kernel=kernel8.img was mentioned in first post.
I'm just building a firmware that will go out later that make that no longer necessary.

[dom]

I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.

You need BCM2837 (PI3/Pi3+) or BCM2838 (Pi4).
Later models of Pi2 switched to using BCM2837, but the majority sold will be BCM2836 (so no 64-bit capability)

The revision code is /proc/cpuinfo is the definitive information.

[DougieLawson]

I'd already got the revision codes. What I was trying to work out (aka remember) was which are 64-bit (kernel=kernel8.img) capable.

You need BCM2837 (PI3/Pi3+) or BCM2838 (Pi4).
Later models of Pi2 switched to using BCM2837, but the majority sold will be BCM2836 (so no 64-bit capability)

The revision code is /proc/cpuinfo is the definitive information.

Thank you. One 3A+ is running already, the 3Bs/3B+ are just waiting for the rpi-update to finish.

[dom]

The requirement for kernel=kernel8.img was mentioned in first post.
I'm just building a firmware that will go out later that make that no longer necessary.

Firmware pushed. arm_64bit=1 is all you need to enable 64-bit kernel on Pi3 too now.

[geev03]

....

Firmware pushed. arm_64bit=1 is all you need to enable 64-bit kernel on Pi3 too now.

Code: Select all

*** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to c472ab43407d3bf1ed162a9f7c3f5060e6f99d14
 *** A reboot is needed to activate the new firmware

OK, the same Pi4B sd card works on Pi3B, ( with the same problems with some of the programs that worked fine on 32 bit buster)

[dom]

- Mathematica and Wolfram are not showing up

If you are daring, edit /opt/Wolfram/WolframEngine/12.0/Executables/mathematica
find lines:

Code: Select all

				armv?l)
					SystemIDList="Linux-ARM";;

and add after:

Code: Select all

				aarch64)
					SystemIDList="Linux-ARM";;

You can also make a comparable change to /opt/Wolfram/WolframEngine/12.0/Executables/wolfram

[geev03]

..

If you are daring, edit /opt/Wolfram/WolframEngine/12.0/Executables/mathematica
find lines:

Code: Select all

				armv?l)
					SystemIDList="Linux-ARM";;

and add after:

Code: Select all

				aarch64)
					SystemIDList="Linux-ARM";;

You can also make a comparable change to /opt/Wolfram/WolframEngine/12.0/Executables/wolfram

Thanks a lot, perfect.

Code: Select all

pi@raspberrypi:~ $ sudo wolfram
Mathematica 12.0.1 Kernel for Linux ARM (32-bit)
Copyright 1988-2019 Wolfram Research, Inc.

In[1]:=

Mathematica is also working as expected, again many thanks.

[DougieLawson]

Interestingly running rpi-update in a chroot on a V7 system gets

Code: Select all

You appear to be using a custom kernel file.
Skipping installation of new kernel, as bundled dtb files may be incompatible with your kernel.

and blows up doing nothing useful.

I've sharpened my hacking pencil.

[jdonald]

Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).

Got it. In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?

[trejan]

In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?

The Pi 4 uses the xhci driver for the 4 main USB ports as it is a USB 3 controller connected via PCIe.

[6by9]

Dwc_otg is only used on the Pi3 (except for the device mode via USB c connector on 4).

Got it. In that case is USB performance on the Pi 4 already slower than that of the Pi 3B+? Or did fast interrupt handling get implemented for dwc2 at some point?

The Pi4 has a VLI, PCI-e based USB interface, which supports (amongst other things) USB3. USB3 will be significantly faster than the USB2 interface available via dwc_otg or dwc2.

The old dwc interface is still present on the SoC and is presented via the USB-C (power) connector as an option for supporting device mode (the VLI chip doesn't support that at all), but it is not intended to be in use on 99.99% of systems which only need USB host mode.

[dom]

Interestingly running rpi-update in a chroot on a V7 system gets

Code: Select all

You appear to be using a custom kernel file.
Skipping installation of new kernel, as bundled dtb files may be incompatible with your kernel.

Do you still have "kernel=xxx" set? That is what triggers this. It shouldn't be required now.

[180lifer]

Since Raspbian is being converted to 64-bit (woohoo!!) does that mean Raspbian Desktop for PCs will become 64-bit as well? I have had trouble getting it to run in a virtual machine using VMware Workstation Player because (if I remember right) the 32-bit binaries confused the VM since the underlying Debian OS was 64-bit.

Also, I sincerely hope that PiServer works with the Pi 4 in the not too distant future. PiServer really has a lot of promise and I’d love to see how the fast Pi 4 does with it.