Moving Linux Kernel to 5.10

[dom]

Edit: 5.10 kernel is now available through apt. rpi-update is not required to get the stable 5.10 kernel.


We are planning to move to the 5.10 kernel in the near future (perhaps a few months).

One advantage is there is more raspberry pi specific support in the newer upstream kernels which reduces the number of downstream raspberry pi patches and so reduces maintenance effort.

The KMS driver (arm side driver for hardware video scaler and displays like hdmi) is being developed upstream and has a number of fixes and new features compared to 5.4, and backporting them is not trivial.

The kms driver also includes a new arm side hdmi alsa driver. See this post to enable.

Obviously a newer kernel will bring additional kernel features and support for more devices (e.g. USB devices like wifi and dvb).
You can read up on what appeared in 5.5, 5.6, 5.7, 5.8, 5.9 and 5.10.

We will do this in a few stages. Starting now, we are updating the "next" firmware branch to the 5.10 kernel.
Assuming testing is positive, we'll update the master rpi-update firmware branch.
Eventually the 5.10 kernel will appear with "apt upgrade" and on new RPiOS images in the future.

Update with:

Code: Select all

sudo rpi-update

and reboot. Ideally nothing much will change. "uname -a" will report 5.10
Please report if there are any regressions compared to the 5.4 kernel.

If you are feeling especially bold, you can try the kms driver. Rather than dtoverlay=vc4-fkms-v3d, switch to dtoverlay=vc4-kms-v3d (not recommended on Pi0/1).

Note: kms won’t ever support applications that use firmware apis for display like dispmanx, or mmal/openmax video_render, including omxplayer, raspivid and non-gbm kodi. Similarly tvservice is not useful in this environment.

Obviously this release is for testers who like to live on the bleeding edge. It's not intended for beginners, or users who need stability.
Backing up is always advisable. There may be regressions. There may be improvements. But we won't know without testers trying it and reporting back.


EDIT: The next upstream LTS (Long Term Support) kernel is 5.10, and we have now switched to testing this.
EDIT: master branch of rpi-update now gets 5.10 kerne. next branch is now dormant


Known Issues:
With kms driver, sometimes HDMI experiences a 1 pixel shift (rightmost pixels appear on leftmost side of display).
USB bootloader booting results in non-working USB when booted.

[davidcoton]

Updated over VNC. Rebooted (sudo not required). Pi came back up with new kernel, VNC reconnected. Restarted my CPU-intensive process, work resumed.

All good, excellent start. Will change fkms to kms later, after initial stability test.
EDIT: Using RPiOS-64 on Pi4B 2GB.

[DougieLawson]

Obviously this release is for testers who like to live on the bleeding edge. It's not intended for beginners, or users who need stability.
Backing up is always advisable. There may be regressions. There may be improvements. But we won't know without testers trying it and reporting back.

Thank you. It's installing on a 3A+ right now.

[gtechn]

Compiled it with the exFAT module enabled for the Pi 4 2GB. It's running a Plex Server headless with an external exFAT SSD attached for the media files. I know this isn't much surface area being tested, but (as of writing) it's the very latest on the rpi-5.9.y branch (commit ae39d1c) and is working perfectly.

[pasman1]

Good news.

Code: Select all

sudo BRANCH=next rpi-update

this is for 32 and 64 bit RaspbOs?

Regards.

[davidcoton]

Good news.

Code: Select all

sudo BRANCH=next rpi-update

this is for 32 and 64 bit RaspbOs?

Regards.

It includes four versions, covering all Pi versions (0-4) and both 32- and 64-bit flavours.

[DougieLawson]

Code: Select all

sudo BRANCH=next rpi-update

this is for 32 and 64 bit RaspbOs?

Yep. It installs 5.9.0+, 5.9.0-v7+, 5.9.0-v7l+ and 5.9.0-v8+.

My 64-bit raspberries are running 32-bit userland with a 64-bit kernel. I'm not testing the pure 64-bit stuff yet.

[pasman1]

Ok. Installed.

Code: Select all

pi@raspberrypi:~ $ sudo BRANCH=next rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Your firmware is already up to date
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.4.51-v8+ #1333 SMP PREEMPT Mon Aug 10 16:58:35 BST 2020 aarch64 GNU/Linux

[trejan]

Existing swap files are broken on new kernel with a "swapon: swapfile has holes" message. It needs a kernel patch or the swap file to be recreated using dd.

https://github.com/raspberrypi/linux/issues/3902

Fixed by https://github.com/raspberrypi/linux/co ... d2c9650449. Need to wait for next rpi-update kernel build for it to appear in the pre-compiled kernel.

[RonR]

Ok. Installed.

Code: Select all

pi@raspberrypi:~ $ sudo BRANCH=next rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Your firmware is already up to date
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.4.51-v8+ #1333 SMP PREEMPT Mon Aug 10 16:58:35 BST 2020 aarch64 GNU/Linux

Should be showing:

Code: Select all

root@raspberrypi:~# uname -a
Linux raspberrypi 5.9.0-v8+ #1351 SMP PREEMPT Wed Oct 14 12:50:37 BST 2020 aarch64 GNU/Linux

[kerry_s]

no hdmi sound with kms, just analog(headphones/3.5 jack).

other than that, seems to be pretty good. i returned to fkms for more testing.

[CaptainMidnight]

As a test, upgraded one of the Pi4B2 PiOS Lite ntp servers, these are running 64-bit kernel, 32-bit userland. Unit is configured to run at fixed frequencies, arm 1.5Ghz and core 500Mhz with generic heatsink+fan.

Initial observations now running on 5.9.0-V8+

- ntpd 4.2.8p12@1.3728-o (1) running as expected
- kernel ntp pps functionality performing as expected
- dmesg reported network link up @6.4s during boot
- base temperature 27-29°c

Pi4 NTP servers appear good to go with base build and Uputronics GPS HAT (2019 version without RTC).

[dom]

no hdmi sound with kms, just analog(headphones/3.5 jack).

Did you remove dtparam=audio=on from config.txt?

[pasman1]

Should be showing:

Code: Select all

root@raspberrypi:~# uname -a
Linux raspberrypi 5.9.0-v8+ #1351 SMP PREEMPT Wed Oct 14 12:50:37 BST 2020 aarch64 GNU/Linux

looks like multiboot do not support kernel 5.9

regards.

[kerry_s]

no hdmi sound with kms, just analog(headphones/3.5 jack).

Did you remove dtparam=audio=on from config.txt?

nope, didn't know i needed to.

[dom]

Did you remove dtparam=audio=on from config.txt?

nope, didn't know i needed to.

Mentioned in the link about the arm side hdmi alsa in the first post.

[kerry_s]

Did you remove dtparam=audio=on from config.txt?

nope, didn't know i needed to.

Mentioned in the link about the arm side hdmi alsa in the first post.

i'll check it out.

[noee]

Pi4 4GB, running pihole, jriver mediacenter server, vnc and occasionally used as a desktop plugged in to a HDTV.

No issues so far with these basics. I tried KMS and it now boots with a functioning desktop and everything basic seems to be working, but no video playback with MPV (I understand why VLC does not work with KMS).

HDMI audio working with "new" VC4_hdmi as well.

Thx.

[gtechn]

I downloaded a fresh copy of Raspbian, ran 'rpi-update' to get onto the 5.9 branch, switched from FKMS to KMS, and enabled the V8 kernel.

When I finally got to the desktop (I'm assuming this is a new KMS glitch), I noticed that the leftmost 1px column of the desktop on my 1920x1080 monitor was messed up and not matching the wallpaper or the menu. That's the only glitch I've found so far.

https://pasteboard.co/JvM23Yx.jpg

[dom]

When I finally got to the desktop (I'm assuming this is a new KMS glitch), I noticed that the leftmost 1px column of the desktop on my 1920x1080 monitor was messed up and not matching the wallpaper or the menu. That's the only glitch I've found so far.

Yes, on some hdmi mode changes you get a shift by one pixel so the rightmost pixel incorrectly appears on the left side of display.
We had this with firmware driver around launch time and it was tricky to solve (but has been).
The same issue is currently affecting kms. I'll add it to known issues in first post.

Reboot and you may not get the issue, but there is a chance of it appearing on a HDMI mode change.
Once you have it, it remains after subsequent mode changes.

[gtechn]

Actually, I just discovered another, completely different glitch that might negate the pixel shift.

I ran 'uname -a' after I posted about the pixel shift, and noticed that it said I was on... 5.4? I ran 'sudo BRANCH=next rpi-update', and it said I was up-to-date.

Later reading the messages, it finally dawned on me what had happened. When I installed Raspbian, I set config.txt immediately to load the V8 5.4 kernel from the start rather than loading the V7L 5.4 kernel. When I ran `rpi-update` for the next branch, it failed to recognize the old V8 5.4 kernel, and saying that the DTBs might be incompatible because the V8 5.4 was a "custom kernel," it actually skipped installing the 5.9 kernel but left the `.firmware_revision` file behind saying it was 5.9. So, when it then "finished" the install, it said to reboot to jump to 5.9 without actually installing 5.9. And, when I rebooted, I ran rpi-update again and it said I was up-to-date.

You might want to add that to the caveats. I deleted the `.firmware_revision` file and set my config.txt to v7l on 5.4. Of course, `rpi-update` recognized that kernel as old and from there, it _appears_ to be downloading and installing 5.9 correctly. I'll keep you posted on whether the same pixel shift happens once I'm actually on 5.9.

EDIT: I am now on 5.9 V8 after the confusing update problem. I have not noticed a pixel shift yet, but will update this if it occurs.

[hippy]

It seemed to go okay, but there's something definitely broken for me ...

Before

Code: Select all

pi@Pi4B:~/apps/sol $ ./vcc fibo.c -p
VCC Reading : fibo.c
VCC Writing : fibo.c.zpu
VCC Writing : fibo.c.zpu.pi
EXE Writing : fibo.c.zpu.pi.exe
pi@Pi4B:~/apps/sol $ ./fibo.c.zpu.pi.exe 
Fibo(24) = 46368

After

Code: Select all

pi@Pi4B:~/apps/sol $ ./vcc fibo -p
VCC Reading : fibo.c
VCC Writing : fibo.c.zpu
VCC Writing : fibo.c.zpu.pi
EXE Writing : fibo.c.zpu.pi.exe
pi@Pi4B:~/apps/sol $ ./fibo.c.zpu.pi.exe 
Segmentation fault

That's running a Python program which takes the textual assembly language output of one GCC compiler, converts it to ARM assembler and uses ARM GCC tools to turn it into an executable. Longhand ...

Code: Select all

pi@Pi4B:~/apps/sol $ as -o xyzzy.o fibo.c.zpu.pi
pi@Pi4B:~/apps/sol $ gcc -o xyzzy xyzzy.o
pi@Pi4B:~/apps/sol $ ./xyzzy 
Segmentation fault

or

Code: Select all

pi@Pi4B:~/apps/sol $ gcc -o xyzzy_gcc -x assembler fibo.c.zpu.pi
pi@Pi4B:~/apps/sol $ ./xyzzy_gcc
Segmentation fault

I guess that whatever my ARM assembly language code was doing no longer works on the new 5.9 kernel. Not sure where to start but I guess it's time to start digging.


Update : I am using 32-bit userland, and my code fails with both the 32-bit and 64-bit 5.9 kernel ( "arm_64bit=1" in config.txt ) -

Code: Select all

Linux Pi4B 5.9.0-v7l+ #1351 SMP Wed Oct 14 12:48:16 BST 2020 armv7l GNU/Linux

Code: Select all

Linux Pi4B 5.9.0-v8+ #1351 SMP PREEMPT Wed Oct 14 12:50:37 BST 2020 aarch64 GNU/Linux

Reverted that same SD Card back to default and re-booted, and, without any other changes, my code double-checks as fine, runs without segmentation fault, with both 32-bit and 64-bit kernels -

Code: Select all

Linux Pi4B 5.4.51-v7l+ #1333 SMP Mon Aug 10 16:51:40 BST 2020 armv7l GNU/Linux

Code: Select all

Linux Pi4B 5.4.51-v8+ #1333 SMP PREEMPT Mon Aug 10 16:58:35 BST 2020 aarch64 GNU/Linux

Just for completeness I am running a fully updated Raspberry Pi OS Desktop on a Pi 4B 1GB.

Yay! Another Circle of Hell.

[trejan]

When I installed Raspbian, I set config.txt immediately to load the V8 5.4 kernel from the start rather than loading the V7L 5.4 kernel. When I ran `rpi-update` for the next branch, it failed to recognize the old V8 5.4 kernel, and saying that the DTBs might be incompatible because the V8 5.4 was a "custom kernel,"

The rpi-update weirdness is because you're using the kernel= option in /boot/config.txt. rpi-update isn't checking the kernel file contents. It is looking for the kernel= option and assuming you're doing something unusual.

If you're using Raspberry Pi OS/Raspbian then you don't need to set kernel= as it should be all automatic. If you want the 64-bit kernel then you set arm_64bit=1 and it will automatically load kernel8.img.

[dom]

It seemed to go okay, but there's something definitely broken for me ...

Can you run gdb on the executable that seg-faults and identify the instructions being used?
It could be the code is always doing something bad (e.g. dereferencing an initialised pointer)
and that, by chance, just happens to be unluckier with 5.9 kernel.

[hippy]

It seemed to go okay, but there's something definitely broken for me ...

Can you run gdb on the executable that seg-faults and identify the instructions being used?
It could be the code is always doing something bad (e.g. dereferencing an initialised pointer)
and that, by chance, just happens to be unluckier with 5.9 kernel.

I'm not familiar with 'gdb' but I did run it to see if that would identify anything. I simply 'r' ran it I think - I was guessing at commands - and it reported something like a SEGsomething which referenced what seems to be the first 'bl' branch to subroutine in my code.

I'll have to upgrade to 5.9 again to check further but I've attached my source code -

Code: Select all

wget https://www.raspberrypi.org/forums/download/file.php?id=40623
unzip 'file.php?id=40623'
gcc -o hippy -g hippy.s
./hippy

With 5.9 ...

Code: Select all

pi@Pi4B:~/apps/sol/tmp $ gdb hippy
GNU gdb (Raspbian 8.2.1-2) 8.2.1
... snip ...
Reading symbols from hippy...done.
(gdb) r
Starting program: /home/pi/apps/sol/tmp/hippy 

Program received signal SIGSEGV, Segmentation fault.
0x0002102c in main ()
(gdb) disassemble 0x0002102c
Dump of assembler code for function main:
=> 0x0002102c:  bl      0x211e8
   0x00021030:  b       0x21144
End of assembler dump.
(gdb) 

hippy.zip

(3.34 KiB) Downloaded 137 times