dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6043
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Moving Linux Kernel to 5.10

Wed Oct 14, 2020 3:10 pm

Edit: 5.10 kernel is now available through apt. rpi-update is not required to get the stable 5.10 kernel.


We are planning to move to the 5.10 kernel in the near future (perhaps a few months).

One advantage is there is more raspberry pi specific support in the newer upstream kernels which reduces the number of downstream raspberry pi patches and so reduces maintenance effort.

The KMS driver (arm side driver for hardware video scaler and displays like hdmi) is being developed upstream and has a number of fixes and new features compared to 5.4, and backporting them is not trivial.

The kms driver also includes a new arm side hdmi alsa driver. See this post to enable.

Obviously a newer kernel will bring additional kernel features and support for more devices (e.g. USB devices like wifi and dvb).
You can read up on what appeared in 5.5, 5.6, 5.7, 5.8, 5.9 and 5.10.

We will do this in a few stages. Starting now, we are updating the "next" firmware branch to the 5.10 kernel.
Assuming testing is positive, we'll update the master rpi-update firmware branch.
Eventually the 5.10 kernel will appear with "apt upgrade" and on new RPiOS images in the future.

Update with:

Code: Select all

sudo rpi-update
and reboot. Ideally nothing much will change. "uname -a" will report 5.10
Please report if there are any regressions compared to the 5.4 kernel.

If you are feeling especially bold, you can try the kms driver. Rather than dtoverlay=vc4-fkms-v3d, switch to dtoverlay=vc4-kms-v3d (not recommended on Pi0/1).

Note: kms won’t ever support applications that use firmware apis for display like dispmanx, or mmal/openmax video_render, including omxplayer, raspivid and non-gbm kodi. Similarly tvservice is not useful in this environment.

Obviously this release is for testers who like to live on the bleeding edge. It's not intended for beginners, or users who need stability.
Backing up is always advisable. There may be regressions. There may be improvements. But we won't know without testers trying it and reporting back.


EDIT: The next upstream LTS (Long Term Support) kernel is 5.10, and we have now switched to testing this.
EDIT: master branch of rpi-update now gets 5.10 kerne. next branch is now dormant


Known Issues:
With kms driver, sometimes HDMI experiences a 1 pixel shift (rightmost pixels appear on leftmost side of display).
USB bootloader booting results in non-working USB when booted.

User avatar
davidcoton
Posts: 6747
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 3:25 pm

Updated over VNC. Rebooted (sudo not required). Pi came back up with new kernel, VNC reconnected. Restarted my CPU-intensive process, work resumed.

All good, excellent start. Will change fkms to kms later, after initial stability test.
EDIT: Using RPiOS-64 on Pi4B 2GB.
Last edited by davidcoton on Wed Oct 14, 2020 3:51 pm, edited 1 time in total.
Location: 345th cell on the right of the 210th row of L2 cache

User avatar
DougieLawson
Posts: 42157
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 3:49 pm

dom wrote:
Wed Oct 14, 2020 3:10 pm
Obviously this release is for testers who like to live on the bleeding edge. It's not intended for beginners, or users who need stability.
Backing up is always advisable. There may be regressions. There may be improvements. But we won't know without testers trying it and reporting back.
Thank you. It's installing on a 3A+ right now.
Languages using left-hand whitespace for syntax are ridiculous

DMs sent on https://twitter.com/DougieLawson or LinkedIn will be answered next month.
Fake doctors - are all on my foes list.

The use of crystal balls and mind reading is prohibited.

gtechn
Posts: 161
Joined: Thu Jan 07, 2016 5:32 pm

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 4:47 pm

Compiled it with the exFAT module enabled for the Pi 4 2GB. It's running a Plex Server headless with an external exFAT SSD attached for the media files. I know this isn't much surface area being tested, but (as of writing) it's the very latest on the rpi-5.9.y branch (commit ae39d1c) and is working perfectly.

User avatar
pasman1
Posts: 238
Joined: Mon Aug 10, 2020 3:01 pm
Location: Poland

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 5:36 pm

Good news.

Code: Select all

sudo BRANCH=next rpi-update
this is for 32 and 64 bit RaspbOs?

Regards.
Pi4-4G RaspiOS 32 BuLlSeYe

User avatar
davidcoton
Posts: 6747
Joined: Mon Sep 01, 2014 2:37 pm
Location: Cambridge, UK

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 5:50 pm

pasman1 wrote:
Wed Oct 14, 2020 5:36 pm
Good news.

Code: Select all

sudo BRANCH=next rpi-update
this is for 32 and 64 bit RaspbOs?

Regards.
It includes four versions, covering all Pi versions (0-4) and both 32- and 64-bit flavours.
Location: 345th cell on the right of the 210th row of L2 cache

User avatar
DougieLawson
Posts: 42157
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 5:58 pm

pasman1 wrote:
Wed Oct 14, 2020 5:36 pm

Code: Select all

sudo BRANCH=next rpi-update
this is for 32 and 64 bit RaspbOs?
Yep. It installs 5.9.0+, 5.9.0-v7+, 5.9.0-v7l+ and 5.9.0-v8+.

My 64-bit raspberries are running 32-bit userland with a 64-bit kernel. I'm not testing the pure 64-bit stuff yet.
Languages using left-hand whitespace for syntax are ridiculous

DMs sent on https://twitter.com/DougieLawson or LinkedIn will be answered next month.
Fake doctors - are all on my foes list.

The use of crystal balls and mind reading is prohibited.

User avatar
pasman1
Posts: 238
Joined: Mon Aug 10, 2020 3:01 pm
Location: Poland

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 6:27 pm

Ok. Installed.

Code: Select all

pi@raspberrypi:~ $ sudo BRANCH=next rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Your firmware is already up to date
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.4.51-v8+ #1333 SMP PREEMPT Mon Aug 10 16:58:35 BST 2020 aarch64 GNU/Linux
Pi4-4G RaspiOS 32 BuLlSeYe

trejan
Posts: 4462
Joined: Tue Jul 02, 2019 2:28 pm

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 7:03 pm

Existing swap files are broken on new kernel with a "swapon: swapfile has holes" message. It needs a kernel patch or the swap file to be recreated using dd.

https://github.com/raspberrypi/linux/issues/3902


Fixed by https://github.com/raspberrypi/linux/co ... d2c9650449. Need to wait for next rpi-update kernel build for it to appear in the pre-compiled kernel.
Last edited by trejan on Wed Oct 14, 2020 11:38 pm, edited 1 time in total.

RonR
Posts: 2770
Joined: Tue Apr 12, 2016 10:29 pm
Location: US

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 8:03 pm

pasman1 wrote:
Wed Oct 14, 2020 6:27 pm
Ok. Installed.

Code: Select all

pi@raspberrypi:~ $ sudo BRANCH=next rpi-update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Performing self-update
 *** Relaunching after update
 *** Raspberry Pi firmware updater by Hexxeh, enhanced by AndrewS and Dom
 *** Your firmware is already up to date
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.4.51-v8+ #1333 SMP PREEMPT Mon Aug 10 16:58:35 BST 2020 aarch64 GNU/Linux

Should be showing:

Code: Select all

root@raspberrypi:~# uname -a
Linux raspberrypi 5.9.0-v8+ #1351 SMP PREEMPT Wed Oct 14 12:50:37 BST 2020 aarch64 GNU/Linux

User avatar
kerry_s
Posts: 3764
Joined: Thu Jan 30, 2020 7:14 pm

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 9:24 pm

no hdmi sound with kms, just analog(headphones/3.5 jack).

other than that, seems to be pretty good. i returned to fkms for more testing.
Attachments
2020-10-14-112341_1680x1050_scrot.png
2020-10-14-112341_1680x1050_scrot.png (125.81 KiB) Viewed 43021 times

User avatar
CaptainMidnight
Posts: 315
Joined: Sun Nov 03, 2019 4:32 pm
Location: UK

Re: Moving Linux Kernel to 5.9

Wed Oct 14, 2020 10:02 pm

As a test, upgraded one of the Pi4B2 PiOS Lite ntp servers, these are running 64-bit kernel, 32-bit userland. Unit is configured to run at fixed frequencies, arm 1.5Ghz and core 500Mhz with generic heatsink+fan.

Initial observations now running on 5.9.0-V8+

- ntpd 4.2.8p12@1.3728-o (1) running as expected
- kernel ntp pps functionality performing as expected
- dmesg reported network link up @6.4s during boot
- base temperature 27-29°c

Pi4 NTP servers appear good to go with base build and Uputronics GPS HAT (2019 version without RTC).
"Never get out of the boat."
Absolutely goddamn right!
Unless you were goin' all the way...

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6043
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 9:47 am

kerry_s wrote:
Wed Oct 14, 2020 9:24 pm
no hdmi sound with kms, just analog(headphones/3.5 jack).
Did you remove dtparam=audio=on from config.txt?

User avatar
pasman1
Posts: 238
Joined: Mon Aug 10, 2020 3:01 pm
Location: Poland

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 9:52 am

RonR wrote:
Wed Oct 14, 2020 8:03 pm

Should be showing:

Code: Select all

root@raspberrypi:~# uname -a
Linux raspberrypi 5.9.0-v8+ #1351 SMP PREEMPT Wed Oct 14 12:50:37 BST 2020 aarch64 GNU/Linux
looks like multiboot do not support kernel 5.9

regards.
Pi4-4G RaspiOS 32 BuLlSeYe

User avatar
kerry_s
Posts: 3764
Joined: Thu Jan 30, 2020 7:14 pm

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 10:26 am

dom wrote:
Thu Oct 15, 2020 9:47 am
kerry_s wrote:
Wed Oct 14, 2020 9:24 pm
no hdmi sound with kms, just analog(headphones/3.5 jack).
Did you remove dtparam=audio=on from config.txt?
nope, didn't know i needed to.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6043
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 10:28 am

kerry_s wrote:
Thu Oct 15, 2020 10:26 am
dom wrote:
Thu Oct 15, 2020 9:47 am
Did you remove dtparam=audio=on from config.txt?
nope, didn't know i needed to.
Mentioned in the link about the arm side hdmi alsa in the first post.

User avatar
kerry_s
Posts: 3764
Joined: Thu Jan 30, 2020 7:14 pm

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 10:35 am

dom wrote:
Thu Oct 15, 2020 10:28 am
kerry_s wrote:
Thu Oct 15, 2020 10:26 am
dom wrote:
Thu Oct 15, 2020 9:47 am
Did you remove dtparam=audio=on from config.txt?
nope, didn't know i needed to.
Mentioned in the link about the arm side hdmi alsa in the first post.
i'll check it out.

noee
Posts: 25
Joined: Thu Nov 21, 2019 5:30 pm

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 11:39 am

Pi4 4GB, running pihole, jriver mediacenter server, vnc and occasionally used as a desktop plugged in to a HDTV.

No issues so far with these basics. I tried KMS and it now boots with a functioning desktop and everything basic seems to be working, but no video playback with MPV (I understand why VLC does not work with KMS).

HDMI audio working with "new" VC4_hdmi as well.

Thx.

gtechn
Posts: 161
Joined: Thu Jan 07, 2016 5:32 pm

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 3:11 pm

I downloaded a fresh copy of Raspbian, ran 'rpi-update' to get onto the 5.9 branch, switched from FKMS to KMS, and enabled the V8 kernel.

When I finally got to the desktop (I'm assuming this is a new KMS glitch), I noticed that the leftmost 1px column of the desktop on my 1920x1080 monitor was messed up and not matching the wallpaper or the menu. That's the only glitch I've found so far.

https://pasteboard.co/JvM23Yx.jpg

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6043
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 3:15 pm

gtechn wrote:
Thu Oct 15, 2020 3:11 pm
When I finally got to the desktop (I'm assuming this is a new KMS glitch), I noticed that the leftmost 1px column of the desktop on my 1920x1080 monitor was messed up and not matching the wallpaper or the menu. That's the only glitch I've found so far.
Yes, on some hdmi mode changes you get a shift by one pixel so the rightmost pixel incorrectly appears on the left side of display.
We had this with firmware driver around launch time and it was tricky to solve (but has been).
The same issue is currently affecting kms. I'll add it to known issues in first post.

Reboot and you may not get the issue, but there is a chance of it appearing on a HDMI mode change.
Once you have it, it remains after subsequent mode changes.

gtechn
Posts: 161
Joined: Thu Jan 07, 2016 5:32 pm

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 3:26 pm

Actually, I just discovered another, completely different glitch that might negate the pixel shift.

I ran 'uname -a' after I posted about the pixel shift, and noticed that it said I was on... 5.4? I ran 'sudo BRANCH=next rpi-update', and it said I was up-to-date.

Later reading the messages, it finally dawned on me what had happened. When I installed Raspbian, I set config.txt immediately to load the V8 5.4 kernel from the start rather than loading the V7L 5.4 kernel. When I ran `rpi-update` for the next branch, it failed to recognize the old V8 5.4 kernel, and saying that the DTBs might be incompatible because the V8 5.4 was a "custom kernel," it actually skipped installing the 5.9 kernel but left the `.firmware_revision` file behind saying it was 5.9. So, when it then "finished" the install, it said to reboot to jump to 5.9 without actually installing 5.9. And, when I rebooted, I ran rpi-update again and it said I was up-to-date.

You might want to add that to the caveats. I deleted the `.firmware_revision` file and set my config.txt to v7l on 5.4. Of course, `rpi-update` recognized that kernel as old and from there, it _appears_ to be downloading and installing 5.9 correctly. I'll keep you posted on whether the same pixel shift happens once I'm actually on 5.9.

EDIT: I am now on 5.9 V8 after the confusing update problem. I have not noticed a pixel shift yet, but will update this if it occurs.
Last edited by gtechn on Thu Oct 15, 2020 4:12 pm, edited 1 time in total.

hippy
Posts: 11966
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 3:31 pm

It seemed to go okay, but there's something definitely broken for me ...

Before

Code: Select all

pi@Pi4B:~/apps/sol $ ./vcc fibo.c -p
VCC Reading : fibo.c
VCC Writing : fibo.c.zpu
VCC Writing : fibo.c.zpu.pi
EXE Writing : fibo.c.zpu.pi.exe
pi@Pi4B:~/apps/sol $ ./fibo.c.zpu.pi.exe 
Fibo(24) = 46368
After

Code: Select all

pi@Pi4B:~/apps/sol $ ./vcc fibo -p
VCC Reading : fibo.c
VCC Writing : fibo.c.zpu
VCC Writing : fibo.c.zpu.pi
EXE Writing : fibo.c.zpu.pi.exe
pi@Pi4B:~/apps/sol $ ./fibo.c.zpu.pi.exe 
Segmentation fault
That's running a Python program which takes the textual assembly language output of one GCC compiler, converts it to ARM assembler and uses ARM GCC tools to turn it into an executable. Longhand ...

Code: Select all

pi@Pi4B:~/apps/sol $ as -o xyzzy.o fibo.c.zpu.pi
pi@Pi4B:~/apps/sol $ gcc -o xyzzy xyzzy.o
pi@Pi4B:~/apps/sol $ ./xyzzy 
Segmentation fault
or

Code: Select all

pi@Pi4B:~/apps/sol $ gcc -o xyzzy_gcc -x assembler fibo.c.zpu.pi
pi@Pi4B:~/apps/sol $ ./xyzzy_gcc
Segmentation fault
I guess that whatever my ARM assembly language code was doing no longer works on the new 5.9 kernel. Not sure where to start but I guess it's time to start digging.


Update : I am using 32-bit userland, and my code fails with both the 32-bit and 64-bit 5.9 kernel ( "arm_64bit=1" in config.txt ) -

Code: Select all

Linux Pi4B 5.9.0-v7l+ #1351 SMP Wed Oct 14 12:48:16 BST 2020 armv7l GNU/Linux

Code: Select all

Linux Pi4B 5.9.0-v8+ #1351 SMP PREEMPT Wed Oct 14 12:50:37 BST 2020 aarch64 GNU/Linux
Reverted that same SD Card back to default and re-booted, and, without any other changes, my code double-checks as fine, runs without segmentation fault, with both 32-bit and 64-bit kernels -

Code: Select all

Linux Pi4B 5.4.51-v7l+ #1333 SMP Mon Aug 10 16:51:40 BST 2020 armv7l GNU/Linux

Code: Select all

Linux Pi4B 5.4.51-v8+ #1333 SMP PREEMPT Mon Aug 10 16:58:35 BST 2020 aarch64 GNU/Linux
Just for completeness I am running a fully updated Raspberry Pi OS Desktop on a Pi 4B 1GB.

Yay! Another Circle of Hell.

trejan
Posts: 4462
Joined: Tue Jul 02, 2019 2:28 pm

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 4:48 pm

gtechn wrote:
Thu Oct 15, 2020 3:26 pm
When I installed Raspbian, I set config.txt immediately to load the V8 5.4 kernel from the start rather than loading the V7L 5.4 kernel. When I ran `rpi-update` for the next branch, it failed to recognize the old V8 5.4 kernel, and saying that the DTBs might be incompatible because the V8 5.4 was a "custom kernel,"
The rpi-update weirdness is because you're using the kernel= option in /boot/config.txt. rpi-update isn't checking the kernel file contents. It is looking for the kernel= option and assuming you're doing something unusual.

If you're using Raspberry Pi OS/Raspbian then you don't need to set kernel= as it should be all automatic. If you want the 64-bit kernel then you set arm_64bit=1 and it will automatically load kernel8.img.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6043
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 4:55 pm

hippy wrote:
Thu Oct 15, 2020 3:31 pm
It seemed to go okay, but there's something definitely broken for me ...
Can you run gdb on the executable that seg-faults and identify the instructions being used?
It could be the code is always doing something bad (e.g. dereferencing an initialised pointer)
and that, by chance, just happens to be unluckier with 5.9 kernel.

hippy
Posts: 11966
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Moving Linux Kernel to 5.9

Thu Oct 15, 2020 5:47 pm

dom wrote:
Thu Oct 15, 2020 4:55 pm
hippy wrote:
Thu Oct 15, 2020 3:31 pm
It seemed to go okay, but there's something definitely broken for me ...
Can you run gdb on the executable that seg-faults and identify the instructions being used?
It could be the code is always doing something bad (e.g. dereferencing an initialised pointer)
and that, by chance, just happens to be unluckier with 5.9 kernel.
I'm not familiar with 'gdb' but I did run it to see if that would identify anything. I simply 'r' ran it I think - I was guessing at commands - and it reported something like a SEGsomething which referenced what seems to be the first 'bl' branch to subroutine in my code.

I'll have to upgrade to 5.9 again to check further but I've attached my source code -

Code: Select all

wget https://www.raspberrypi.org/forums/download/file.php?id=40623
unzip 'file.php?id=40623'
gcc -o hippy -g hippy.s
./hippy
With 5.9 ...

Code: Select all

pi@Pi4B:~/apps/sol/tmp $ gdb hippy
GNU gdb (Raspbian 8.2.1-2) 8.2.1
... snip ...
Reading symbols from hippy...done.
(gdb) r
Starting program: /home/pi/apps/sol/tmp/hippy 

Program received signal SIGSEGV, Segmentation fault.
0x0002102c in main ()
(gdb) disassemble 0x0002102c
Dump of assembler code for function main:
=> 0x0002102c:  bl      0x211e8
   0x00021030:  b       0x21144
End of assembler dump.
(gdb) 
hippy.zip
(3.34 KiB) Downloaded 137 times
Last edited by hippy on Thu Oct 15, 2020 6:19 pm, edited 3 times in total.

Return to “Advanced users”