Faussaire
Posts: 3
Joined: Fri Sep 03, 2021 11:45 am

Devices not detected with rt kernel

Tue Sep 14, 2021 1:34 pm

Hi,

I installed Rapsberry Pi OS lite using the rpi-imager. I updated and upgraded everything with apt.

https://www.raspberrypi.org/documentati ... ernel.html

Then, I installed the kernel version of the rpi-kernel with realtime capabilities (see below) using the kernel guide (see above).

https://github.com/raspberrypi/linux/tree/rpi-4.19.y-rt

After installing the kernel and rebooting the raspberry, it seems like the rasp stopped being able to detect usb devices.

The command lsusb only show one entry : Bus 001 Device 001: ID 1d6b:0002 Linux foundation 2.0 root hub.
Before installing the Kernel, other plugged devices would show up.

Using udevadm monitor results in no events when plugging/unplugging a device.
Without the rt kernel, some events would show up when plugging/unplugging.

Is this a known issue with the kernel rpi-4.19.y-rt ?
Any ideas about the cause of this problem ?

ncx
Posts: 9
Joined: Thu Sep 09, 2021 8:17 pm
Location: Texas
Contact: Website

Re: Devices not detected with rt kernel

Tue Sep 14, 2021 8:17 pm

That is quite an old kernel version. Current Raspberry Pi OS runs on Linux kernel v5.10.y. You likely need to merge -rt on your own if there isn't a branch already available.
"Optimized" Raspberry Pi(4) Kernel: https://github.com/notcarbide/linux

Faussaire
Posts: 3
Joined: Fri Sep 03, 2021 11:45 am

Re: Devices not detected with rt kernel

Wed Sep 15, 2021 8:27 am

Thanks for the insight, I'll be looking into that.

V10lator
Posts: 39
Joined: Wed Jul 07, 2021 11:18 am

Re: Devices not detected with rt kernel

Wed Sep 15, 2021 8:45 am

Here are the realtime patches: http://cdn.kernel.org/pub/linux/kernel/ ... s/rt/5.10/
Sadly there's currently none for 5.10.63 and while the one for 5.10.59 applies and compiles on top of 5.10.63 it crashes while booting. So you will have to either downgrade the PI kernel to 5.10.59 (not sure if possible) or wait untill the patches get updated. Also you need another patch called "0001-usb-dwc_otg-fix-system-lockup.patch".

ncx
Posts: 9
Joined: Thu Sep 09, 2021 8:17 pm
Location: Texas
Contact: Website

Re: Devices not detected with rt kernel

Wed Sep 15, 2021 1:25 pm

V10lator wrote: Here are the realtime patches: http://cdn.kernel.org/pub/linux/kernel/ ... s/rt/5.10/
Sadly there's currently none for 5.10.63 and while the one for 5.10.59 applies and compiles on top of 5.10.63 it crashes while booting. So you will have to either downgrade the PI kernel to 5.10.59 (not sure if possible) or wait untill the patches get updated. Also you need another patch called "0001-usb-dwc_otg-fix-system-lockup.patch".
It's possible.

The latest tag is on v5.10.60.
https://github.com/raspberrypi/linux/tree/1.20210831

You could checkout to that, revert the last Linux v5.10.y merge, fetch kernel.org sources and then merge tag v5.10.59 from there.
"Optimized" Raspberry Pi(4) Kernel: https://github.com/notcarbide/linux

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29574
Joined: Sat Jul 30, 2011 7:41 pm

Re: Devices not detected with rt kernel

Wed Sep 15, 2021 1:57 pm

Make sure you have also copied across the correct modules - missing USB is a symptoms of changing a kernel img file, but not the associated modules.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Applications Team.

V10lator
Posts: 39
Joined: Wed Jul 07, 2021 11:18 am

Re: Devices not detected with rt kernel

Wed Sep 15, 2021 2:40 pm

ncx wrote:
Wed Sep 15, 2021 1:25 pm
You could checkout to that, revert the last Linux v5.10.y merge, fetch kernel.org sources and then merge tag v5.10.59 from there.
I decided to git checkout the commit right before the 5.10.60 merge instead but it also doesn't boot my headless machine far enough to SSH in. Now I didn't bother debugging this any more but my guess is that the 0001-usb-dwc_otg-fix-system-lockup.patch needs an update (cause at least with 5.10.63 dwc-otg was mentioned in the crashlog).

ncx
Posts: 9
Joined: Thu Sep 09, 2021 8:17 pm
Location: Texas
Contact: Website

Re: Devices not detected with rt kernel

Wed Sep 15, 2021 4:50 pm

V10lator wrote:
Wed Sep 15, 2021 2:40 pm
I decided to git checkout the commit right before the 5.10.60 merge instead but it also doesn't boot my headless machine far enough to SSH in. Now I didn't bother debugging this any more but my guess is that the 0001-usb-dwc_otg-fix-system-lockup.patch needs an update (cause at least with 5.10.63 dwc-otg was mentioned in the crashlog).
That IS a quicker way to test it lol
I only kinda said it to keep the rest of the pi-specific changes.

If that's the case, it doesn't seem too difficult to update if it's like this:
https://github.com/raspberrypi/linux/co ... 47f2963557
"Optimized" Raspberry Pi(4) Kernel: https://github.com/notcarbide/linux

V10lator
Posts: 39
Joined: Wed Jul 07, 2021 11:18 am

Re: Devices not detected with rt kernel

Wed Sep 15, 2021 9:31 pm

ncx wrote:
Wed Sep 15, 2021 4:50 pm
If that's the case, it doesn't seem too difficult to update if it's like this:
https://github.com/raspberrypi/linux/co ... 47f2963557
Thanks for the heads up. Now I had a look at https://github.com/raspberrypi/linux/co ... st/dwc_otg and sadly it seems the driver didn't change since I was able to use RT patches (was also 5.10.y but I can't remember what version exactly. Sad thing I'm compiling the kernel directly on the RPI and reformatted/reflashed the SD card in between with just the kernel config backuped), so the problem must be somewhere else. Anyway, I don't have the time to debug this right now and an RT kernel also isn't super important for my project as it is (but would be a great addition), so hopefull somebody other needing RT will step in and help us. :)

//EDIT: To the mods: Thanks for deleting the accidential double-post from me but next time keep an eye at what post I reported exactly as you removed the wrong one. No big thing through, just had to edit this post again so it reflects the one I wanted to post.

V10lator
Posts: 39
Joined: Wed Jul 07, 2021 11:18 am

Re: Devices not detected with rt kernel

Sat Sep 25, 2021 11:16 am

I tried something but am stuck. What I tried is
  • Grabbed the RPI kernel sources.
  • Updated them to 5.10.65.
  • Compiled and tested - Worked.
  • Applied patch-5.10.65-rt53.patch and 0001-usb-dwc_otg-fix-system-lockup.patch.
  • Compiled and tested - Crashed:
    Image
    Image
Here are my patched kernel sources: https://github.com/V10lator/rpi-rt-linu ... 10.65-rt53

LTolledo
Posts: 5950
Joined: Sat Mar 17, 2018 7:29 am
Location: Anime Heartland

Re: Devices not detected with rt kernel

Sat Sep 25, 2021 11:45 am

why the need for a RT kernel for your setup?

what specific use case that your setup requires RT kernel?
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

V10lator
Posts: 39
Joined: Wed Jul 07, 2021 11:18 am

Re: Devices not detected with rt kernel

Sat Sep 25, 2021 12:24 pm

V10lator wrote:
Sat Sep 25, 2021 11:16 am
Here are my patched kernel sources: https://github.com/V10lator/rpi-rt-linu ... 10.65-rt53
As a workaround I was able to boot the RT kernel with

Code: Select all

dwc_otg.fiq_enable=0 dwc_otg.fiq_fsm_enable=0
added to /boot/cmdline.txt

ncx
Posts: 9
Joined: Thu Sep 09, 2021 8:17 pm
Location: Texas
Contact: Website

Re: Devices not detected with rt kernel

Mon Sep 27, 2021 3:08 am

V10lator wrote:
Sat Sep 25, 2021 12:24 pm
V10lator wrote:
Sat Sep 25, 2021 11:16 am
Here are my patched kernel sources: https://github.com/V10lator/rpi-rt-linu ... 10.65-rt53
As a workaround I was able to boot the RT kernel with

Code: Select all

dwc_otg.fiq_enable=0 dwc_otg.fiq_fsm_enable=0
added to /boot/cmdline.txt
Interesting. Maybe patching like this article will allow you to boot without disabling fiq:
https://www.osadl.org/Single-View.111+M ... c57.0.html
"Optimized" Raspberry Pi(4) Kernel: https://github.com/notcarbide/linux

V10lator
Posts: 39
Joined: Wed Jul 07, 2021 11:18 am

Re: Devices not detected with rt kernel

Mon Sep 27, 2021 8:20 am

ncx wrote:
Mon Sep 27, 2021 3:08 am
Maybe patching like this article will allow you to boot without disabling fiq
Thanks but that's what the 0001-usb-dwc_otg-fix-system-lockup.patch does: https://github.com/V10lator/rpi-rt-linu ... cd05ed0093

In case I'm interpreting the crashlog correctly one of the macros introduced by the patch (fiq_fsm_spin_lock_irqsave) is crashing at the line local_fiq_disable(); - note that fiq_fsm_spin_lock_irqsave is a preprocessor macro, not a function, so it is not visible in the stacktrace: https://github.com/V10lator/rpi-rt-linu ... #L399-L404
Image

So for me it seems like either something around the driver (maybe IRQ handling?) or the RT patches changed, breaking the old solution.

Return to “Advanced users”