CM4: I/O board USB unavailable in yocto, working on Pi 4B

[jovial_leavitt]

I have been developing a yocto product for some time, with many builds as I iterate on my configuration and learn the ropes. All of these builds have had USB work fine. I wouldn't have been able to get far without a keyboard after all. I did most of my development on a Pi 4B, and have switched to a CM4 and I/O board now that I was able to get one. I have been using the CM4 for a couple weeks now, but now find myself unable to make an image that will boot with USB support on the CM4. USB devices do not power up (I test by checking the capslock light on a usb keyboard). I can switch the same sd card from the CM4 to my Pi 4B and USB works.

I recorded logs over uart for both devices using the same sd card running poky/core-image-base. Here's the start of the diff where you can see bcrm-pcie doesn't load correctly (timestamps filtered out for ease of comparison)

Code: Select all

diff --git a/cm4-USB-filtered.log b/pi4b-uart-filtered.log
index 45b534b..851c1fc 100644
--- a/cm4-USB-filtered.log
+++ b/pi4b-uart-filtered.log
@@ -3,7 +3,7 @@
  CPU: ARMv7 Processor [410fd083] revision 3 (ARMv7), cr=30c5383d
  CPU: div instructions available: patching division code
  CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
- OF: fdt: Machine model: Raspberry Pi Compute Module 4 Rev 1.0
+ OF: fdt: Machine model: Raspberry Pi 4 Model B Rev 1.1
  random: fast init done
  Memory policy: Data cache writealloc
  Reserved memory: created CMA memory pool at 0x000000001ac00000, size 320 MiB
@@ -19,7 +19,7 @@
  Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
  percpu: Embedded 20 pages/cpu s51404 r8192 d22324 u81920
  Built 1 zonelists, mobility grouping on.  Total pages: 1011008
- Kernel command line: coherent_pool=1M 8250.nr_uarts=1 snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 video=HDMI-A-1:1440x900M@60 smsc95xx.macad
dr=E4:5F:01:2D:59:AE vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  dwc_otg.lpm_enable=0 console=ttyS0,115200 root=/dev/mmcblk0p2 rootfstype=ext4 roo
twait
+ Kernel command line: coherent_pool=1M 8250.nr_uarts=1 snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 video=HDMI-A-1:1280x800M@60 smsc95xx.macad
dr=DC:A6:32:0D:6F:29 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  dwc_otg.lpm_enable=0 console=ttyS0,115200 root=/dev/mmcblk0p2 rootfstype=ext4 roo
twait
  Dentry cache hash table entries: 131072 (order: 7, 524288 bytes, linear)
  Inode-cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
  mem auto-init: stack:off, heap alloc:off, heap free:off
@@ -96,10 +96,10 @@
  VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
  FS-Cache: Loaded
  CacheFiles: Loaded
- simple-framebuffer 3e6c5000.framebuffer: framebuffer at 0x3e6c5000, 0x502800 bytes, mapped to 0x(ptrval)
- simple-framebuffer 3e6c5000.framebuffer: format=a8r8g8b8, mode=1440x900x32, linelength=5760
- Console: switching to colour frame buffer device 180x56
- simple-framebuffer 3e6c5000.framebuffer: fb0: simplefb registered!
+ simple-framebuffer 3e7df000.framebuffer: framebuffer at 0x3e7df000, 0x3e8000 bytes, mapped to 0x(ptrval)
+ simple-framebuffer 3e7df000.framebuffer: format=a8r8g8b8, mode=1280x800x32, linelength=5120
+ Console: switching to colour frame buffer device 160x50
+ simple-framebuffer 3e7df000.framebuffer: fb0: simplefb registered!
  NET: Registered protocol family 2
  IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear)
  tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
@@ -129,13 +129,27 @@
  Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)
  io scheduler mq-deadline registered
  io scheduler kyber registered
- gpio-507 (ant1): hogged as output/high
- gpio-511 (ant2): hogged as output/low
  brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:
  brcm-pcie fd500000.pcie:   No bus range found for /scb/pcie@7d500000, using [bus 00-ff]
  brcm-pcie fd500000.pcie:      MEM 0x0600000000..0x063fffffff -> 0x00c0000000
- brcm-pcie fd500000.pcie:   IB MEM 0x0000000000..0x00ffffffff -> 0x0400000000
- brcm-pcie fd500000.pcie: link down
+ brcm-pcie fd500000.pcie:   IB MEM 0x0000000000..0x00bfffffff -> 0x0400000000
+ brcm-pcie fd500000.pcie: link up, 5.0 GT/s PCIe x1 (SSC)
+ brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
+ pci_bus 0000:00: root bus resource [bus 00-ff]
+ pci_bus 0000:00: root bus resource [mem 0x600000000-0x63fffffff] (bus address [0xc0000000-0xffffffff])
+ pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
+ pci 0000:00:00.0: PME# supported from D0 D3hot
+ PCI: bus0: Fast back to back transfers disabled
+ pci 0000:01:00.0: [1106:3483] type 00 class 0x0c0330
+ pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00000fff 64bit]
+ pci 0000:01:00.0: PME# supported from D0 D3cold
+ PCI: bus1: Fast back to back transfers disabled
+ pci 0000:00:00.0: BAR 8: assigned [mem 0x600000000-0x6000fffff]
+ pci 0000:01:00.0: BAR 0: assigned [mem 0x600000000-0x600000fff 64bit]
+ pci 0000:00:00.0: PCI bridge to [bus 01]
+ pci 0000:00:00.0:   bridge window [mem 0x600000000-0x6000fffff]
+ pcieport 0000:00:00.0: enabling device (0140 -> 0142)
+ pcieport 0000:00:00.0: PME: Signaling with IRQ 82
  Serial: 8250/16550 driver, 1 ports, IRQ sharing enabled
  bcm2835-aux-uart fe215040.serial: there is not valid maps for state default
  iproc-rng200 fe104000.rng: hwrng registered

Then pcieport enables the xhci host and usb hub. The CM4 log doesn't seem to have any errors that don't show up in the Pi4b log as well.

I don't know what changed in my builds to cause this. I have removed all my software and am using only core open-embedded and poky layers + meta-raspberrypi on kirkstone. I have deleted my whole build folder and started over fresh and still have this issue. How could this carry over after that? I am at a loss as to what to try next.

Any help or ideas are appreciated.

[cdsteinkuehler]

I don't know Yocto (I use Buildroot), but the CM4 + I/O board doesn't have anything connected to the PCIe interface unless you plug something into the x1 PCIe connector on the I/O board. The RPi 4B connects the PCIe bus to a USB3 controller. The CM4 + I/O board uses the on-chip XHCI USB controller connected to a 4-port USB hub chip.

You need to make appropriate device-tree changes to enable the USB controller on the CM4. With the default OS, this is handled by adding "otg_mode=1" to your config.txt on the boot partition. You will need to figure out how to do something similar with Yocto depending on how much of the existing boot process you are leaving in place.

https://www.raspberrypi.com/documentati ... -pi-4-only

If you're not using the firmware loaded device tree, you may find the dtsi file I use for Buildroot to be helpful:

https://github.com/cdsteinkuehler/br2ra ... m-cm4.dtsi

[thagrol]

I don't know Yocto (I use Buildroot), but the CM4 + I/O board doesn't have anything connected to the PCIe interface unless you plug something into the x1 PCIe connector on the I/O board. The RPi 4B connects the PCIe bus to a USB3 controller. The CM4 + I/O board uses the on-chip XHCI USB controller connected to a 4-port USB hub chip.

This.

The equivalent port/controller on a 4B is the USB C port not the USB A ports. The USB A ports working on a 4B and no USB working on a CM4 is only useful information when you have a VL805 based PCIe USB card connected to it.

There is actually a choice of two in built USB controllers on 2711 based Pi: the above mentioned XHCI one or the old one that uses the dwc2 driver. The XHCI controller is host only, the other can do both host and device but not both at the same time.

[jovial_leavitt]

From that documentation link:

Because CM4 and CM4IO don’t include the external USB 3 controller, Raspberry Pi OS images set otg_mode=1 on CM4 for better performance.

And my cmdline shows dwc_otg.lpm_enable=0. I'm away from the office but I'll try to set that for my next build.

Thanks for the pointers.

[trejan]

Because CM4 and CM4IO don’t include the external USB 3 controller, Raspberry Pi OS images set otg_mode=1 on CM4 for better performance.

And my cmdline shows dwc_otg.lpm_enable=0. I'm away from the office but I'll try to set that for my next build.

otg_mode=1 goes into /boot/config.txt

[jovial_leavitt]

I see I was barking up the wrong tree with the dwc_otg kernel parameter, thanks. Looking into config.txt options I found this in the meta-raspberrypi yocto layer:

Enable USB host support

By default in case of the Compute Module 4 IO Board the standard USB driver that usually supports host mode operations is disabled for power saving reasons. Users who want to use the 2 USB built-in ports or the other ports provided via the header extension should set the following in local.conf:

ENABLE_DWC2_HOST = "1"

I set that in my distro conf and it resulted in this added to my /boot/config.txt:

Code: Select all

dtoverlay=dwc2,dr_mode=host

My usb keyboard works now! I don't know why the meta-rasperrypi docs don't mention otg_mode. I'm not clear on the advantages of one over the other for CM4.

[jovial_leavitt]

I can confirm that "otg_mode=1" also enables the USB on the CM4 I/O board. In my yocto layer I set this in my distro config with

Code: Select all

RPI_EXTRA_CONFIG="\n\
otg_mode=1\n\
"

I used RPI_EXTRA_CONFIG as there is no specific flag to add it to the config.txt in meta-raspberrypi/recipes-bsp/bootfiles/rpi-config_git.bb.

[trejan]

I'm not clear on the advantages of one over the other for CM4.

The SoC has two USB controllers. The old Synopsys DWC USB 2.0 OTG controller that can be a host or device and a new XHCI USB 2.0 host only controller that got added in the BCM2711. If you don't need device mode or OTG then the new XHCI controller is recommended as its much faster and less weird than the old DWC controller.

[jovial_leavitt]

Perfect, that's exactly what I needed to know. Thank you for the clarification. It's strange to me that otg_mode=1 enables the host-only xHCI controller.

I don't need device-mode/OTG so I was wary of using the otg_mode switch. But since it enables the xHCI controller I will move forward with otg_mode=1