tryingthepi
Posts: 27
Joined: Wed Jun 30, 2021 4:36 pm

Buildroot on CM3+ pitfalls? Stuck during boot

Mon Oct 11, 2021 11:45 am

Edit: Hello mods, only now saw sticky thread saying this is only for the known distributions, Buildroot custom stuff isn't really one I guess. Can this thread be moved to a more appropriate forum, or do I need to repost (where?) & delete?

----

Hello all,

in one of my threads on here someone was suggesting to try Buildroot to get a more minimalist custom image compared to pi-gen.

So I have been looking more into Buildroot lately.
They don't have the exact fitting premade configuration for my CM3+ module, but one for the regular Raspi3, so I took that one to start off of.

Since I don't recall changing anything when still playing with pi-gen, with regards to MMC partitions, to make something that runs on the CM3+ module, I concluded the eMMC of the CM3+ is probably connected the same way the SDcard slot would be on regular RasPi boards, as "it just works".
But perhaps that's wrong and some automagical detection is going on behind the scenes?

Because my first attempt at a Buildroot image gets stuck at:

Code: Select all

[    3.107402] mmc0: new high speed MMC card at address 0001
[    3.116501] mmcblk0: mmc0:0001 8GTF4R 7.28 GiB
[    3.124075] mmcblk0boot0: mmc0:0001 8GTF4R partition 1 4.00 MiB
[    3.133084] mmcblk0boot1: mmc0:0001 8GTF4R partition 2 4.00 MiB
[    3.141959] mmcblk0rpmb: mmc0:0001 8GTF4R partition 3 512 KiB, chardev (245:0)
[    3.153976]  mmcblk0: p1 p2
...and there it waits forever. Is there a problem with the MMC partitioning or something in that department?
I waited for 30min or so, so I don't think it's the 1st boot partition extension thing taking some time.

At least you can see that my providing altered cmdline.txt and config.txt worked to the extent that the UART pin remapping is in effect (hence the visible output here), which I just copied kfrom my pi-gen image (only those parts, the rest of the cmdline etc I left as the Buildroot Raspi3 file suggested, i.e. I did not copy over the whole cmdline/config.).
The only other changes I made were using the Buildroot menuconfig to add a few software packages.
The image so far is < 160 MB big, 1/10 of the pi-gen Lite image, so in that regard, this would be perfect. If only it worked ;)

The image that buildroot generates - I shoved it over to the eMMC, seen as mass-storage device /dev/sda after running "rpiboot" program, by using the "dd" command. IIRC, that's the way it's also mentioned in the official tutorial.

I'll add the complete serial console log below, just in case.

Any ideas what might be wrong, anything else I should paste here?

Code: Select all

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 5.10.46-v7 (sk@debian) (arm-buildroot-linux-uclibcgnueabihf-gcc.br_real (Buildroot -gf0e204d) 10.3.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP Wed Sep 22 21:53:00 CEST 2021
[    0.000000] CPU: ARMv7 Processor [410fd034] revision 4 (ARMv7), cr=10c5383d
[    0.000000] CPU: div instructions available: patching division code
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] OF: fdt: Machine model: Raspberry Pi Compute Module 3 Plus Rev 1.0
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] Reserved memory: created CMA memory pool at 0x35c00000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x0000000039bfffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000039bfffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000039bfffff]
[    0.000000] percpu: Embedded 20 pages/cpu s50700 r8192 d23028 u81920
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 234465
[    0.000000] Kernel command line: coherent_pool=1M snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1 bcm2708_fb.fbwidth=720 bcm2708_fb.fbheight=480 bcm2708_fb.fbswap=1 smsc95xx.macaddr=B8:27:EB:7C:6C:4F vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyAMA0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait modules-load=dwc2
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes, linear)
[    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 854452K/946176K available (10240K kernel code, 1312K rwdata, 2916K rodata, 1024K init, 862K bss, 26188K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] ftrace: allocating 31047 entries in 61 pages
[    0.000000] ftrace: allocated 61 pages with 5 groups
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000]  Rude variant of Tasks RCU enabled.
[    0.000000]  Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[    0.000000] random: get_random_bytes called from start_kernel+0x3b0/0x598 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000007] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.000023] Switching to timer-based delay loop, resolution 52ns
[    0.000315] Console: colour dummy device 80x30
[    0.001141] printk: console [tty1] enabled
[    0.001208] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=192000)
[    0.001266] pid_max: default: 32768 minimum: 301
[    0.001474] LSM: Security Framework initializing
[    0.001727] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
[    0.001775] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes, linear)
[    0.003236] cgroup: Disabling memory control group subsystem
[    0.003512] CPU: Testing write buffer coherency: ok
[    0.004013] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[    0.005251] Setting up static identity map for 0x100000 - 0x10003c
[    0.005459] rcu: Hierarchical SRCU implementation.
[    0.006410] smp: Bringing up secondary CPUs ...
[    0.007587] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
[    0.008862] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002
[    0.010172] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003
[    0.010326] smp: Brought up 1 node, 4 CPUs
[    0.010378] SMP: Total of 4 processors activated (153.60 BogoMIPS).
[    0.010408] CPU: All CPU(s) started in HYP mode.
[    0.010434] CPU: Virtualization extensions available.
[    0.011435] devtmpfs: initialized
[    0.028063] VFP support v0.3: implementor 41 architecture 3 part 40 variant 3 rev 4
[    0.028351] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.028411] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.031620] pinctrl core: initialized pinctrl subsystem
[    0.032818] NET: Registered protocol family 16
[    0.036695] DMA: preallocated 1024 KiB pool for atomic coherent allocations
[    0.042472] audit: initializing netlink subsys (disabled)
[    0.042782] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.043380] thermal_sys: Registered thermal governor 'step_wise'
[    0.043927] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[    0.043988] hw-breakpoint: maximum watchpoint size is 8 bytes.
[    0.044290] Serial: AMBA PL011 UART driver
[    0.060543] bcm2835-mbox 3f00b880.mailbox: mailbox enabled
[    0.080154] raspberrypi-firmware soc:firmware: Attached to firmware from 2021-07-19T13:37:22, variant start
[    0.090164] raspberrypi-firmware soc:firmware: Firmware hash is 1c9a23ddbd2fe5f73ace75b62e9c5a2ff10974ab
[    0.133998] Kprobes globally optimized
[    0.139139] bcm2835-dma 3f007000.dma: DMA legacy API manager, dmachans=0x1
[    0.141520] SCSI subsystem initialized
[    0.141801] usbcore: registered new interface driver usbfs
[    0.141885] usbcore: registered new interface driver hub
[    0.141975] usbcore: registered new device driver usb
[    0.143510] clocksource: Switched to clocksource arch_sys_counter
[    1.790978] VFS: Disk quotas dquot_6.6.0
[    1.791121] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    1.791348] FS-Cache: Loaded
[    1.791653] CacheFiles: Loaded
[    1.803053] NET: Registered protocol family 2
[    1.803345] IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    1.804739] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
[    1.804835] TCP established hash table entries: 8192 (order: 3, 32768 bytes, linear)
[    1.805069] TCP bind hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    1.805283] TCP: Hash tables configured (established 8192 bind 8192)
[    1.805484] UDP hash table entries: 512 (order: 2, 16384 bytes, linear)
[    1.805561] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear)
[    1.806236] NET: Registered protocol family 1
[    1.807093] RPC: Registered named UNIX socket transport module.
[    1.807126] RPC: Registered udp transport module.
[    1.807155] RPC: Registered tcp transport module.
[    1.807182] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.808421] hw perfevents: enabled with armv7_cortex_a7 PMU driver, 7 counters available
[    1.812229] Initialise system trusted keyrings
[    1.812526] workingset: timestamp_bits=14 max_order=18 bucket_order=4
[    1.822171] zbud: loaded
[    1.824266] FS-Cache: Netfs 'nfs' registered for caching
[    1.825155] NFS: Registering the id_resolver key type
[    1.825226] Key type id_resolver registered
[    1.825255] Key type id_legacy registered
[    1.825426] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    1.825462] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    1.826647] Key type asymmetric registered
[    1.826680] Asymmetric key parser 'x509' registered
[    1.826767] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 249)
[    1.826807] io scheduler mq-deadline registered
[    1.826835] io scheduler kyber registered
[    1.831886] bcm2708_fb soc:fb: FB found 1 display(s)
[    1.845882] Console: switching to colour frame buffer device 90x30
[    1.854379] bcm2708_fb soc:fb: Registered framebuffer for display 0, size 720x480
[    1.862777] bcm2835-rng 3f104000.rng: hwrng registered
[    1.866268] vc-mem: phys_addr:0x00000000 mem_base=0x3ec00000 mem_size:0x40000000(1024 MiB)
[    1.873265] gpiomem-bcm2835 3f200000.gpiomem: Initialised: Registers at 0x3f200000
[    1.888978] brd: module loaded
[    1.904592] loop: module loaded
[    1.909477] Loading iSCSI transport class v2.0-870.
[    1.914227] libphy: Fixed MDIO Bus: probed
[    1.917594] usbcore: registered new interface driver lan78xx
[    1.920752] usbcore: registered new interface driver smsc95xx
[    1.923839] dwc_otg: version 3.00a 10-AUG-2012 (platform bus)
[    1.927465] usbcore: registered new interface driver usb-storage
[    1.930799] mousedev: PS/2 mouse device common for all mice
[    1.935063] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.940720] sdhci: Secure Digital Host Controller Interface driver
[    1.943856] sdhci: Copyright(c) Pierre Ossman
[    1.947649] sdhost-bcm2835 3f202000.mmc: could not get clk, deferring probe
[    1.951079] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.954922] ledtrig-cpu: registered to indicate activity on CPUs
[    1.958281] hid: raw HID events driver (C) Jiri Kosina
[    1.961504] usbcore: registered new interface driver usbhid
[    1.964706] usbhid: USB HID core driver
[    1.972681] Initializing XFRM netlink socket
[    1.975868] NET: Registered protocol family 17
[    1.979047] Key type dns_resolver registered
[    1.982188] Registering SWP/SWPB emulation handler
[    1.985262] registered taskstats version 1
[    1.988062] Loading compiled-in X.509 certificates
[    1.991603] Key type ._fscrypt registered
[    1.994267] Key type .fscrypt registered
[    1.996894] Key type fscrypt-provisioning registered
[    2.011254] uart-pl011 3f201000.serial: cts_event_workaround enabled
[    2.014147] 3f201000.serial: ttyAMA0 at MMIO 0x3f201000 (irq = 114, base_baud = 0) is a PL011 rev2
[    2.921856] printk: console [ttyAMA0] enabled
[    2.931486] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    2.944359] sdhost: log_buf @ (ptrval) (f5d01000)
[    3.000592] mmc0: sdhost-bcm2835 loaded - DMA enabled (>1)
[    3.012018] of_cfs_init
[    3.017476] of_cfs_init: OK
[    3.024257] Waiting for root device ROOTDEV...
[    3.107402] mmc0: new high speed MMC card at address 0001
[    3.116501] mmcblk0: mmc0:0001 8GTF4R 7.28 GiB
[    3.124075] mmcblk0boot0: mmc0:0001 8GTF4R partition 1 4.00 MiB
[    3.133084] mmcblk0boot1: mmc0:0001 8GTF4R partition 2 4.00 MiB
[    3.141959] mmcblk0rpmb: mmc0:0001 8GTF4R partition 3 512 KiB, chardev (245:0)
[    3.153976]  mmcblk0: p1 p2

incognitum
Posts: 789
Joined: Tue Oct 30, 2018 3:34 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Mon Oct 11, 2021 4:20 pm

[ 3.024257] Waiting for root device ROOTDEV...
Any chance you are using a custom cmdline.txt that says "root=ROOTDEV", instead of "root=/dev/mmcblk0p2" ?

(The Linux kernel has no logic to resolve file system labels by itself, so root=FILESYSTEMLABEL is NOT going to work.
Linux distributions that do offer that have custom startup scripts to handle that, stored in a helper initramfs.).

tryingthepi
Posts: 27
Joined: Wed Jun 30, 2021 4:36 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Tue Oct 12, 2021 10:25 am

incognitum wrote:
Mon Oct 11, 2021 4:20 pm
[ 3.024257] Waiting for root device ROOTDEV...
Any chance you are using a custom cmdline.txt that says "root=ROOTDEV", instead of "root=/dev/mmcblk0p2" ?

(The Linux kernel has no logic to resolve file system labels by itself, so root=FILESYSTEMLABEL is NOT going to work.
Linux distributions that do offer that have custom startup scripts to handle that, stored in a helper initramfs.).
Weird! You're right, in the generated image it is like that, but not in the cmdline.txt laying around in the Buildroot folder.
It's actually been a couple weeks since I fiddled with that and forgot how/where I actually changed the cmdline.txt such that the image will have the UART pin remapping and loads dwc2. I think I just mounted the image as writable and edited by hand, to look at how to do this "properly" (in the Buildroot flow of doing things) some time later; but that =ROOTDEV thing, I didn't touch that, no idea where it got that from then.
I'll fix that, by hand, too, and see what happens.

tryingthepi
Posts: 27
Joined: Wed Jun 30, 2021 4:36 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Tue Oct 12, 2021 1:01 pm

It's getting further now, but not alot. It complains about g_cdc not working apparently, although support for gadget stuff looks to be enabled in the linux-menuconfig. No idea whether that's why it's now stuck at this:

Code: Select all

[    3.154559]  mmcblk0: p1 p2
[    3.178992] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
[    3.192893] VFS: Mounted root (ext4 filesystem) readonly on device 179:2.
[    3.203287] devtmpfs: mounted
[    3.216915] Freeing unused kernel memory: 1024K
[    3.234012] Run /sbin/init as init process
[    3.319814] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null)
[    3.469753] random: dd: uninitialized urandom read (512 bytes read)
[    3.534438] udc-core: couldn't find an available UDC - added [g_cdc] to list of pending drivers
[    3.816709] NET: Registered protocol family 10
[    3.825878] Segment Routing with IPv6
[    9.133567] random: crng init done

incognitum
Posts: 789
Joined: Tue Oct 30, 2018 3:34 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Tue Oct 12, 2021 1:20 pm

tryingthepi wrote:
Tue Oct 12, 2021 1:01 pm
It's getting further now, but not alot. It complains about g_cdc not working apparently, although support for gadget stuff looks to be enabled in the linux-menuconfig. No idea whether that's why it's now stuck at this:
1) g_* modules are deprecated, and could disappear in a future kernel version. Building your gadget devices through configfs in a startup script is a better long term solution.

2) It is not seeing an USB Device Controller. Double check "dtoverlay=dwc2,dr_mode=peripheral" is in config.txt, that your config.txt did make it to the image, and that there is is a dwc2.dbto in your overlays directory inside the FAT partition of your image...
Recall buildroot does not compile overlays by default, so file may be missing.

3) Not sure if it is actual stuck. Could be that the login prompt is waiting for you on a different terminal than where you are copying the kernel messages from. What tty did you configure in buildroot system setup?

tryingthepi
Posts: 27
Joined: Wed Jun 30, 2021 4:36 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Wed Oct 13, 2021 2:56 pm

incognitum wrote:
Tue Oct 12, 2021 1:20 pm
tryingthepi wrote:
Tue Oct 12, 2021 1:01 pm
It's getting further now, but not alot. It complains about g_cdc not working apparently, although support for gadget stuff looks to be enabled in the linux-menuconfig. No idea whether that's why it's now stuck at this:
1) g_* modules are deprecated, and could disappear in a future kernel version. Building your gadget devices through configfs in a startup script is a better long term solution.

2) It is not seeing an USB Device Controller. Double check "dtoverlay=dwc2,dr_mode=peripheral" is in config.txt, that your config.txt did make it to the image, and that there is is a dwc2.dbto in your overlays directory inside the FAT partition of your image...
Recall buildroot does not compile overlays by default, so file may be missing.

3) Not sure if it is actual stuck. Could be that the login prompt is waiting for you on a different terminal than where you are copying the kernel messages from. What tty did you configure in buildroot system setup?
1) Oh! Good to know. Configfs sounds cool, I'll look into that at a later time - right now I'd like to get running with a Buildroot image what was working before with pi-gen, before fiddling with new stuff and introducing more error sources and (even more) unfamiliarity, though.

2) "dtoverlay=dwc2" is in there, but not the "dr_mode=peripheral" part - which isn't in my pi-gen image either, though, and which works fine. What does it do?
The "dwc2.dbto" is also in the 0.fat/overlays folder.

3) The cmdline.txt has this in it: "console=serial0,115200 console=tty1"
Also in the config.txt, I am doing apin remap with dtoverlay=uart0,...;

These things all look the same as in my pi-gen image.
One difference I noticed is that the pi-gen image has the rc.local file to start things.
It wasn't present on the Buildroot-generated image, so I added a file in the /etc/init.d folder, which seems to be how that image works, judging from other stuff in there - to initialize my stuff, which I did in the rc.local before, in the pi-gen image. E.g. it loads the driver: "modprobe g_cdc", which I don't do in the cmdline as there is apparently some race condition between that and other stuff if you load it early - I forgot what it was.
This way worked when loding it in rc.local in the pi-gen image.
---
Edit:
Is there a way to diagnose where/how it gets stuck?

The part about missing g_cdc is missing now (I started from scratch once with the buildroot setup, and my Linux option selection was less driven by "let's see what else looks useless")

Code: Select all

[    3.147102]  mmcblk0: p1 p2
[    3.178383] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
[    3.192225] VFS: Mounted root (ext4 filesystem) readonly on device 179:2.
[    3.202612] devtmpfs: mounted
[    3.216176] Freeing unused kernel memory: 1024K
[    3.233439] Run /sbin/init as init process
[    3.350831] EXT4-fs (mmcblk0p2): re-mounted. Opts: (null)
[    3.521835] random: dd: uninitialized urandom read (512 bytes read)
[    8.892976] random: crng init done
[   13.264437] NET: Registered protocol family 10
[   13.273561] Segment Routing with IPv6

incognitum
Posts: 789
Joined: Tue Oct 30, 2018 3:34 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Thu Oct 14, 2021 8:05 am

tryingthepi wrote:
Wed Oct 13, 2021 2:56 pm
3) The cmdline.txt has this in it: "console=serial0,115200 console=tty1"
That typically means: "send kernel messages to both serial0 and tty1 (HDMI), let /dev/console point to tty1 (HDMI), as that is the last parameter"
Might want to check if your login prompt isn't at HDMI then...

Or set your serial tty explicitly in buildroot if you prefer it there, instead of the default of /dev/console
buildroot-menuconfig-system-config.png
buildroot-menuconfig-system-config.png (45.55 KiB) Viewed 415 times
That is assuming that with it being "stuck" you mean you are expecting to see a login prompt, and are not seeing that.
If your custom startup script is supposed to start something else, and it is not doing that, you may want to be a bit more specific about what else it is not doing.
2) "dtoverlay=dwc2" is in there, but not the "dr_mode=peripheral" part - which isn't in my pi-gen image either, though, and which works fine. What does it do?
Forces USB controller into gadget mode.
Instead of trusting the OTG magic to auto-detect if it should be USB host or USB gadget works.

tryingthepi
Posts: 27
Joined: Wed Jun 30, 2021 4:36 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Thu Oct 14, 2021 11:50 am

incognitum wrote:
Thu Oct 14, 2021 8:05 am
tryingthepi wrote:
Wed Oct 13, 2021 2:56 pm
3) The cmdline.txt has this in it: "console=serial0,115200 console=tty1"
That typically means: "send kernel messages to both serial0 and tty1 (HDMI), let /dev/console point to tty1 (HDMI), as that is the last parameter"
Might want to check if your login prompt isn't at HDMI then...

That is assuming that with it being "stuck" you mean you are expecting to see a login prompt, and are not seeing that.
If your custom startup script is supposed to start something else, and it is not doing that, you may want to be a bit more specific about what else it is not doing.
2) "dtoverlay=dwc2" is in there, but not the "dr_mode=peripheral" part - which isn't in my pi-gen image either, though, and which works fine. What does it do?
Forces USB controller into gadget mode.
Instead of trusting the OTG magic to auto-detect if it should be USB host or USB gadget works.
Thanks a ton for that already! I wouldn't have thought that setting something the same as when using pi-gen would be a problem, but it seems it is.
So by stuck, I meant both: no login (and no further output) on the console where there is some initial output, and also, my stuff evidently not running, as it sets up g_cdc for both ethernet and serial over USB, then starts a program that waits for an IP/MAC config string over serial, and then the CM3 appears as ethernet thingy on the host system. That's not happening.

So now I removed the console=tty1 part from the cmdline.txt, and I added "dr_mode=peripheral" to config.txt, that should at least help later.
Now I do get a login on the same console where things started to output (and I can't find the actual login username in the Buildroot menu, lol! The password is there... and a machine name).

Now there's this thing again:

Code: Select all

[    3.415084] udc-core: couldn't find an available UDC - added [g_cdc] to list of pending drivers
So "my stuff" couldn't progress, because the expected devices (a certain tty, and usb network) don't exist.
In linux-menuconfig, I added everything seemingly related to gadget stuff, e.g. anything mentioning ACM, EEM, composite (?).
What else could be missing?

incognitum
Posts: 789
Joined: Tue Oct 30, 2018 3:34 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Thu Oct 14, 2021 12:00 pm

tryingthepi wrote:
Thu Oct 14, 2021 11:50 am
Now I do get a login on the same console where things started to output (and I can't find the actual login username in the Buildroot menu, lol! The password is there... and a machine name).
root
In linux-menuconfig, I added everything seemingly related to gadget stuff, e.g. anything mentioning ACM, EEM, composite (?).
What else could be missing?
dwc2?

This list of options is sufficient for gadget through configfs: https://github.com/raspberrypi/scriptex ... agment#L36

tryingthepi
Posts: 27
Joined: Wed Jun 30, 2021 4:36 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Thu Oct 14, 2021 12:14 pm

incognitum wrote:
Thu Oct 14, 2021 12:00 pm
tryingthepi wrote:
Thu Oct 14, 2021 11:50 am
Now I do get a login on the same console where things started to output (and I can't find the actual login username in the Buildroot menu, lol! The password is there... and a machine name).
root
In linux-menuconfig, I added everything seemingly related to gadget stuff, e.g. anything mentioning ACM, EEM, composite (?).
What else could be missing?
dwc2?

This list of options is sufficient for gadget through configfs: https://github.com/raspberrypi/scriptex ... agment#L36
root didn't work, I guess I must have messed something up and forgot about it. Never mind me confusing 1 (one) with l (lower case L) in one of those stupid fonts.

dwc2 is on, but now that I see you mention "OTG magic" earlier - OTG is unselected, as I though that's only for when you (also) need host behavior?

I'll check against that list, thanks!

--
Edit:
I saw someone who had this "no UDC" problem when having the external pin not at the corect voltage level.
Now, I'm aware of the pin and am setting it according to the mode, before powering the CM3, and this worked with the pi-gen image.
Is there a reason to suspect that this could behave differently with the Buildroot image, perhaps some special configuration missing or different? After all, the Buildroot settings I started with were for the regular Raspi3, not the CM3+.

incognitum
Posts: 789
Joined: Tue Oct 30, 2018 3:34 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Fri Oct 15, 2021 10:15 am

tryingthepi wrote:
Thu Oct 14, 2021 12:14 pm
dwc2 is on
"on" as statically build into the kernel ( =y ) ?
As opposed to being a module (=m). Which are only loaded automatically if you are using (e)udev or systemd for device node management. Not if you had chosen say the devtmpfs only option.
Edit:
I saw someone who had this "no UDC" problem when having the external pin not at the corect voltage level.
It could theoretically also happen if for some reason the g_* module is loaded earlier than dwc2 is loaded.
Then it does should work in the end after dwc2 is loaded (that is the pending part mentioned in the message).
In that case there should be lines mentioning "dwc" afterwards in the kernel messages though.

tryingthepi
Posts: 27
Joined: Wed Jun 30, 2021 4:36 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Wed Oct 20, 2021 4:41 pm

incognitum wrote:
Fri Oct 15, 2021 10:15 am
tryingthepi wrote:
Thu Oct 14, 2021 12:14 pm
dwc2 is on
"on" as statically build into the kernel ( =y ) ?
As opposed to being a module (=m). Which are only loaded automatically if you are using (e)udev or systemd for device node management. Not if you had chosen say the devtmpfs only option.
Edit:
I saw someone who had this "no UDC" problem when having the external pin not at the corect voltage level.
It could theoretically also happen if for some reason the g_* module is loaded earlier than dwc2 is loaded.
Then it does should work in the end after dwc2 is loaded (that is the pending part mentioned in the message).
In that case there should be lines mentioning "dwc" afterwards in the kernel messages though.
I've now set that to =y and also juggling with some other options.
I now get as far as the g_cdc loading and my program fetching the ip from serial, setting up usb0 IP etc, and I can ping the CM3 now.
So thanks for your hints, that stuff works.

Only that now, when I thought I was done and could get to the "actual work", other buildroot stuff is imploding.
Apparently some sort of other misconfiguration. I did heed every mention of dependnecies in menuconfig.
It lured me initially by looking so simple, just pick what you want in your image from menuconfig, build, and go, eh? Not quite ;)

Now I can't log in via SSH - I did set the PermitRootLogin to yes in the sshd_config, but all I get from the other end is "connection reset".
I also see kernel messages on the CM3 like

Code: Select all

audit: type=1326 ... uid=1000 gid=1000 ... comm="sshd" ... sig=31 arch=40000028 syscall=413 compat=0 ... code=0x0
.
I was given the hint that something with SELinux ma<y be misconfigured, albeit "raspberrypi3_defconfig" supposedly activates AppArmor, not SELinux - while I cannot find evidence of any of them being present on the runing system. There is one process running with "audit" in its name, which is kauditd - from what I found, there would normally also some userspace audit process be running. /sys/kernel/security is empty.

I made no specific choices anywhere with regards to these things, wasn't even aware of their existence. So I wonder how I could have misconfigured that.

And htop isn't working, because it doesn't like "No btime in /proc/stat", (it is in there, but value 0; /proc/uptime is not 0, though).

As this is further removed from Raspi specific issues, I guess, rather more into Buildroot, I'm asking elsewhere.
But if you happen to have some hints about this, do feel free to mention them... ;)

Edit:
Okay... seems there is a toolchain problem...
https://bugs.busybox.net/show_bug.cgi?id=13671

I don't really like the options I see there, but oh well.

incognitum
Posts: 789
Joined: Tue Oct 30, 2018 3:34 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Wed Oct 20, 2021 7:17 pm

tryingthepi wrote:
Wed Oct 20, 2021 4:41 pm
It lured me initially by looking so simple, just pick what you want in your image from menuconfig, build, and go, eh? Not quite ;)
It is simple if you came from the "Linux from scratch" route.
Not if you are used to full distributions. 8-)

Code: Select all

audit: type=1326 ... uid=1000 gid=1000 ... comm="sshd" ... sig=31 arch=40000028 syscall=413 compat=0 ... code=0x0
Think most in embedded land (where every kilobyte counts) use dropbear instead of normal openssh.

epoch1970
Posts: 7087
Joined: Thu May 05, 2016 9:33 am
Location: France

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Wed Oct 20, 2021 9:39 pm

incognitum wrote:
Wed Oct 20, 2021 7:17 pm
Think most in embedded land (where every kilobyte counts) use dropbear instead of normal openssh.
+1.
I used to build dropbear’s ssh client as a buildroot host package and use that one to connect to the target. Never did manage to get the regular openssh Linux client to connect.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

tryingthepi
Posts: 27
Joined: Wed Jun 30, 2021 4:36 pm

Re: Buildroot on CM3+ pitfalls? Stuck during boot

Thu Oct 21, 2021 9:12 am

incognitum wrote:
Wed Oct 20, 2021 7:17 pm
tryingthepi wrote:
Wed Oct 20, 2021 4:41 pm
It lured me initially by looking so simple, just pick what you want in your image from menuconfig, build, and go, eh? Not quite ;)
It is simple if you came from the "Linux from scratch" route.
Yeah, and probably as a requirement, too, rather than just a comparison ;)

Code: Select all

audit: type=1326 ... uid=1000 gid=1000 ... comm="sshd" ... sig=31 arch=40000028 syscall=413 compat=0 ... code=0x0
Think most in embedded land (where every kilobyte counts) use dropbear instead of normal openssh.
That's what I ended up with now.
I've read some claims that the maintenance status of dropbear wasn't good, up to claims the project was "dead", or some things weren't well implemented. It so far works for my scenario

Return to “Other”