Pi4 DRM questions

[dividuum]

I'm currently porting info-beamer pi to the Pi4. I'm using DRM to set up a GL surface without using X at all. A few questions came up while working on that:

• When is /dev/dri/card0 and when is /dev/dri/card1 active? Usually it's card1, but I remember seeing card0 successfully working as well.
• Are there always two crtc->encoder->connector "pipelines" for each connected display? I'm trying to get a grasp on how drm works, and from what I understood, it seems that it should also be possible to have a single crtc and then drives to encoders?
• If I use multiple drmModeSetCrtc for each display for a cloned output, I guess I have to drmModePageFlip for each of them? I'm confused on how that's supposed to work if there's a single eglSwapBuffers followed by two drmModePageFlip calls. In the worst case, I guess one of the flips handlers might have to wait for full frame cycle (1/60s) in case it just missed the vsync event? Any insight on how something like that is usually handled? Is there a way to synchronize the vsync on the two connectors somehow? What happens if the refresh rate doesn't match?
• Querying drmModeGetResources doesn't seem to detect a newly connected screen. The number of connectors returned seems to depend on which screen was connected when the Pi started. Disconnecting a screen changes the modes available to the corresponding connector, but it's still marked as DRM_MODE_CONNECTED. How can I detect new displays and correctly detect their status?
• Is there any guaranteed mapping between the DRM connectors and the dispmanx displays? So far it seems that DISPMANX_ID_HDMI0 is connector 0 and DISPMANX_ID_HDMI1 is connector 1.
• Is there a vc_gencmd hdmi_adjust_clock for HDMI1? Or does hdmi_adjust_clock adjust both HDMI0 and HDMI1?
• Is it possible to create a GL surface with a resolution not matching the screen resolution? Previously I could create an arbitrarily sized dispmanx layer, place/scale that on the screen and then eglCreateWindowSurface on it.
• Possibly related: Is it possible to create a GL surface spanning two screens? I would guess so if I create a gbm surface with the combined size and then call drmModeAddFB2WithModifiers with correct offset/strides/etc?
• Is there any useful documentation on drm/gbm and how all that plays together? I'd really prefer a less trial&error approach. I've yet to find a man page for drmModeSetCrtc for example. Documentation seem to be horrible overall.

[6by9]

I'm currently porting info-beamer pi to the Pi4. I'm using DRM to set up a GL surface without using X at all. A few questions came up while working on that:

• When is /dev/dri/card0 and when is /dev/dri/card1 active? Usually it's card1, but I remember seeing card0 successfully working as well.

Pi0-3 you only have the vc4 driver, therefore render is via /dev/dri/card0. Pi4 has both vc4 for render, and v3d for 3D. You can probe the devices for their capabilities - only one should acknowledge that it has DRIVER_RENDER or DRIVER_MODESET capabilities.

[*]Are there always two crtc->encoder->connector "pipelines" for each connected display? I'm trying to get a grasp on how drm works, and from what I understood, it seems that it should also be possible to have a single crtc and then drives to encoders?

I'd only expect one crtc->encoder->connector per display. What makes you think there are two?
modetest is the lowest level DRM test tool. "modetest -M vc4" for my single headed display lists

Code: Select all

Encoders:
id      crtc    type    possible crtcs  possible clones 
50      49      TMDS    0x00000001      0x00000000

Connectors:
id      encoder status          name            size (mm)       modes   encoders
51      50      connected       HDMI-A-1        1600x900                81      50
  modes:
        name refresh (Hz) hdisp hss hse htot vdisp vss vse vtot)
  4096x2160 30 4096 4184 4272 4400 2160 2168 2178 2250 297000 flags: phsync, pvsync, 2D; type: driver
...
CRTCs:
id      fb      pos     size
49      60      (0,0)   (1920x1080)
  1920x1080 60 1920 2008 2052 2200 1080 1084 1089 1125 148500 flags: phsync, pvsync, 2D; type: driver
  props:
  ...

Some hardware supports cloning using the output of one crtc to feed multiple encoders. The Pi SoC does not.
If I get my relationships right, then:
Planes are fed into the HVS
CRTC = PixelValve
encoder = DSI/DPI/HDMI blocks.
connector is a slightly abstract concept as the outputs of the encoder blocks can't be remapped to alternate connectors, and most connector parameters actually modify encoder parameters.

Pixelvalves can only feed one destination.

[*] If I use multiple drmModeSetCrtc for each display for a cloned output, I guess I have to drmModePageFlip for each of them? I'm confused on how that's supposed to work if there's a single eglSwapBuffers followed by two drmModePageFlip calls. In the worst case, I guess one of the flips handlers might have to wait for full frame cycle (1/60s) in case it just missed the vsync event? Any insight on how something like that is usually handled? Is there a way to synchronize the vsync on the two connectors somehow? What happens if the refresh rate doesn't match?

I don't believe there is any guarantee that the displays will be in sync, and there's nothing stopping the displays having different refresh rates other than your application. The beat effects could be fun if one is at 24Hz and one at 25Hz.
You'll need to be very careful with buffer management, and only return the old buffer to egl once both displays have completed the respective flip.
X gives up in disgust and only single buffers!

[*] Querying drmModeGetResources doesn't seem to detect a newly connected screen. The number of connectors returned seems to depend on which screen was connected when the Pi started. Disconnecting a screen changes the modes available to the corresponding connector, but it's still marked as DRM_MODE_CONNECTED. How can I detect new displays and correctly detect their status?

Hot plug is still not supported with FKMS. With 7 different display outputs (DSI0, DSI1, DPI, VEC, HDMI0, HDMI1, and transposer) but only 3 HVS channels there has to be a decision made as to how the resources are allocated, and that is done at boot time within the firmware.
If there is an HDMI screen connected at boot, then we may be able to add hotplug notifications for subsequent connections/removal, but it's not that high on the priority list.

vc4-kms-v3d on a Pi0-3 should support hotplug. The driver changes haven't been made yet for Pi4 (it's on the list).

[*] Is there any guaranteed mapping between the DRM connectors and the dispmanx displays? So far it seems that DISPMANX_ID_HDMI0 is connector 0 and DISPMANX_ID_HDMI1 is connector 1.

If in doubt, then enumerate and check the display types.
The enumeration order in the firmware is always the same, and FKMS enumerates the displays in the same order as DispmanX. HDMI0 will always be enumerated before HDMI1. If HDMI0 is not connected, then HDMI1 will be assigned HDMI-A-1. I did look into trying to make the connector names match, but the names that can be assigned in the kernel are never passed out to userspace!
DSI and DPI displays will always both be reported as DSI.

[*] Is there a vc_gencmd hdmi_adjust_clock for HDMI1? Or does hdmi_adjust_clock adjust both HDMI0 and HDMI1?

I don't believe that is hooked up at all. It's certainly not something I've tested.

[*] Is it possible to create a GL surface with a resolution not matching the screen resolution? Previously I could create an arbitrarily sized dispmanx layer, place/scale that on the screen and then eglCreateWindowSurface on it.

From a DRM perspective, absolutely. How to get GL and DRM to co-operate and actually do it I couldn't say.

[*] Possibly related: Is it possible to create a GL surface spanning two screens? I would guess so if I create a gbm surface with the combined size and then call drmModeAddFB2WithModifiers with correct offset/strides/etc?

That sounds correct. X works in this way - GL renders the screen to a framebuffer that encompasses the entire desktop, and then each DRM plane is given a source rectangle on that buffer.

[*] Is there any useful documentation on drm/gbm and how all that plays together? I'd really prefer a less trial&error approach. I've yet to find a man page for drmModeSetCrtc for example. Documentation seem to be horrible overall.

Documentation - that million dollar question. If you find any really good docs then please let me know!
The kernel API docs are at https://www.kernel.org/doc/html/latest/ ... -uapi.html#
There was a post on dri-devel recently of someone trying to pull together userland documentation, but it didn't seem to go very far. https://www.spinics.net/lists/dri-devel/msg214986.html for reference.

[dividuum]

Hey. Thanks a lot for the quick response and very useful insight. I'll see how far I get

[6by9]

Please don't take what I say as necessarily 100% gospel truth - I'm still learning how a lot of this hangs together, and it's not obvious in many places. I'm generally referring to the vc4-kms-v3d driver as I can relate to how the hardware works, and then work backwards as to how that is represented in DRM!

[dividuum]

Another day and a few follow up comments and questions.

modetest is the lowest level DRM test tool. "modetest -M vc4" for my single headed display lists [...]

Thanks for that. That makes it easier to see what's going on.

I don't believe there is any guarantee that the displays will be in sync, and there's nothing stopping the displays having different refresh rates other than your application. The beat effects could be fun if one is at 24Hz and one at 25Hz.

Thought so. I toyed around with various vsync approaches and I can't get a frame rate higher than 30fps unless I ignore vsync on one of the connectors: drmModePageFlip for one of the connectors is basically instantaneous, while a second drmModePageFlip for the other connector then takes 30ms, effectively resulting in a lost vsync and half the frame rate. Calling drmModePageFlip for only one of the connectors results in 60fps, but then there's tearing on the other display. Oh well. If you or anyone else has any idea what to try here I'd be very happy.

Hot plug is still not supported with FKMS. With 7 different display outputs (DSI0, DSI1, DPI, VEC, HDMI0, HDMI1, and transposer) but only 3 HVS channels there has to be a decision made as to how the resources are allocated, and that is done at boot time within the firmware.
If there is an HDMI screen connected at boot, then we may be able to add hotplug notifications for subsequent connections/removal, but it's not that high on the priority list.

Is there a way to specify that HDMI0 and HDMI1 should be configured via /boot/config.txt regardless of whether or not there's actually anything connected? That way drm would at least see both connectors and once a display is connected (or switched on?) later, it can use those without rebooting the Pi. Otherwise any kind of reliable dual display setup might be a bit wonky as the state of the system is fixed at boot time.

[*] Possibly related: Is it possible to create a GL surface spanning two screens? I would guess so if I create a gbm surface with the combined size and then call drmModeAddFB2WithModifiers with correct offset/strides/etc?

That sounds correct. X works in this way - GL renders the screen to a framebuffer that encompasses the entire desktop, and then each DRM plane is given a source rectangle on that buffer.

I was able to get that working. Now I can (for example) clone a single FullHD surface or span a single 3840x1080 or 1920x2160 across two screens. And I can also freely overlap them. Pretty neat, except for the vsync problem.

Documentation - that million dollar question. If you find any really good docs then please let me know!
The kernel API docs are at https://www.kernel.org/doc/html/latest/ ... -uapi.html#
There was a post on dri-devel recently of someone trying to pull together userland documentation, but it didn't seem to go very far. https://www.spinics.net/lists/dri-devel/msg214986.html for reference.

I guess we'll have to wait until 2029 then.

Another question about video support. Previously I've used dispmanx in combination with the video_render OMX component for fullscreen video output. Now that OMX is on its way out, I'm looking into mmal. Especially as I can't see any way to make a single video_render for dual screen output as there's only one OMX_DISPLAY_SET_NUM for that component. I guess it would be possible to split the video and use two renderers (?) but that's sounds clunky and potentially slow.

Do you know if it's possible with mmal in combination with DRM to have a single video stretched on two displays? Right now I don't really understand any of the drmModeSetPlane code that might be required for that. Any insight might be helpful. Thanks!

[6by9]

I don't believe there is any guarantee that the displays will be in sync, and there's nothing stopping the displays having different refresh rates other than your application. The beat effects could be fun if one is at 24Hz and one at 25Hz.

Thought so. I toyed around with various vsync approaches and I can't get a frame rate higher than 30fps unless I ignore vsync on one of the connectors: drmModePageFlip for one of the connectors is basically instantaneous, while a second drmModePageFlip for the other connector then takes 30ms, effectively resulting in a lost vsync and half the frame rate. Calling drmModePageFlip for only one of the connectors results in 60fps, but then there's tearing on the other display. Oh well. If you or anyone else has any idea what to try here I'd be very happy.

Two threads, each independently wait on a drmModePageFlip for their respective display?
There are the atomic calls for DRM. Can you make an atomic commit over multiple displays?

Hot plug is still not supported with FKMS. With 7 different display outputs (DSI0, DSI1, DPI, VEC, HDMI0, HDMI1, and transposer) but only 3 HVS channels there has to be a decision made as to how the resources are allocated, and that is done at boot time within the firmware.
If there is an HDMI screen connected at boot, then we may be able to add hotplug notifications for subsequent connections/removal, but it's not that high on the priority list.

Is there a way to specify that HDMI0 and HDMI1 should be configured via /boot/config.txt regardless of whether or not there's actually anything connected? That way drm would at least see both connectors and once a display is connected (or switched on?) later, it can use those without rebooting the Pi. Otherwise any kind of reliable dual display setup might be a bit wonky as the state of the system is fixed at boot time.

hdmi_force_hotplug:0=1 and hdmi_force_hotplug:1=1, but you really then want the genuine hotplug status passed through KMS as well to try and trigger X to do something on that.

Another question about video support. Previously I've used dispmanx in combination with the video_render OMX component for fullscreen video output. Now that OMX is on its way out, I'm looking into mmal. Especially as I can't see any way to make a single video_render for dual screen output as there's only one OMX_DISPLAY_SET_NUM for that component. I guess it would be possible to split the video and use two renderers (?) but that's sounds clunky and potentially slow.

Do you know if it's possible with mmal in combination with DRM to have a single video stretched on two displays? Right now I don't really understand any of the drmModeSetPlane code that might be required for that. Any insight might be helpful. Thanks!

DispmanX still exists, and indeed FKMS is passing frames to DispmanX to render.
IL and MMAL video_render components also rely on DispmanX to drive the hardware.
Stretching a single buffer over multiple displays requires the image to be presented as two DispmanX elements, one for each display. DispmanX has no notion of how multiple displays might be arranged, therefore it's near impossible to set it up to create the required source rectangles for itself.

If your buffer is either allocated via vcsm, or is a dmabuf that you import into vcsm, then you can set MMAL_PARAMETER_ZERO_COPY on the buffer and pass the vcsm gpu handle in buffer->data. That avoids any copying and is about the most efficient you can get.
If you look at yavta then you'll see an example of that on the output of the isp component passing the buffers on to render and/or encoder. The source buffer is returned to the pool when all replicated buffers have also been released.
raspiraw does a similar thing in passing buffers to multiple destinations, and/or passing them onto processing threads to do stuff with.

IL, MMAL, and DispmanX will remain whilst we have the firmware in control of the HVS/PV/HDMI etc. Going for DRM isolates you from all of those issues anyway, as that will be the guaranteed API in future.
IL is not as efficient as MMAL in that ALL buffers have to be copied from ARM to GPU memory, and it really is a pain to get right.

[dividuum]

Two threads, each independently wait on a drmModePageFlip for their respective display?

Not at the moment. Is that how it's supposed to work? Right now I eglSwapBuffers, then gbm_surface_lock_front_buffer, optionally create a drm fb from if none exists yet and then issue two drmModePageFlip calls for both connectors. I then use `select` to wait on those two flips calling drmHandleEvent once the drm fd becomes readable. So it's more or less similar to https://gitlab.freedesktop.org/mesa/kms ... gacy.c#L72

There are the atomic calls for DRM. Can you make an atomic commit over multiple displays?

Good question. kmscube with -A argument didn't work (sorry, don't remember the exact error message at the moment) in my naive tests, so I didn't investigate how that is supposed to work.

hdmi_force_hotplug:0=1 and hdmi_force_hotplug:1=1, but you really then want the genuine hotplug status passed through KMS as well to try and trigger X to do something on that.

Thanks! I've already use hdmi_force_hotplug in other circumstance but didn't know how to use that on the Pi4. Having the true display status would be neat, but for a less dynamic hardware setup where the expected display configuration is known and rarely changes, hdmi_force_hotplug is sufficient.

DispmanX still exists, and indeed FKMS is passing frames to DispmanX to render.
IL and MMAL video_render components also rely on DispmanX to drive the hardware.
Stretching a single buffer over multiple displays requires the image to be presented as two DispmanX elements, one for each display. DispmanX has no notion of how multiple displays might be arranged, therefore it's near impossible to set it up to create the required source rectangles for itself.

Understood and I didn't expect any magic there, but wasn't sure how the Pi4 handles this. I'm wondering about expected performance hits from showing a video on two displays. I guess with OMX, I'd have to set up the video_splitter between the video_decoder and two video_render components. Except for the additional work for the HVS the overhead of 2 vs 1 video_render should be manageable? I expect the approach with mmal to be quite similar?

If your buffer is either allocated via vcsm, or is a dmabuf that you import into vcsm, then you can set MMAL_PARAMETER_ZERO_COPY on the buffer and pass the vcsm gpu handle in buffer->data. That avoids any copying and is about the most efficient you can get.

Yuk. I'm sure I'll understand eventually how all that works. Thanks

If you look at yavta then you'll see an example of that on the output of the isp component passing the buffers on to render and/or encoder. The source buffer is returned to the pool when all replicated buffers have also been released.
raspiraw does a similar thing in passing buffers to multiple destinations, and/or passing them onto processing threads to do stuff with.

Thanks. Will look into those. Is http://www.jvcref.com/files/PI/document ... index.html still be best/only documentation for mmal?