Is it possible to use two HQ cameras with the libcamera Framework?

[jamaar]

Hi all,

I have a Compute Module 4 with the official I/O Board. On this board I have attached two Raspberry Pi HQ cameras.
I want to use two cameras at the same time.
Last week I triggered the following Bug with the stereo mode of raspistill: https://forum.stereopi.com/viewtopic.php?t=997
I have tried the new libcamera framework. I have gotten all libraries and tools working, but I can only use one camera.
I have changed the source code of the CameraManager to use the second camera, but I think the camera is not recognized.

The output of

Code: Select all

vcgencmd get_camera

is

Code: Select all

supported=2 detected=1

Code: Select all

v4l2-ctl --list-devices

gives me the following output:

Code: Select all

bcm2835-codec-decode (platform:bcm2835-codec):
        /dev/video10
        /dev/video11
        /dev/video12

bcm2835-isp (platform:bcm2835-isp):
        /dev/video13
        /dev/video14
        /dev/video15
        /dev/video16

mmal service 16.1 (platform:bcm2835-v4l2-0):
        /dev/video0

unicam (platform:fe801000.csi):
        /dev/video1
        /dev/video2

Code: Select all

vcdbg log msg

gives me the following shortened output:

Code: Select all

029704.277: camsubs: Ignoring camera 0 as unicam device not available
029715.391: camsubs: Looking for camera 1: i2c_port = 0, led gpio = -1, power enable gpio = 133
030016.035: camsubs: Camera not found
030016.206: camsubs: Looking for camera 1: i2c_port = 0, led gpio = -1, power enable gpio = 133
030316.845: camsubs: Camera found OK

As far as I understand, v4l can handle two cameras.
viewtopic.php?t=107255

I think I need a dtoverlay to activate the second camera for libcamera framework.

I have the following questions:

Is it possible to use both HQ cameras with libcamera at the same time?

I have found this thread: viewtopic.php?f=43&t=299892&p=1803924&h ... 0#p1803924

As far as I understand, the only thing, what is missing is a dtoverlay for activating the second camera for libcamera framework.
I have never written an own device tree overlay.

Can you point me in the correct direction or provide an example of this?

Any hints are welcome.

Thanks in advance for your help.

[6by9]

dt-blob.bin and vcgencmd get_camera are totally irrelevant for libcamera work.

You are correct that you need an updated overlay that loads imx477 on CAM0 instead of CAM1. I'm working on just that at the moment. https://github.com/raspberrypi/linux/pull/4140 is my work in progress, but includes an overlay for imx477 (I haven't done imx219 or ov5647 yet). Grab the blob from there, compile, and "dtoverlay=imx477,cam0" should load up the imx477 driver on the CAM0 connector using the default pin assignments. (Add "dtoverlay=imx477" as well to get imx477 on CAM1 as well).

However AFAIK libcamera hasn't considered the issue of multiple simultaneous cameras yet, let alone stereoscopic. For a start you'll need a second V4L2 ISP device (/dev/video13-16).

I'll let davidp comment on the magenta cast.
Memory says there was an issue where the three colour channels don't saturate simultaneously, so if green saturates and red and blue don't, then you'll get extra red and blue (ie magenta) in the highlights. Dropping the exposure back slightly using EV is probably the easiest workaround.

[therealdavidp]

Just to answer the magenta cast question: yes, I'm sure someone is lowering all the digital gains on one sensor to try and match the exposure of the other. With green always being the lowest for the HQ cam, once it goes less than one it will be de-saturating green pixels so the highlights will go magenta.

How this is all determined will be well buried in the Broadcom camera stack. I imagine one sensor may simply be following the AGC (i.e. exposure and analogue gain) of the other, and then it seems someone is trying to make one match the other using digital gain. Doing this by lowering the digital gains on one of them would seem ill-advised.

With the HQ cam you have the additional question of the aperture. If these aren't exactly matched you could really see that making this problem very obvious as the exposures for each sensor would have to be driven independently. Clearly matching apertures as precisely as possible would recommended.

[jamaar]

First, thank you 6by9 and therealdavidp for your responses and your help.

@6by9: I have tried out your changed imx477-overlay.dts . For my opinion it is working.

Now the output of

Code: Select all

 v4l2-ctl --list-devices

is:

Code: Select all

bcm2835-codec-decode (platform:bcm2835-codec):
        /dev/video10
        /dev/video11
        /dev/video12

bcm2835-isp (platform:bcm2835-isp):
        /dev/video13
        /dev/video14
        /dev/video15
        /dev/video16

unicam (platform:fe800000.csi):
        /dev/video0
        /dev/video1

unicam (platform:fe801000.csi):
        /dev/video2
        /dev/video3

The devices for video2 and video3 are new. Unfortunately the CameraManager from libcamera seems not to recognize the second camera.

@therealdavidp: I have switched to two single instances of raspistill. I'am unsure how, this bug is caused. I would say, I can see a similar behavior, even on single instances, but in somewhat lesser degree.

Again, thanks for your help.

[jamaar]

I have continued trying to understand, why I cannot use both cameras of Compute Module 4 with libcamera.
I have create this github issue: https://github.com/kbingham/libcamera/issues/24
But it seems, that this was not the right place to answer my questions.

I have debugged the libcamera framework. For my understanding only one camera is recognized by the framework.
If I add debug prints in the CameraManager, I can see only one camera with id:

Code: Select all

/base/soc/i2c0mux/i2c@0/imx477@1a

At the moment I try to understand, where the missing part is. Maybe my knowledge is sufficient to implement something.

Is the missing part in the 4vl kernel driver or is it in the libcamera framework itself?
Can someone guide me in the correct direction?

If I should create a new post for this topic, please give me a hint.

Thanks for your help.