wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Apr 30, 2020 6:43 pm

6by9 wrote:
Thu Apr 30, 2020 2:43 pm
wkeeling wrote:
Wed Apr 29, 2020 3:23 pm
I have written a camera program that basically setup camera and encoder (h.264) components. These components are connected, and the output written by the encoder callback.

With stereo mode off it works at all resolutions. With side-by-side and works at all widths that are multiples of 32 (as noted in the rules at start of thread even if not 32 aligned generates frame with garbage) and any height. With Top-Bottom it only works and certain resolutions (480X864, 1040X1824, 1056X1856, 1080X1920, 1088X1920, 1112X1920, 1120X1920, 1144X1920 and 1152X1920).
The issue is I need TB for the application (race car stream front and rear cams) and need 720X1280 for throughput.

Any ideas on what I am doing wrong?

Program build from userland GitHub download. Source is at https://github.com/wkeeling63/userland/ ... cam/rcam.c. Camera mode is 5. Kernel is 4.19.108-v7+ #1298 SMP.
Sorry, not really got time to test things out at present. Does the same thing work via raspivid?
I did notice something odd going on with the StereoPi rig which I do intend to get back to when I get a chance, and that was related to resolution not being a multiple of something. You may be seeing a similar thing.
yes same in raspivid 480X864 and 1088X1920 works -- and all in between just generate the h264 header frame only.
Willie Keeling

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Apr 30, 2020 6:55 pm

Realizator wrote:
Thu Apr 30, 2020 3:02 pm
6by9 wrote:
Thu Apr 30, 2020 2:43 pm
wkeeling wrote:
Wed Apr 29, 2020 3:23 pm
... With stereo mode off it works at all resolutions. With side-by-side and works at all widths that are multiples of 32 (as noted in the rules at start of thread even if not 32 aligned generates frame with garbage) and any height.
....
Sorry, not really got time to test things out at present. Does the same thing work via raspivid?
I did notice something odd going on with the StereoPi rig which I do intend to get back to when I get a chance, and that was related to resolution not being a multiple of something. You may be seeing a similar thing.
@wkeeling and @6by9
It is mentioned in the PiCamera docs, that horizontal resolution should be multiple of 32, and vertical by 16. @wkeeling, in your post you did not mentioned about vertical resolution test (division by 16), but horizontal only.
for sbs I found the only width a multiple of 32 is needed and any height works. If remember from raspivid code the use VCOS_ALIGN_UP to set height to a multiple of 16 and width to multiple of 32. So you have to use your own code to get other resolutions.


for tb I have found only a few resolutions that work 480X864, 1040X1824, 1056X1856, 1080X1920, 1088X1920, 1112X1920, 11120X1920, 1144X1920 and 1152X1920
Willie Keeling

User avatar
Realizator
Posts: 83
Joined: Thu Jul 14, 2016 12:53 pm

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Fri May 01, 2020 1:20 pm

6by9 wrote:
Thu Apr 30, 2020 2:58 pm
Realizator wrote:
Thu Apr 30, 2020 1:31 pm
As for the brand new Raspberry Pi High Quality Camera - if it has some caveats for the stereoscopic mode? If RPi's GPU will allow us to capture 2x12 = 24 Mpix stereoscopic images? Do we need to fix dt-blob?
Also one more question - if "LED" line on CSI-2 still unused? (or on this module it is possible to use some lines for the cameras synchronization?..)
I must confess to not having tested IMX477 in stereo mode. I only have one here (working from home) so can't test at present. But the theory is certainly that you should be able to combine the two sensors together into a stereo system delivering 24MPix images.
dt-blob is the same as before as you're only defining the control lines.
Ok, great!!!
6by9 wrote:
Thu Apr 30, 2020 2:58 pm
The LED line is indeed still unused. It's actually been removed from the Pi side on the Pi4 so you can't turn the LED of a v1 on even if you wanted to.
Ok, got it!
6by9 wrote:
Thu Apr 30, 2020 2:58 pm
I've got an early production board here, and it has additional pads breaking out GPO, FSTROBE, XVS, and GPI, as well as the I2C. They aren't used by the firmware.
I see these pads on the photos. I hope Raspberry Pi Foundation will keep these pads on a mass-produced edition!
6by9 wrote:
Thu Apr 30, 2020 2:58 pm
It looks like you link the XVS lines between the two sensors. Master sets it to output, slave sets it as an input.
Yes, this is one of our approaches. It's a great feature for the industrial solutions, but a bit tricky for the beginners we're focused on. In any case, I'd like to thank you again for the exposed pads!
6by9 wrote:
Thu Apr 30, 2020 2:58 pm
It's not something that we've investigated, and not really a priority at present. And, no, I can't release the datasheet!
I'm not asking about a full datasheet (google helps us). I just interesting in understanding of the actual CSI-2 connector pinout. If any of the breaking pads are exposed on CSI connector lines?.. (i.e. can we use some tricks without soldering to the breaking pads?)

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Fri May 01, 2020 2:25 pm

Another data point on the TB issue – Tried raspividyuv and it works for all stereo modes at 720X1280, so it looks like the issue is not with the camera component but with the encoder or the connection.
Willie Keeling

matus.jurecka
Posts: 14
Joined: Tue Apr 14, 2020 6:54 am
Location: Martin, Slovakia

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Mon May 11, 2020 2:13 pm

Hi everyone, I did some experiments with camera component in stereoscopic mode. I made tunnelled connection from camera video port to resizer input. When I read resizer output buffer using callback I got picture only from one eye (now I am not sure if it is left or right). Without resizer I have both eyes in camera video output buffer. Are there any restrictions using resizer component with stereoscopic camera?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13126
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Mon May 11, 2020 2:56 pm

matus.jurecka wrote:
Mon May 11, 2020 2:13 pm
Hi everyone, I did some experiments with camera component in stereoscopic mode. I made tunnelled connection from camera video port to resizer input. When I read resizer output buffer using callback I got picture only from one eye (now I am not sure if it is left or right). Without resizer I have both eyes in camera video output buffer. Are there any restrictions using resizer component with stereoscopic camera?
IL or MMAL?
With IL this is the "fun" of proprietary tunnelling hiding stuff. WIth MMAL I'm assuming you've set MMAL_ENCODING_OPAQUE. Either way it presents the images to resize in a way that it's not expecting, and so it only converts on of the two images.

In IL, setting OMX_IndexParamBrcmDisableProprietaryTunnels on the port should make it push plain image buffers. Under MMAL use MMAL_ENCODING_I420.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

matus.jurecka
Posts: 14
Joined: Tue Apr 14, 2020 6:54 am
Location: Martin, Slovakia

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Mon May 11, 2020 7:19 pm

Sorry, I forgot to mention that I am using IL. I set OMX_IndexParamBrcmDisableProprietaryTunnels on camera port 71 and IT WORKS!
Thank you very much for your advice.

farzanehnakhaee
Posts: 4
Joined: Mon Jul 16, 2018 3:04 pm

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sat Jun 06, 2020 7:25 am

Is there any solution for the settings which are just applied for primary camera?
I recently have the problem to set the shutter speed for the secondary camera (It is only applied for the primary camera).
Thanks for your help.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13126
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sat Jun 06, 2020 10:45 am

farzanehnakhaee wrote:
Sat Jun 06, 2020 7:25 am
Is there any solution for the settings which are just applied for primary camera?
I recently have the problem to set the shutter speed for the secondary camera (It is only applied for the primary camera).
Thanks for your help.
I don't have a stereo rig hooked up currently.
Are you just using raspistill -ss <value>, or something more convoluted?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

farzanehnakhaee
Posts: 4
Joined: Mon Jul 16, 2018 3:04 pm

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Sun Jun 07, 2020 4:41 am

6by9 wrote:
Sat Jun 06, 2020 10:45 am
farzanehnakhaee wrote:
Sat Jun 06, 2020 7:25 am
Is there any solution for the settings which are just applied for primary camera?
I recently have the problem to set the shutter speed for the secondary camera (It is only applied for the primary camera).
Thanks for your help.
I don't have a stereo rig hooked up currently.
Are you just using raspistill -ss <value>, or something more convoluted?
Thanks for your reply. I'm using picamera library.
Also in this post, someone has similar problem for sharpness.
viewtopic.php?t=85012&start=50

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Wed Jun 24, 2020 2:36 am

any word on the TB bugs?
Willie Keeling

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13126
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Jul 09, 2020 9:26 am

wkeeling wrote:
Wed Jun 24, 2020 2:36 am
any word on the TB bugs?
Sorry it's taken so long, but I'm having a look now as I happen to have a CM rig set up.

The code that allocates the pool has assumed it can just double the width or height to create an image buffer big enough for 2 channels, but with the funny internal format used for the video encoder that isn't the case. I suspect that the sizes you report just happen to align with sizes where there isn't any additional padding, and therefore the two sizes happen to tally.

I think I know the fix, so we'll see how easily it falls out.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Jul 16, 2020 11:04 pm

6by9 wrote:
Thu Jul 09, 2020 9:26 am
wkeeling wrote:
Wed Jun 24, 2020 2:36 am
any word on the TB bugs?
Sorry it's taken so long, but I'm having a look now as I happen to have a CM rig set up.

The code that allocates the pool has assumed it can just double the width or height to create an image buffer big enough for 2 channels, but with the funny internal format used for the video encoder that isn't the case. I suspect that the sizes you report just happen to align with sizes where there isn't any additional padding, and therefore the two sizes happen to tally.

I think I know the fix, so we'll see how easily it falls out.
Thanks this is great to hear. I have another MMAL question -- can the quantisation of the encoding be varied after the encoder component is created and encoding has started. Would like to dynamically change quantisation in response current cellular bandwidth.
Willie Keeling

fraserbarton
Posts: 1
Joined: Thu Jul 23, 2020 12:52 pm

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Jul 23, 2020 1:00 pm

6by9 wrote:
Sat Jun 06, 2020 10:45 am
farzanehnakhaee wrote:
Sat Jun 06, 2020 7:25 am
Is there any solution for the settings which are just applied for primary camera?
I recently have the problem to set the shutter speed for the secondary camera (It is only applied for the primary camera).
Thanks for your help.
I don't have a stereo rig hooked up currently.
Are you just using raspistill -ss <value>, or something more convoluted?
Interested to hear if you have made any progress on this issue as my project also requires stereo support for setting the shutter speed of the camera.
Similarly to farzanehnakhaee I am using PiCamera e.g. camera.shutter_speed = xxx

User avatar
Realizator
Posts: 83
Joined: Thu Jul 14, 2016 12:53 pm

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Wed Aug 19, 2020 10:39 am

We have a repeatable bug with the magenta coloring of the very bright image parts on the slave camera in the stereoscopic setup.
This bug is repeatable with:
1. V2 official camera, HQ official camera, IMX219-based cameras from other manufacturers.
2. This bug exists for both old firmware (old AWB algorithm) and with the newest AWB algorithm.
3. This issue repeats for both video and still image capturing.
4. All overlighted zones, displayed at the master camera as a white, are displayed at the slave camera as magenta.

I'm attaching a few examples of images taken with this issue. All problems are on the right part of the images.
Are there any ideas on this?
Attachments
imx219-Sainsmart_200degree_fov_test-res.jpg
IMX219 200 degree
imx219-Sainsmart_200degree_fov_test-res.jpg (158.65 KiB) Viewed 2197 times
official-v2-camera.jpg
V2 official camera
official-v2-camera.jpg (93.16 KiB) Viewed 2197 times
HQ-camera-long-exposure-photo-20200818-075816.jpg
HQ official camera
HQ-camera-long-exposure-photo-20200818-075816.jpg (203.4 KiB) Viewed 2197 times

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Wed Sep 16, 2020 10:39 pm

I am seeing one more issue with stereo video (both raspivid and my custom version) – frames are dropped (at about :33 and 1:07 of https://youtu.be/gX_DRFIBqZk). Not sure if the frames are dropped between camera and encoder or between encoder and callback. Or if larger queues would help. Or how to debug this.

Thanks for any ideas you have.

/usr/bin/raspivid -n -ih -vf -hf -i 20 -w 1920 -h 1080 -t 0 -qp 24 -fps 25 -cs 1 -3d tb -o testfile.flv
Willie Keeling

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Thu Sep 17, 2020 1:34 am

wkeeling wrote:
Wed Sep 16, 2020 10:39 pm
I am seeing one more issue with stereo video (both raspivid and my custom version) – frames are dropped (at about :33 and 1:07 of https://youtu.be/gX_DRFIBqZk). Not sure if the frames are dropped between camera and encoder or between encoder and callback. Or if larger queues would help. Or how to debug this.

Thanks for any ideas you have.

/usr/bin/raspivid -n -ih -vf -hf -i 20 -w 1920 -h 1080 -t 0 -qp 24 -fps 25 -cs 1 -3d tb -o testfile.flv
Tried setting camera video output buffer number to 80 no real change. Then upped the encode output buffer to 80 (from 1) and saw some improvement so I up it to 160 and got the following that looks to be free of dropped frames.

https://youtu.be/PW3xMZAh-Y4
Willie Keeling

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Fri Sep 18, 2020 2:28 pm

wkeeling wrote:
Thu Sep 17, 2020 1:34 am
wkeeling wrote:
Wed Sep 16, 2020 10:39 pm
I am seeing one more issue with stereo video (both raspivid and my custom version) – frames are dropped (at about :33 and 1:07 of https://youtu.be/gX_DRFIBqZk). Not sure if the frames are dropped between camera and encoder or between encoder and callback. Or if larger queues would help. Or how to debug this.

Thanks for any ideas you have.

/usr/bin/raspivid -n -ih -vf -hf -i 20 -w 1920 -h 1080 -t 0 -qp 24 -fps 25 -cs 1 -3d tb -o testfile.flv
Tried setting camera video output buffer number to 80 no real change. Then upped the encode output buffer to 80 (from 1) and saw some improvement so I up it to 160 and got the following that looks to be free of dropped frames.

https://youtu.be/PW3xMZAh-Y4
Sorry more testing show the bigger buffers for camera and encoder did not fixed the issue :(
Willie Keeling

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13126
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Fri Sep 18, 2020 4:19 pm

Save the pts values of each buffer to confirm how smoothly (or not) the stream is. Add "-pts pts.txt".

The output of the encoder is unlikely to have much effect unless you're being very slow in saving data - the codec has a 2MB output FIFO that would have to fill before it backed up. Dropping encoded frames would also be "a bad thing" as they include temporal prediction.

You can try increasing the number of buffers allocated to the ISP output pool, as these are the same buffers fed to the input of the encoder.
Within RaspiVid.c, create_camera_component has a clause

Code: Select all

   //  set up the camera configuration
   {
      MMAL_PARAMETER_CAMERA_CONFIG_T cam_config =
      {
         { MMAL_PARAMETER_CAMERA_CONFIG, sizeof(cam_config) },
         .max_stills_w = state->common_settings.width,
         .max_stills_h = state->common_settings.height,
         .stills_yuv422 = 0,
         .one_shot_stills = 0,
         .max_preview_video_w = state->common_settings.width,
         .max_preview_video_h = state->common_settings.height,
         .num_preview_video_frames = 3 + vcos_max(0, (state->framerate-30)/10), <<<<<< INCREASE ME!
         .stills_capture_circular_buffer_height = 0,
         .fast_preview_resume = 0,
         .use_stc_timestamp = MMAL_PARAM_TIMESTAMP_MODE_RAW_STC
      };
      mmal_port_parameter_set(camera->control, &cam_config.hdr);
   }
I'm suspecting that actually one sensor has reset and stopped producing data. Because you're in stereo mode this will stop both sensor outputs from being combined into the final image.
There's a timeout within the firmware on not receiving an input frame that is set at the higher of 3 frame periods, or 400ms. With your request of 25fps, 3 frame periods would be 120ms, so 400ms would be greater. When this timeout fires, it stops and powers down the sensors and then fully reinitialises them. The reset is going to take 100ms or so as it has to reconfigure the sensor, so looking at the PTS values to see how big the discontinuity is would be able to confirm this hypothesis.
If this fires 5 times with no frames being produced on any attempt, we produce the infamous message of "No sensor data received, check all connections including the SUNNY one".

How long are your camera cables? And how well routed away from sources of interference?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Mon Sep 21, 2020 11:58 pm

6by9 wrote:
Fri Sep 18, 2020 4:19 pm
Save the pts values of each buffer to confirm how smoothly (or not) the stream is. Add "-pts pts.txt".

The output of the encoder is unlikely to have much effect unless you're being very slow in saving data - the codec has a 2MB output FIFO that would have to fill before it backed up. Dropping encoded frames would also be "a bad thing" as they include temporal prediction.

You can try increasing the number of buffers allocated to the ISP output pool, as these are the same buffers fed to the input of the encoder.
Within RaspiVid.c, create_camera_component has a clause

Code: Select all

   //  set up the camera configuration
   {
      MMAL_PARAMETER_CAMERA_CONFIG_T cam_config =
      {
         { MMAL_PARAMETER_CAMERA_CONFIG, sizeof(cam_config) },
         .max_stills_w = state->common_settings.width,
         .max_stills_h = state->common_settings.height,
         .stills_yuv422 = 0,
         .one_shot_stills = 0,
         .max_preview_video_w = state->common_settings.width,
         .max_preview_video_h = state->common_settings.height,
         .num_preview_video_frames = 3 + vcos_max(0, (state->framerate-30)/10), <<<<<< INCREASE ME!
         .stills_capture_circular_buffer_height = 0,
         .fast_preview_resume = 0,
         .use_stc_timestamp = MMAL_PARAM_TIMESTAMP_MODE_RAW_STC
      };
      mmal_port_parameter_set(camera->control, &cam_config.hdr);
   }
I'm suspecting that actually one sensor has reset and stopped producing data. Because you're in stereo mode this will stop both sensor outputs from being combined into the final image.
There's a timeout within the firmware on not receiving an input frame that is set at the higher of 3 frame periods, or 400ms. With your request of 25fps, 3 frame periods would be 120ms, so 400ms would be greater. When this timeout fires, it stops and powers down the sensors and then fully reinitialises them. The reset is going to take 100ms or so as it has to reconfigure the sensor, so looking at the PTS values to see how big the discontinuity is would be able to confirm this hypothesis.
If this fires 5 times with no frames being produced on any attempt, we produce the infamous message of "No sensor data received, check all connections including the SUNNY one".

How long are your camera cables? And how well routed away from sources of interference?
added the write of PTS and found the following dropped (only looked for the first 2 in a 2 minute video).

.
.
.
36862.756
36902.736
36942.718
39941.316
40021.277
40061.260
.
.
.
63530.285
63570.266
63610.250
64449.857
64489.840
64529.820
.
.
.
one is long 3 seconds and the next is about half second.

The cables are 15cm (cam 0) and 30cm (cam 1). I have a 4G/LTE hat that might be of interference so I turned it off for this test. The CM is mounted in a Waveshare Compute Module PoE board.

How/Where do I find the message you posted? How can I shield the ribbon cables and sensor? Should I increase the num_preview_video_frames? If so how much to you think?

Thanks
Willie Keeling

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Tue Sep 22, 2020 3:40 pm

wkeeling wrote:
Mon Sep 21, 2020 11:58 pm
6by9 wrote:
Fri Sep 18, 2020 4:19 pm
Save the pts values of each buffer to confirm how smoothly (or not) the stream is. Add "-pts pts.txt".

The output of the encoder is unlikely to have much effect unless you're being very slow in saving data - the codec has a 2MB output FIFO that would have to fill before it backed up. Dropping encoded frames would also be "a bad thing" as they include temporal prediction.

You can try increasing the number of buffers allocated to the ISP output pool, as these are the same buffers fed to the input of the encoder.
Within RaspiVid.c, create_camera_component has a clause

Code: Select all

   //  set up the camera configuration
   {
      MMAL_PARAMETER_CAMERA_CONFIG_T cam_config =
      {
         { MMAL_PARAMETER_CAMERA_CONFIG, sizeof(cam_config) },
         .max_stills_w = state->common_settings.width,
         .max_stills_h = state->common_settings.height,
         .stills_yuv422 = 0,
         .one_shot_stills = 0,
         .max_preview_video_w = state->common_settings.width,
         .max_preview_video_h = state->common_settings.height,
         .num_preview_video_frames = 3 + vcos_max(0, (state->framerate-30)/10), <<<<<< INCREASE ME!
         .stills_capture_circular_buffer_height = 0,
         .fast_preview_resume = 0,
         .use_stc_timestamp = MMAL_PARAM_TIMESTAMP_MODE_RAW_STC
      };
      mmal_port_parameter_set(camera->control, &cam_config.hdr);
   }
I'm suspecting that actually one sensor has reset and stopped producing data. Because you're in stereo mode this will stop both sensor outputs from being combined into the final image.
There's a timeout within the firmware on not receiving an input frame that is set at the higher of 3 frame periods, or 400ms. With your request of 25fps, 3 frame periods would be 120ms, so 400ms would be greater. When this timeout fires, it stops and powers down the sensors and then fully reinitialises them. The reset is going to take 100ms or so as it has to reconfigure the sensor, so looking at the PTS values to see how big the discontinuity is would be able to confirm this hypothesis.
If this fires 5 times with no frames being produced on any attempt, we produce the infamous message of "No sensor data received, check all connections including the SUNNY one".

How long are your camera cables? And how well routed away from sources of interference?
added the write of PTS and found the following dropped (only looked for the first 2 in a 2 minute video).

.
.
.
36862.756
36902.736
36942.718
39941.316
40021.277
40061.260
.
.
.
63530.285
63570.266
63610.250
64449.857
64489.840
64529.820
.
.
.
one is long 3 seconds and the next is about half second.

The cables are 15cm (cam 0) and 30cm (cam 1). I have a 4G/LTE hat that might be of interference so I turned it off for this test. The CM is mounted in a Waveshare Compute Module PoE board.

How/Where do I find the message you posted? How can I shield the ribbon cables and sensor? Should I increase the num_preview_video_frames? If so how much to you think?

Thanks
Thanks 6by9,
The issue was all me. My origami folding of the ribbon cables either damaged them or introduced interference. I have replaced the cables and now the PTS counts match what they should be for the time recorded within a few frames. Sorry for my incorrect assumption that this was a bug.

Image
Attachments
ribbon_cable.JPG
ribbon_cable.JPG (62.28 KiB) Viewed 1912 times
Willie Keeling

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13126
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Tue Sep 22, 2020 3:54 pm

:shock: :shock: :shock: :shock: :shock:

I'm surprised you got anything down those cables reliably!
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Wed Oct 28, 2020 11:24 pm

I replaced cables and it was/is not the issue. Reinstalled Raspbian to eliminate any of the other software and changes I had made to the config and retested still had the issue of dropped frames (stereo and single cam). I tested single cam on another pi with no lost frames. So, it looks to be a hardware issue that is causing the system to hang for from 1 to 4 seconds and since the encoder output queue of raspivid has only 2 buffers (and I am capturing 25 FPS) these frames are lost. I have upped the output queue of my program to a very high value (200) and this solves the lost of video frames. This work around is not a viable solution for 2 reasons 1) this is a live streaming program so the delay caused by the hangs would affect the live stream smoothness 2) I am using ALSA for my sound and it’s buffer can not be increase enough to not loss sound frames. It is also curious that this hang only affect CPU and the GPU still captures, encodes and queues the frames.

I tried to do this project on the cheap so I am using the Wave Share Compute Module PoE Board (https://www.waveshare.com/wiki/Compute_Module_PoE_Board) as my CM3 carrier board for prototyping and I suspect this is issue but will not be able to test until I buy some other CM IO board.

I have CM4 and CM4 IO board on order, but they will not ship until end of November and since Pi trading went will the 22-pin camera connector will still need to solve that before trying that. Don’t know if the old CMCDA adaptor boards will work or if they can still be purchased. Not sure why they did this again on the CM4 IO board?

If I get impatient I will either have to order StereoPi ($80) or CM3 IO board ($100).
Willie Keeling

wkeeling
Posts: 150
Joined: Fri Aug 25, 2017 2:16 pm
Location: Houston Texas

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Tue Nov 10, 2020 3:30 pm

Thanks for the fix for top/bottom H.264 encoding bug it tests fine at normal resolutions (480X864, 720X1280 and 1080X1920) with firmware 9ae94aeb1241adffec586ae451862e3fa6e05e6a. Most of the other resolutions deliver garbage images.

I have read the rules at the beginning of this thread but I guess I am not very bright :) as I don’t understand the resolution rules for h.264 encoded video. What are the rules on valid height and width? Do they differ for SideBySide and TopBottom?

Please remember that I am not very bright.
Thanks,
William
Willie Keeling

ChristianIV
Posts: 1
Joined: Tue Nov 17, 2020 7:56 am

Re: Stereoscopic camera capture (for COMPUTE MODULE) - now implemented (2014).

Tue Nov 17, 2020 8:36 am

Realizator wrote:
Wed Aug 19, 2020 10:39 am
We have a repeatable bug with the magenta coloring of the very bright image parts on the slave camera in the stereoscopic setup.
This bug is repeatable with:
1. V2 official camera, HQ official camera, IMX219-based cameras from other manufacturers.
2. This bug exists for both old firmware (old AWB algorithm) and with the newest AWB algorithm.
3. This issue repeats for both video and still image capturing.
4. All overlighted zones, displayed at the master camera as a white, are displayed at the slave camera as magenta.

I'm attaching a few examples of images taken with this issue. All problems are on the right part of the images.
Are there any ideas on this?
I'm having the same issue with the Stereo Pi, the official HQ Pi Cameras and the 6mm wide-angle lenses. After performing sudo update and sudo upgrade the magenta colour seems to be less intrusive, but still present. Attached below is an example.

Does anyone know how to fix this issue?
stereo1.jpg
HQ Pi Cam Stereo Image of Ceiling Lights
stereo1.jpg (148.57 KiB) Viewed 1358 times

Return to “Camera board”