peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Libcamera Dual Camera Setup

Thu Jun 23, 2022 7:54 pm

Hello all,

I am working on implementing dual camera support into libcamera for a VR application I am working on. I have already modified the EGL preview to use my shader and am now working on adding in a second camera. If I run the following command, I get the desired result and performance, but it is in two separate windows since I have to call libcamera twice.

Code: Select all

./libcamera-hello --width 960 --height 1080 -p 0,0,960,1080 --camera 0 -t 0 & ./libcamera-hello --width 960 --height 1080 -p 960,0,960,1080 --camera 1 -t 0
I would like to display both camera feeds in the same X11 preview window and I think I have figured out the general track, but am stuck on implementation. My code for the following changes and implementations is found here.

My first attempt was to just duplicate the libcamera_hello calls for app.OpenCamera(), app.ConfigureViewfinder() and app.StartCamera(), but that did not work. Next step was to look into each function and see how it works. I was able to modify OpenCamera() to open both cameras like so:

Code: Select all

	std::string const &cam_id = cameras[0]->id();
	camera_ = camera_manager_->get(cam_id);
	if (!camera_)
		throw std::runtime_error("failed to find camera " + cam_id);

	if (camera_->acquire())
		throw std::runtime_error("failed to acquire camera " + cam_id);
	camera_acquired_ = true;
	
	std::string const &cam_id2 = cameras[1]->id();
	camera2_ = camera_manager_->get(cam_id2);
	if (!camera2_)
		throw std::runtime_error("failed to find camera " + cam_id2);
     	if (camera2_->acquire())
		throw std::runtime_error("failed to acquire camera " + cam_id2);
Looking through ConfigureViewfinder(), I only added in an additional configuration_ and post_processor_ so that I could call post_processor_.Configure() again since this appears to be the configure() call for the libcamera cameras.

ConfigureViewfinder() then calls SetupCapture(), where I duplicated the code for creating the frame buffers. This then calls startPreview(), which in turn calls previewThread(). In previewThread() I once again duplicate the frame buffer code so that when I call the show function, I now pass in the framebuffer from each camera, like so:

Code: Select all

preview_->Show(fd, span, info, fd2, span2, info2);
The next modification is to StartCamera(), where I add in the following lines:

Code: Select all

	if (camera2_->start(&controls_))
		throw std::runtime_error("failed to start camera");
	camera2_->requestCompleted.connect(this, &LibcameraApp::requestComplete);

	for (std::unique_ptr<Request> &request : requests_)
	{
		if (camera2_->queueRequest(request.get()) < 0)
			throw std::runtime_error("Failed to queue request");
	}
Finally, I modified the makeRequests(), queueRequest(), and requestComplete() functions to include both camera feeds. Everything compiles without an issue, but I cannot get both camera feeds to show. Instead, I get the following error message in the console:
Camera camera.cpp:528 Camera in Acquired state trying start() requiring state Configured
[65:06:11.162689446] [18614] ERROR Camera camera.cpp:549 Camera in Running state trying release() requiring state between Available and Configured
[65:06:11.162989720] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video0[16:cap]: Unable to request 0 buffers: Device or resource busy
[65:06:11.163097422] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video13[17:out]: Unable to request 0 buffers: Device or resource busy
[65:06:11.163161810] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video14[18:cap]: Unable to request 0 buffers: Device or resource busy
[65:06:11.163230198] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video15[19:cap]: Unable to request 0 buffers: Device or resource busy
[65:06:11.163297975] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video16[20:cap]: Unable to request 0 buffers: Device or resource busy
[65:06:11.163391381] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video16[20:cap]: Unable to request 0 buffers: Device or resource busy
[65:06:11.165828311] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video15[19:cap]: Unable to request 0 buffers: Device or resource busy
[65:06:11.168345184] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video14[18:cap]: Unable to request 0 buffers: Device or resource busy
[65:06:11.173952848] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video13[17:out]: Unable to request 0 buffers: Device or resource busy
[65:06:11.176534016] [18615] ERROR V4L2 v4l2_videodevice.cpp:1211 /dev/video0[16:cap]: Unable to request 0 buffers: Device or resource busy
terminate called without an active exception
Aborted
It appears from the libcamera documentation that the second camera is aquired but cannot start because it was never configured. I thought this was handled by the creation of post_processor2_ and calling post_processor2_.Configure() in the ConfigureViewfinder() function, but I may have set it up wrong. Any ideas on how to correctly configure the second camera?

Note: I know that dual camera support is not implemented yet, but I am trying to implement it based on previous experience using dual cameras with MMAL and reading through the libcamera source code and documentation. Any pointers or tests anyone has done would be super helpful and would be much appreciated!

peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Re: Libcamera Dual Camera Setup

Mon Jul 04, 2022 11:49 pm

I have figured out the camera initialization issue from the previous post. Calling post_processor_.Configure() did not configure the cameras as I initially thought, they are instead initialized like so:

Code: Select all

	if (camera_->configure(configuration_.get()) < 0)
		throw std::runtime_error("failed to configure streams for camera 1");
	if (camera2_->configure(configuration2_.get()) < 0)
		throw std::runtime_error("failed to configure streams for camera 2");
I am now able to get both cameras to start up and run without any issues, but I cannot get both to display to the screen as I am not entirely sure how to make a framebuffer for each camera or put both camera feeds into the same framebuffer, as mentioned by 6by9 in this thread.

Here is the steps I have taken thus far to setup the cameras by modifying the libcamera_app source code to include two cameras:
  • 1. Acquire each camera using camera_->acquire()
    2. Create a configuration for each camera using camera_->generateConfiguration()
    3. Configure each camera using camera_->configure()
    4. Allocate individual buffers for each camera using FrameBufferAllocator(camera_)
    5. Start the preview thread - I don't know how to properly modify this thread
    6. Initialize requests using makeRequests() - this function is duplicated to handle both cameras
    7. Start each camera using camera_->start(&controls_)
    8. Connect the requests to each camera using camera_->requestCompleted.connect()
    9. Queue requests to each camera using camera_->queueRequest(request.get())
    10. Get completed requests and show them to the preview thread - duplicated the LibcameraApp::requestComplete() function so each camera has its own requestComplete() function
I would like to feed the Show() function in the preview thread the individual framebuffer from each camera as I can then easily display each camera to half the screen using the glViewport() function. Unfortunately, I'm not sure I entirely understand the preview thread as it has no clear mention of the camera or the frame_buffers_ allocated in SetupCapture() and filled in makeRequests(). Therefore, I am not sure what I need to duplicate in order to access the individual framebuffer of each camera.

I tried duplicating PreviewItem item so I could make duplicates of the StreamInfo, buffer, and span, but that only led to segmentation fault and I cannot see a clear way that it accesses the frame_buffers_ created and filled earlier on.

Any idea on how to access both framebuffers in the preview thread or combine them like 6by9 suggested?

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1072
Joined: Tue Jan 07, 2020 9:15 am

Re: Libcamera Dual Camera Setup

Tue Jul 05, 2022 8:48 am

Hi, thanks for posting this. There's a lot going on here and it will probably take a while to understand what you've been doing! But I can try and give you some pointers...
I tried duplicating PreviewItem item so I could make duplicates of the StreamInfo, buffer, and span, but that only led to segmentation fault and I cannot see a clear way that it accesses the frame_buffers_ created and filled earlier on.
I'm not sure why there would be a problem here. The PreviewItem takes a copy of the completed request shared_ptr, and a pointer to the stream, so I would have expected one could make this work. I assume when completed requests get recycled by the custom deleter, they get queued back to the right camera? Maybe building with debug and fishing out a stack trace would help?
Any idea on how to access both framebuffers in the preview thread or combine them like 6by9 suggested?
Frame buffers are accessed through their file descriptors. In egl_preview.cpp, it's the EglPreview::makeBuffer method that creates an OpenGL texture that points to the DMA buf, though its "fd". I would hope that if you supply the right file descriptor and dimensions that it would work...

peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Re: Libcamera Dual Camera Setup

Tue Jul 05, 2022 12:16 pm

Thank you so much for the reply therealdavidp!
The PreviewItem takes a copy of the completed request shared_ptr, and a pointer to the stream
Is the stream pointer the same one setup in CongfigureViewfinder() ? If this is the case, then I only have one stream setup using the configuration for one camera. I would assume I would need to make a second stream using the configuration for the second camera?
I assume when completed requests get recycled by the custom deleter, they get queued back to the right camera?
Correct! I modified queueRequest() to take in a camera argument so that when called, it queues to the correct camera. I also duplicated the code inside of makeRequests to update the buffer for both cameras, as well as made a copy of requestComplete() for the addition of the second camera.
In egl_preview.cpp, it's the EglPreview::makeBuffer method that creates an OpenGL texture that points to the DMA buf, though its "fd". I would hope that if you supply the right file descriptor and dimensions that it would work...
In my modified EGL code for the OpenGL shader, I have the correct descriptor and dimensions , which I will use for the dual camera feed! To clarify a little bit, this is my intent: I would like to pass in a second fd, info, and span into EglPreview::Show() for the second camera. Then, I would duplicate the EglPreview::makeBuffer call by copying these lines for the second fd, info, and span.

Code: Select all

	Buffer &buffer = buffers_[fd];
	if (buffer.fd == -1)
		makeBuffer(fd, span.size(), info, buffer);
From there, I can use glViewport to display the buffer from each camera to one half of the screen and create a stereoscopic camera feed!

peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Re: Libcamera Dual Camera Setup

Fri Jul 08, 2022 2:04 am

Going with my hunch that I need a stream for each camera and I have run into an issue with adding in the second stream. If this is not the correct route to go, please let me know!
  • 1. Created a streams2_ in ConfigureViewfinder() with the configuration for camera 2
    2. Added a second StreamInfo parameter to LibcameraApp::GetStream() and LibcameraApp:ViewfinderStream()
    3. Duplicated the streams_ code in LibcameraApp::GetMainStream() for streams2_
    3. Duplicated PreviewItem item to create item2 in LibcameraApp:previewThread()
    4. Created info2, buffer2, and span2 for item2
I get a segmentation fault error when creating info2, buffer2, and span2 and I am pretty sure this is because it is still trying to access the preview_item_ associated with the first camera. Unfortunately preview_item_ is created in LibcameraApp::ShowPreview(), which is called by the libcamera_hello app like so:

Code: Select all

app.ShowPreview(completed_request, app.ViewfinderStream());
I'm not sure how to include the second stream I added to ViewfinderStream() so that I can create a preview_item2_ for the second camera. ShowPreview() only takes in one stream argument, which is given as ViewfinderStream(), so how can I add the second stream for the other camera?

Or is this what 6by9 meant by combining the steams from the two cameras?

peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Re: Libcamera Dual Camera Setup

Sun Aug 07, 2022 9:05 pm

I've continued my attempt of modifying libcamera-hello to display stereoscopic feed to no avail. I have tried duplicating every function and call to mimic running libcamera-hello twice (once for each camera) and a whole range of other duplications and attempts to get it to work. I figured that duplicating every function and passing in two completed requests from libcamera-hello would get the job done as each camera would have its own requests. Alas, only one camera actually filled its buffers and the other did not, causing a segmentation fault.

After extensive testing, I think the problem lies all the way in the beginning with the StartCamera() function, as it appears that camera2_ never creates its framebuffers. Here is the code I used to connect the cameras to their completedrequests and allocated the buffers for the cameras to use later on:

Code: Select all

	
	camera_->requestCompleted.connect(this, &LibcameraApp::requestComplete);
	camera2_->requestCompleted.connect(this, &LibcameraApp::requestComplete2);
	
	for (std::unique_ptr<Request> &request : requests_)
	{
		std::cout << camera_->queueRequest(request.get()) << std::endl;
	}
	
	for (std::unique_ptr<Request> &request : requests2_)
	{
		std::cout << camera2_->queueRequest(request.get()) << std::endl;
	}
The print statements read "0, 0, 0,0" for camera_, while nothing pops up for camera2_. If I switch camera2_ to use requests_ instead of requests2_, libcamera complains that camera2_ did not create the requests and the print statement is "-18, -18, -18, -18", which would normally cause a crash if I had it throw an error like libcamera normally does.

I'm not sure what to do because using requests2_ causes camera2_ to never queue any requests, while using requests_ causing a libcamera error. Any suggestions?

Update:
I determined that I was doing an incorrect creation of the buffers in makeRequests() as I had requests_ and requests2_ created in two different while loops, but only the first while loop actually ran. After fixing this, I now get requests_ and requests2_ both filled and working properly!

I think I am almost done with getting stereoscopic cameras setup, but I have run into another issue. As a part of debugging, I am printing out the size of completed_requests for each camera (in the requestComplete() functions), as well as the buffer associated with each camera in the previewThread(). When I do this, completed_requests_ for camera1 gets up to 4 and then completed_requests2_ for camera2 goes to 2 before crashing. The framebuffer for camera1 is full both times before crashing, while the framebuffer for camera2 is full initially and then becomes zero, which I think is causing the crash. Both initial framebuffers are also different, which should mean that I am getting the camera feed from each camera :D

I'm not sure what is causing the second framebuffer to empty and not refill, but I think once that is solved, both camera feeds should be displayed.

Here is the entire code for the dual cameras

peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Re: Libcamera Dual Camera Setup

Mon Aug 22, 2022 12:24 am

Another weekend spent working on trying to add dual camera support and I am even more lost than before. If anyone is reading this and has any ideas, please comment them so I don't feel like I'm talking to the void :lol:

I found libcamera's simple cam application and I added in a second camera as I have done with the libcamera-hello application. The printed results were astonishing, I got each framebuffer filled for each camera with no segmentation faults or any errors!

Unfortunately, the application doesn't have any preview window, so I'm not entirely sure if it's actually working or not (I'll explain further below). It appears that both cameras are returning their own data, because if I hold my hand above one camera, the lux between the two is very different. One reads ~2 and the other reads ~50.

To get this code to work, I only had to make duplicate functions for everything pertaining to the camera itself:
  • 1. camera instance
    3. camera configuration
    4. FrameBufferAllocator
    5. camera requests
    6. RequestComplete function
    7. ProcessRequest function (looks similar to queueRequest in libcamera_app but not 100%)
Using this information, I redid the libcamera-app dual camera setup and what do you know, no segmentation faults or errors. However, there are not two camera feeds, but instead the camera feed from one camera being passed around to both. When I print the buffer and fd for each camera in the previewThread(), they are the exact same values.

Image

I know both cameras are queuing and completing requests, because I have a print statement in each loop to print when a camera has queued/completed a request and the print statements from both camera's loops are being displayed as it runs.

I am passing each camera the same completed_request from the event_loop in libcamera_hello, but this shouldn't cause any issues since the requests coming from each camera in simple-cam were the exact same. If I try to create another completed_request in the event_loop by creating another msg and passing it to ShowPreview(), I get different values for the buffers and fd, but I also get a segmentation fault and the camera feed that displays briefly is the same as before, one camera feed duplicated.

I tried modifying previewDoneCallback like so:

Code: Select all

void LibcameraApp::previewDoneCallback(int fd, int fd2)
{
	std::lock_guard<std::mutex> lock(preview_mutex_);
	auto it = preview_completed_requests_.find(fd);
	if (it == preview_completed_requests_.end())
		throw std::runtime_error("previewDoneCallback: missing fd " + std::to_string(fd));
	preview_completed_requests_.erase(it); // drop shared_ptr reference
	
	auto it2 = preview_completed_requests_.find(fd2);
	if (it2 == preview_completed_requests_.end())
		throw std::runtime_error("previewDoneCallback: missing fd2 " + std::to_string(fd2));
	preview_completed_requests_.erase(it2); // drop shared_ptr reference
}
and EGLPreviewShow() like so:

Code: Select all

void EglPreview::Show(int fd, libcamera::Span<uint8_t> span, StreamInfo const &info, int fd2, libcamera::Span<uint8_t> span2, StreamInfo const &info2)
{
	Buffer &buffer = buffers_[fd];
	if (buffer.fd == -1)
		makeBuffer(fd, span.size(), info, buffer);

	Buffer &buffer2 = buffers2_[fd2];
	if (buffer2.fd == -1)
		makeBuffer(fd2, span2.size(), info2, buffer2);
		
	glClearColor(0, 0, 0, 0);
	glClear(GL_COLOR_BUFFER_BIT);
	glBindTexture(GL_TEXTURE_EXTERNAL_OES, buffer.texture);
	glViewport(0, 0, 960, 1080);
    //glBindFramebuffer(GL_FRAMEBUFFER, 0);
	SSQuad->draw();
	
	glBindTexture(GL_TEXTURE_EXTERNAL_OES, buffer2.texture);
	glViewport(960, 0, 960, 1080);
	glBindFramebuffer(GL_FRAMEBUFFER, 0);
	SSQuad->draw();
	//glDrawArrays(GL_TRIANGLE_FAN, 0, 4);
	EGLBoolean success [[maybe_unused]] = eglSwapBuffers(egl_display_, egl_surface_);
	if (last_fd_ >= 0)
		done_callback_(last_fd_);
	last_fd_ = fd;
	
	if (last_fd2_ >= 0)
		done_callback_(last_fd2_);
	last_fd2_ = fd2;
}
to accommodate for the different fd values created when passing another completed_request into the ShowPreview, but I still got a segmentation fault. I'm not sure if I need to somehow also do another buffer callback or if this is even the right way to go. I have a feeling it's not since simple-cam works with the same request for each camera, but I cannot verify that without getting a preview feed up.

I am truly lost and have tried basically every idea I can think of to get both cameras displaying at once. Does anyone have any ideas or thoughts or is even reading this anymore?

kneave
Posts: 21
Joined: Sun May 03, 2015 1:39 pm
Location: Reading, UK

Re: Libcamera Dual Camera Setup

Sun Oct 16, 2022 11:37 pm

I've been trying to get my StereoPi board back up and running using alternatives to mmal since the 64bit buster beta. In my case I just need to get the images to publish to ROS (Robot operating system) so you may have already solved that!

I don't have the brain power to get my head around what you've done right now but thought I'det you know this is good work and keep at it!

kneave
Posts: 21
Joined: Sun May 03, 2015 1:39 pm
Location: Reading, UK

Re: Libcamera Dual Camera Setup

Sun Oct 16, 2022 11:38 pm

Could that be an option by the way? One process to generate the stream and another that displays it rather than doing both in one?

peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Re: Libcamera Dual Camera Setup

Mon Oct 24, 2022 4:08 am

Hi Kneave,

I plan on modifying libcamera-still to also have dual-camera support as my next step in the troubleshooting process. If I can get an image output from each camera, then I know something is going wrong in the preview code and not elsewhere. Adding support to libcamera-still should satisfy your need for capturing images to publish to the ROS, so keep an eye on my repsotory :D
Could that be an option by the way? One process to generate the stream and another that displays it rather than doing both in one?
Possibly, but I think something is going awry in the way the raspberry pi libcamera apps handles previews since they are intended for displaying the feed from one camera only.

I'm pretty sure I have the individual camera's setup correctly with how they handle queueing and completing requests as I followed the official state flow chart from libcamera and have used print statements to verify this behavior. Unfortunately, I'm not 100% sure how to handle the preview window's control over requests and am probably missing something simple that is leading to the behavior I experience.

User avatar
RichShumaker
Posts: 438
Joined: Tue Jul 31, 2012 4:16 pm
Location: USA

Re: Libcamera Dual Camera Setup

Thu Feb 23, 2023 3:05 am

Did you ever figure this out?

One of my next steps with my 2 HQ cameras was to do some Pi Experimentation and I have a Compute Module setup to run both cameras.

I found this thread as I am simply trying to raise the HDMI 8 bit output from 4:2:0 to 4:2:2.

Have an amazing day.
Rich Shumaker
http://www.instructables.com/id/Pi-Zero-W-NoIR-8MP-Camera-Build-Overview-Introduct/
Pi HQ in Effect on my Podcast
https://www.youtube.com/channel/UCXz1Cr7vBrP3yu0LLls3Uhg

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 33212
Joined: Sat Jul 30, 2011 7:41 pm

Re: Libcamera Dual Camera Setup

Thu Feb 23, 2023 8:11 am

There is a dual camera picamera2 example in the picamera2 repo.

https://github.com/raspberrypi/picamera ... pp_dual.py
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.
Meet me and the Pi team at Embedded World in Nürnberg, April 9th-11th, 2024
Hall 3A Stand 138
https://events.raspberrypi.com/official/24d5151d-dd0f-483d-88a7-a5fddaa5c554

User avatar
RichShumaker
Posts: 438
Joined: Tue Jul 31, 2012 4:16 pm
Location: USA

Re: Libcamera Dual Camera Setup

Thu Feb 23, 2023 8:14 am

jamesh wrote:
Thu Feb 23, 2023 8:11 am
There is a dual camera picamera2 example in the picamera2 repo.

https://github.com/raspberrypi/picamera ... pp_dual.py
Thanks Jamesh.

Is this a realtime output from the HDMI by any chance?
Rich Shumaker
http://www.instructables.com/id/Pi-Zero-W-NoIR-8MP-Camera-Build-Overview-Introduct/
Pi HQ in Effect on my Podcast
https://www.youtube.com/channel/UCXz1Cr7vBrP3yu0LLls3Uhg

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 33212
Joined: Sat Jul 30, 2011 7:41 pm

Re: Libcamera Dual Camera Setup

Thu Feb 23, 2023 3:43 pm

RichShumaker wrote:
Thu Feb 23, 2023 8:14 am
jamesh wrote:
Thu Feb 23, 2023 8:11 am
There is a dual camera picamera2 example in the picamera2 repo.

https://github.com/raspberrypi/picamera ... pp_dual.py
Thanks Jamesh.

Is this a realtime output from the HDMI by any chance?
Real time output in a window on the desktop, which could be HDMI or an LCD panel.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.
Meet me and the Pi team at Embedded World in Nürnberg, April 9th-11th, 2024
Hall 3A Stand 138
https://events.raspberrypi.com/official/24d5151d-dd0f-483d-88a7-a5fddaa5c554

peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Re: Libcamera Dual Camera Setup

Fri Feb 24, 2023 1:13 am

RichShumaker wrote: Did you ever figure this out?

One of my next steps with my 2 HQ cameras was to do some Pi Experimentation and I have a Compute Module setup to run both cameras.

I found this thread as I am simply trying to raise the HDMI 8 bit output from 4:2:0 to 4:2:2.

Have an amazing day.
I did end up figuring it out, just not using the default libcamera-apps like I initially sought to do. Instead, I took the simple-cam example from libcamera and modified it to support dual-cameras. As it currently stands, it supports a 1080p window (both cameras are in one window) at 60fps in both X11 and DRM rendering (i.e. supports Desktop mode and the console window/lite OS). It also has a few command line options to change some camera settings and the preview window size. The pipeline itself is very minimal, so it's very easy to make changes to suit your application and needs.

The repository for my dual-camera simple-cam application can be found here

As Jamesh pointed out, you can also use picamera2, but I figured I'd give you another option as well. Best of luck!

User avatar
MRV
Posts: 214
Joined: Thu Jul 19, 2018 3:09 am

Re: Libcamera Dual Camera Setup

Wed Mar 08, 2023 11:10 am

has there been any progress on getting a “-3D SBS” type stereo video recording feature in Libcamera? (Like Raspivid)

peytonicmaster
Posts: 99
Joined: Wed Mar 03, 2021 5:07 pm

Re: Libcamera Dual Camera Setup

Mon Mar 20, 2023 3:28 am

MRV wrote: has there been any progress on getting a “-3D SBS” type stereo video recording feature in Libcamera? (Like Raspivid)
I haven't done anything in regards to saving the video feed, just for displaying it. With that being said, the pipeline in my simple-cam application is easily modifiable to get the results that you desire, so adding the ability to save video feed should be pretty straightforward. This function is where you'll have direct access to the buffers and may be a point of consideration for saving video feed.

Code: Select all

static void processRequest(Request *request)
{	
	const Request::BufferMap &buffers = request->buffers();
	for (auto bufferPair : buffers) {
		const Stream *stream = bufferPair.first;
		FrameBuffer *buffer = bufferPair.second;
		StreamConfiguration const &cfg = stream->configuration();
		int fd = buffer->planes()[0].fd.get();
		
               //Send buffer to OpenGL to be displayed
		makeBuffer(fd, cfg, buffer, 1);
	}

	/* Re-queue the Request to the camera. */	
	request->reuse(Request::ReuseBuffers);
	camera->queueRequest(request);
}
The following line of code will mmap the buffer and give you access to do whatever you need with it

Code: Select all

uint8_t *ptr = static_cast<uint8_t *>(mmap(NULL, buffer->planes()[0].length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0));
I'm not entirely sure how you would handle the synchronization aspect as my current synchronization method is handled in the event loop, which doesn't have direct access to the frame buffers and calls an OpenGL render function that uses the existing textures rendered from the processRequest function.

Code: Select all

int EventLoop::exec(int width, int height, int timeout)
{
	exitCode_ = -1;
	exit_.store(false, std::memory_order_release);
	auto start_time = std::chrono::high_resolution_clock::now();
	auto lastTime = start_time;

	while(1) {
		auto now = std::chrono::high_resolution_clock::now();
		if (timeout > 0 && now - start_time > std::chrono::milliseconds(timeout*1000))
			break;
			
		if ((calls_.size() % 2 == 0) && (calls_.size() > 1)) {
			dispatchCalls();   //call the processRequest function
			displayFrame(width, height);
		}
	}

	return exitCode_;
}

thisisalloneword
Posts: 167
Joined: Mon Dec 24, 2018 2:13 am

Re: Libcamera Dual Camera Setup

Mon Jul 31, 2023 7:10 pm

I am intrigued with getting the Pi to record stereo videos. I have been using an inexpensive 3D camera from QooCam ( ~$350 ) . It works well but has no API. I want to automate the recordings. Here's an example of what I am trying to achieve:

https://www.youtube.com/watch?v=92gZrtAzzbw

Does the dual camera setup on the Pi use only USB cameras? I am mostly familiar with the native v2 camera that plugs directly into the Pi. Not sure if one can plug two into the pi somehow?

Return to “Camera board”