therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Picamera2 preview release

Tue Feb 15, 2022 1:53 pm

Hi everyone, for folks that haven't spotted it yet, please check this out! There's quite a lot you can do already, and a number of examples, though no support for video recording just yet.

Feel free to post questions below, I'll do my best to answer.

GigiG
Posts: 28
Joined: Tue Jun 29, 2021 8:37 am

Re: Picamera2 preview release

Wed Feb 16, 2022 11:38 am

Hi, I'm trying to build Picamera2 on RPi0 (Buster) but after the command

Code: Select all

git clone --branch picamera2 [email protected]:raspberrypi/libcamera.git
I got the following error:

Code: Select all

[email protected]: Permission denied (publickey).
Regards

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Wed Feb 16, 2022 12:16 pm

Hi, it sounds like maybe you haven't set up an ssh key for accessing GutHub? I believe that repository should be publicly visible (someone please say if that doesn't seem to be the case), so maybe try visiting https://github.com/raspberrypi/picamera2 in your browser. The little "Code" drop-down box should give you an https link to clone the repository, maybe try that?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 15837
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Picamera2 preview release

Wed Feb 16, 2022 12:22 pm

GigiG wrote:
Wed Feb 16, 2022 11:38 am
Hi, I'm trying to build Picamera2 on RPi0 (Buster) but after the command

Code: Select all

git clone --branch picamera2 [email protected]:raspberrypi/libcamera.git
I got the following error:

Code: Select all

[email protected]: Permission denied (publickey).
Regards
Use

Code: Select all

git clone  --branch picamera2 https://github.com/raspberrypi/libcamera.git
Cloning using SSH requires you to have a Github account with an SSH public key registered.

Pull request opened to update the docs. https://github.com/raspberrypi/picamera2/pull/1
*edit* PR now merged to advise using HTTPS instead of SSH.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

GigiG
Posts: 28
Joined: Tue Jun 29, 2021 8:37 am

Re: Picamera2 preview release

Wed Feb 16, 2022 1:04 pm

Use

Code: Select all

git clone  --branch picamera2 https://github.com/raspberrypi/picamera2.git
Cloning using SSH requires you to have a Github account with an SSH public key registered.

Thanks. I suppose to clone a libcamera.git not picamera2.git, right?

Code: Select all

git clone  --branch picamera2 https://github.com/raspberrypi/picamera2.git

Code: Select all

git clone  --branch picamera2 https://github.com/raspberrypi/libcamera.git

dgalland
Posts: 75
Joined: Tue Aug 27, 2019 8:20 am

Re: Picamera2 preview release

Thu Feb 17, 2022 6:46 pm

So I immediately tried all this with the capture_jpeg example

Code: Select all

picam2 = Picamera2()
#preview = NullPreview(picam2)
preview = QtGlPreview(picam2)

cam_config = picam2.still_configuration()
#cam_config = picam2.preview_configuration()
picam2.configure(cam_config)
- With NullPreview and preview_configuration I have a 640x480 jpeg :D :D :D

- With NullPreview and still_configuration :
Unable to request 1 buffers: Cannot allocate memory :x

- With QtGlPreview I have an OpenGL error :x

But then it starts to work!
Raspberry 3B+ 1G

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Thu Feb 17, 2022 6:56 pm

Yes, I haven't been trying this on a Pi 3 yet. Sounds like there may be some issues with the amount of CMA memory, so possibly worth having a look in /proc/meminfo. Not sure why you'd get an OpenGL error. Is Glamor re-enabled?

tvoverbeek
Posts: 116
Joined: Mon Feb 04, 2013 9:50 am
Location: Fieberbrunn, Austria

Re: Picamera2 preview release

Thu Feb 17, 2022 8:12 pm

Build it on a Pi4 2G aarch64. No problems. Went exactly as described.
First tests worked fine: preview(drm, gl, qt and at with x-forwarding)
Also via VNC (except the drm one for obvious reasons).
Have not tried opencv yet on 64-bit

dgalland
Posts: 75
Joined: Tue Aug 27, 2019 8:20 am

Re: Picamera2 preview release

Thu Feb 17, 2022 9:45 pm

Code: Select all

cat /proc/meminfo
MemTotal:         944968 kB
MemFree:          380900 kB
MemAvailable:     648656 kB
Buffers:           25344 kB
Cached:           286856 kB
SwapCached:        32848 kB
Active:           233984 kB
Inactive:         232752 kB
Active(anon):       8100 kB
Inactive(anon):   153948 kB
Active(file):     225884 kB
Inactive(file):    78804 kB
Unevictable:          32 kB
Mlocked:              32 kB
SwapTotal:        102396 kB
SwapFree:          57084 kB
Dirty:               236 kB
Writeback:             0 kB
AnonPages:        125920 kB
Mapped:           113340 kB
Shmem:              7512 kB
KReclaimable:      16248 kB
Slab:              34804 kB
SReclaimable:      16248 kB
SUnreclaim:        18556 kB
KernelStack:        1960 kB
PageTables:         5052 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      574880 kB
Committed_AS:    1140160 kB
VmallocTotal:    1114112 kB
VmallocUsed:        5732 kB
VmallocChunk:          0 kB
Percpu:              576 kB
CmaTotal:         262144 kB
CmaFree:          192512 kB
I don't know what Glamor is and how to re-enable it?

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Fri Feb 18, 2022 8:16 am

A bit surprised not to see CMA listed. i guess the usual questions probably apply - what version of the OS is this (a fresh Bullseye?), what do "uname -a" and "vcgencmd version" report. I'll have to try it on a Pi 3 for myself, but I'm not able to lay my hands on one just at the moment so it'll have to wait a short while.

dgalland
Posts: 75
Joined: Tue Aug 27, 2019 8:20 am

Re: Picamera2 preview release

Fri Feb 18, 2022 9:30 am

Scrolling down:
CmaTotal: 262144 kB
CmaFree: 192512 kB

Yes it is a fresh bullseye just installed
Linux raspberrypi 5.10.92-v7+ #1514 SMP Mon Jan 17 17:36:39 GMT 2022 armv7l GNU/Linux
version bd88f66f8952d34e4e0613a85c7a6d3da49e13e2 (clean) (release) (start)

with only one buffer it works:
cam_config = picam2.still_configuration(buffer_count=1)

User avatar
MRV
Posts: 214
Joined: Thu Jul 19, 2018 3:09 am

Re: Picamera2 preview release

Fri Feb 18, 2022 1:53 pm

Any news on dual camera stereo 3d picamera support?

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Fri Feb 18, 2022 5:55 pm

Stereoscopic camera support in Picamera2 will require it to be present in libcamera first and as yet we don't have any kind of estimated timetable for that.

dgalland
Posts: 75
Joined: Tue Aug 27, 2019 8:20 am

Re: Picamera2 preview release

Sat Feb 19, 2022 5:24 pm

So I did some experimenting with Picamera2.
My current Picamera application captures as fast as possible jpeg encoded frames either in full 4056x3040 resolution or in binned 2028x1520 mode.
First of all it's not clear why three configurations are provided and what are really the differences between them.
Is it really useful since generally we will specify at least size and format:

Code: Select all

cam_config = picam2.still_configuration(buffer_count=1, main={"size": (2028, 1520), "format": "RGB888"})
If I change still to preview or video what is really the difference for the obtained frames ?
I noticed that with "still" I actually get 2016x1520 frames while with "video" or "preview" we stay in 2028x1520 but in this case the make_array fails to numpy reshape (padding?)

Then I compared from the performance point of view.
With my PI3B+ 1G we can allocate only one buffer then in binned mode 2028x1520 the simple loop on the requests without any copy nor treatment reaches without dropped frames 30fps (the default it seems).
But if I add the copy in a numpy array we fall to 10fps without jpeg encoding and to 5fps with jpeg encoding with many dropped frames.
This is not as good as my Picamera legacy stack based application which reaches 13 2028*1520 jpeg encoded frames per sec (in video mode)

It seems that a PI4 with sufficient memory capacity is necessary ?

Code: Select all

startTime = time.time()
count = 100
for i in range(0,count) :
    request = picam2.capture_request()
    image = request.make_array("main")
    jpeg = cv2.imencode('.jpg', image)
    request.release()
stopTime = time.time()
fps = float(count/(stopTime-startTime))
print("Time:", (stopTime-startTime), " Fps:", fps)
Then I did HDR experiments a bit like the Mertens supplied example but in the frame stream without closing and opening the camera by playing with AeEnable and ExposureTime. The behavior is correct and here it is interesting to have the metadata without the frame.

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Sat Feb 19, 2022 6:08 pm

dgalland wrote:
Sat Feb 19, 2022 5:24 pm
...
First of all it's not clear why three configurations are provided and what are really the differences between them.
The three configurations try to give users the settings they're most likely to want "out of the box". But other users may want different things so we'll have to see what the best way is to do this across the user base. So for example:
* Video configurations give you the most buffers, which will help avoiding dropped frames, especially once we have a supported h264 encoder. They also give you a "faster" version of the colour denoise processing, and fix the framerate to 30fps, which is often what people want.
* The still configuration gives you better but slower colour denoise and tries to allocate fewer buffers because the frames will typically be larger.
* The preview configuration doesn't do any special colour denoise at all, I believe, and gives you variable framerates. This means you can switch to still captures and get longer exposure times.
But there's nothing very fundamentally different - by changing what you ask for, and perhaps specifying application specific control values, you can make any one configuration match another.
...
I noticed that with "still" I actually get 2016x1520 frames while with "video" or "preview" we stay in 2028x1520 but in this case the make_array fails to numpy reshape (padding?)
Yes, there are padding issues. I think the "video_configuration" method is forgetting to align the output stream size; it probably should. But you can always change the size again before calling "configure" and it should do what you ask. There's probably no reason not to "slice" the numpy arrays into a "non-aligned" shape, I just haven't tried yet whether it has any performance implications.
...
With my PI3B+ 1G we can allocate only one buffer then in binned mode 2028x1520 the simple loop on the requests without any copy nor treatment reaches without dropped frames 30fps (the default it seems).
Obviously we'll have to look at the memory issues on the Pi 3. I don't think there's any clear reason why it shouldn't be able to allocate as many buffers as libcamera-apps do, so maybe there's something to investigate there.
But if I add the copy in a numpy array we fall to 10fps without jpeg encoding and to 5fps with jpeg encoding with many dropped frames.
This is not as good as my Picamera legacy stack based application which reaches 13 2028*1520 jpeg encoded frames per sec (in video mode)
We haven't released any video encoding as yet, but eventually I expect that to include jpeg ("mjpeg") encoding in due course. There are issues with jpeg encode/decode in that it's generally moving into software, and in the Python world it's hard to use muiti-threading efficiently so there will definitely be some things that will require optimisation here.
It seems that a PI4 with sufficient memory capacity is necessary ?
You certainly shouldn't need a Pi 4, but it will obviously affect performance. This is just a preview release, so it's clearly good to hear about other platforms and we will be spending time on them in the future.

tvoverbeek
Posts: 116
Joined: Mon Feb 04, 2013 9:50 am
Location: Fieberbrunn, Austria

Re: Picamera2 preview release

Mon Feb 21, 2022 9:41 pm

I have been playing with preview_drm.py (no X) since I want to run this eventually on a Pi Zero.
The preview window is always positioned in the upper left hand corner (at 0,0).
I have added a possibility to specify a non zero upper left hand corner, so it can e positioned where you want it
by modifying drm_preview.py.
Should i submit a pull request for this change?
The other preview options are in qt windows, so can be positioned in the desktop.

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Tue Feb 22, 2022 9:16 am

tvoverbeek wrote:
Mon Feb 21, 2022 9:41 pm
...
Should i submit a pull request for this change?
The other preview options are in qt windows, so can be positioned in the desktop.
Yes please, that sounds like a good change!

tvoverbeek
Posts: 116
Joined: Mon Feb 04, 2013 9:50 am
Location: Fieberbrunn, Austria

Re: Picamera2 preview release

Tue Feb 22, 2022 10:03 pm

PR submitted.
Could be enhanced by checking if the window (now specified with x,y,w,h) actually fits on the screen and if not adjust accordingly.

tvoverbeek
Posts: 116
Joined: Mon Feb 04, 2013 9:50 am
Location: Fieberbrunn, Austria

Re: Picamera2 preview release

Wed Feb 23, 2022 9:27 pm

OpenCV and picamera2

When installing picamera2 on Raspi OS 64-bit on a Pi4-2GB I used the apt version of opencv:

Code: Select all

 sudo apt install python3-opencv
This is version 4.5.1 which is more recent than the 4.4.0.46 version suggested in the README.
The Haar classifiers are in a separate package: opencv-data.

Code: Select all

sudo apt install opencv-data
They are then in /usr/share/opencv4/haarcascades/.
After modifying the Haar classifiers path in opencv_face_detect and opencv_face_detect_2 both
examples work.
I have not tried (yet) to use the apt opencv package on Raspi OS 32-bit, but I exepct it to work.

The opencv_mertens_merge examples fails on the last exposure with:
ERROR RPISTREAM rpi_stream.cpp:242 Failed to queue buffer for ISP Output0
I guess it is running out of buffers. This is on a 2G Pi4

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Wed Feb 23, 2022 9:59 pm

Hi, just a note to anyone interested that we've posted an early version of video encode support. Again, it's a "have a look and give it a try" thing, and only for h.264, at this stage.

To see how to use it, please look at https://github.com/raspberrypi/picamera ... e_video.py

Anyone installing Picamera2 will also need the python-v4l2 module now, which can be installed using:

Code: Select all

cd
git clone https://github.com/RaspberryPiFoundation/python-v4l2.git
Don't forget to add /home/pi/python-v4l2 to your PYTHONPATH variable.

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Thu Feb 24, 2022 9:30 am

tvoverbeek wrote:
Wed Feb 23, 2022 9:27 pm
...

Code: Select all

sudo apt install opencv-data
They are then in /usr/share/opencv4/haarcascades/.
After modifying the Haar classifiers path in opencv_face_detect and opencv_face_detect_2 both
examples work.
Ah thanks, maybe this was the incantation I didn't know about. I'll give it a try.
The opencv_mertens_merge examples fails on the last exposure with:
ERROR RPISTREAM rpi_stream.cpp:242 Failed to queue buffer for ISP Output0
I guess it is running out of buffers. This is on a 2G Pi4
Actually I get this message on both 2GB and 4GB Pi4s, but in both cases it seems to produce the output files correctly. I'm not really sure why the message pops out, maybe the camera is still running on somehow and we aren't recycling buffers back to it fast enough. Or something. We'll take a look. It's certainly not running out of memory - if you "watch -n 0.25 grep Cma /proc/meminfo" while it's running it has loads of spare CMA memory on both platforms (actually I see no difference between the two).

tvoverbeek
Posts: 116
Joined: Mon Feb 04, 2013 9:50 am
Location: Fieberbrunn, Austria

Re: Picamera2 preview release

Thu Feb 24, 2022 2:00 pm

See my last commit to fix opencv_mertens_merge. It seems picam2.configure has to be called before each picam2.start.
Then the example works

ianblack
Posts: 2
Joined: Fri Nov 12, 2021 7:46 pm

Re: Picamera2 preview release

Tue Mar 01, 2022 6:49 pm

I've started a branch on my picamera2 fork that is going to have a focus on headless applications. Or applications that don't require a camera preview. Last night I figured out the steps for acquiring, configuring (sort of), starting, stopping, and releasing the camera. I'm having a hard time wrapping my head around the methods for sending and receiving requests. Is there an example that goes straight from point A to point B on how to submit a request for an image and then obtain the data? Or can you provide any insight? Thanks.

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1061
Joined: Tue Jan 07, 2020 9:15 am

Re: Picamera2 preview release

Wed Mar 02, 2022 9:20 am

ianblack wrote:
Tue Mar 01, 2022 6:49 pm
I've started a branch on my picamera2 fork that is going to have a focus on headless applications. Or applications that don't require a camera preview. Last night I figured out the steps for acquiring, configuring (sort of), starting, stopping, and releasing the camera. I'm having a hard time wrapping my head around the methods for sending and receiving requests. Is there an example that goes straight from point A to point B on how to submit a request for an image and then obtain the data? Or can you provide any insight? Thanks.
Hi, have you checked out the various examples here? Please feel free to post back here if you'd like any more help with those.

ianblack
Posts: 2
Joined: Fri Nov 12, 2021 7:46 pm

Re: Picamera2 preview release

Thu Mar 03, 2022 9:58 am

therealdavidp wrote:
Wed Mar 02, 2022 9:20 am
ianblack wrote:
Tue Mar 01, 2022 6:49 pm
I've started a branch on my picamera2 fork that is going to have a focus on headless applications. Or applications that don't require a camera preview. Last night I figured out the steps for acquiring, configuring (sort of), starting, stopping, and releasing the camera. I'm having a hard time wrapping my head around the methods for sending and receiving requests. Is there an example that goes straight from point A to point B on how to submit a request for an image and then obtain the data? Or can you provide any insight? Thanks.
Hi, have you checked out the various examples here? Please feel free to post back here if you'd like any more help with those.
I've looked through the examples a little bit. I think I am looking for something that is a little more granular and pieces out the interactions between the camera and libcamera. I've been working on documenting and simplifying your code here (at least I think I am). I'd like to get it to a point where I can submit a pull request while maintaining similar functionality to what you already have. I think I am having a hard time wrapping my head around the create and queueRequests functions and the entire CompletedRequests class. It may be that when I read requests I think of the python-requests module.

Here is my attempt at dumbing down the requests process.

Camera Starts
1) queueRequests ()
2) getReadyRequests
3) Plug into CompletedRequests which holds the thread until the requests are completed and then encodes them for shipment elsewhere.


Is there a simple way of recreating that, or do I have to jump through the hoops that CompletedRequests goes through?
Does the libcamera_config we submit have all of the information needed to complete requests? It looks like it is restricted to three streams at a single time. Does the order of those streams matter or is the 0th stream always for "main"?

I also seem to be breaking something when I send all of the controls with the start function. Are all of the controls accepted as options? It may also be how I am supplying the data. For example, it looks like the FrameDurationLimits control should be a tuple, while in the default controls it makes it seem like it is an integer. Am I interpreting those tuples correctly when you list the controls? (min, max, default).

Thanks!

Return to “Camera board”