Smatbd
Posts: 2
Joined: Mon Feb 12, 2024 12:20 am

1080p gstreamer pipe with multiple outputs very slow

Mon Feb 12, 2024 2:10 am

Hello,

I am attempting to display 1080p camera stream to hdmi output as well as some other output, file/udpstream/message broker anything on cm4. I am able to stream the camera to display with low latency and 30-60fps, but when I use tee and try to send the frames anywhere else the pipeline complains and pretty much stalls.

Where am I going wrong with this pipeline? I have searched a bit on the forums for solutions with no luck, but there was a good amount of information for me to even get it to where it would start which was helpful. Dropping the frame rate doesn't really seem to help so I believe the pipeline is set up incorrectly.

Code: Select all

# this one is crawling
gst-launch-1.0 libcamerasrc ! capsfilter caps=video/x-raw,width=1920,height=1080,format=NV21,framerate=60/1 ! tee name=t ! queue ! glimagesink t. ! queue ! v4l2convert  ! v4l2h264enc extra-controls="controls,repeat_sequence_header=1" ! 'video/x-h264,level=(string)4.2' ! h264parse ! rtph264pay ! udpsink host=0.0.0.0 port=9000

# this one works 
gst-launch-1.0 libcamerasrc ! capsfilter caps=video/x-raw,width=1920,height=1080,format=NV21,framerate=60/1 ! queue ! fpsdisplaysink

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 15860
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: 1080p gstreamer pipe with multiple outputs very slow

Mon Feb 12, 2024 10:27 am

A couple of observations.

Generally GStreamer will manage to achieve zero copy passing of buffers from the likes of libcamerasrc to many consumers.
The use of a tee component almost certainly means it has to copy the image data from the one source buffer to 2 of more output buffers, so you've dropped a huge amount of memcpying into the system.

You're likely to be overloading the ISP. libcamerasrc, v4l2convert, and v4l2h264enc all use the ISP, and the overall throughput on CM4 is around 180MPix/s under fully ideal situations. 1080p is just over 2MPix/frame, so 1080p60 is 120MPix/s. Use it 3 times and you're trying to process around 360MPix/s.

The H264 encoder is only rated for 1080p30. 1080p50 can normally be achieved if the system isn't doing huge numbers of other tasks (it's generally memory bandwidth). 1080p60 is very much on the edge for most situations.
Software Engineer at Raspberry Pi Ltd. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Smatbd
Posts: 2
Joined: Mon Feb 12, 2024 12:20 am

Re: 1080p gstreamer pipe with multiple outputs very slow

Sun Feb 18, 2024 7:52 pm

Thanks, makes a lot of sense. I thought maybe tee would do something smarter than making copies, but I can also see how it may not know if that is safe to do.

Based on those numbers 25fps does seem to actually work with that pipeline but can jitter at times.

I know that setting gpu_freq in boot config has an impact on the max fps able to display video over hdmi, does this impact the hardware encoders/decoders as well or are these two separate things, the isp and the gpu hardware?

Return to “Compute Module”