RaspiMJPEG

[silvanmelchior]

Important!!!

This description is outdated. All versions of RaspiMJPEG >= 4.0 use a config-file instead of parameters. A working demonstration can be found here: http://www.raspberrypi.org/phpBB3/viewt ... 91#p468491

Hi

I'd like to share my new application: RaspiMJPEG. A first impression can be gained with my uploaded video: http://www.youtube.com/watch?v=k4CiDx53EJc
I wrote it because I googled a long time but wasn't able to find an application that can stream MJPEG from RPi Cam with many frames per second and record it in Full HD at the same time.

Demonstration

I wrote a web interface to demonstrate all features of RaspiMJPEG, you can find it here: http://www.raspberrypi.org/phpBB3/viewt ... 43&t=63276

Description / Usage

RaspiMJPEG is an OpenMAX-Application based on the mmal-library, which is comparable to RaspiVid. Both applications save the recording formated as H264 into a file. But instead of showing the preview on a screen, RaspiMJPEG streams the preview as MJPEG into a file. The update-rate and the size of the preview are customizable with parameters and independent of the recording, which is always 1080p 30fps. Once started, the application receives commands with a unix-pipe and showes its status on stdout and writes it into a status-file. The program terminates itself after receiving a SIGINT or SIGTERM

Usage as MJPEG-Streamer: raspimjpeg -w 320 -h 240 -d 5 -of /path/to/image.jpg

This command creates the image "path/to/image.jpg" with the size 320x240 px and updates it every time, 5 frames are captured, which gives (30/5=) 6fps. This way used, RaspiMJPEG serves as MJPEG-streamer and the preview can be showed on a website for example. To achieve a high update-rate (e.g. "-d 1"), it is recommended to save the file into the ram and not on the sd-card (e.g. "-of /dev/shm/image.jpg").

Usages as MJPEG-Streamer with capture: raspimjpeg -w 320 -h 240 -d 5 -of /path/to/image.jpg -cf /path/to/pipe -vf /path/to/video.h264

This command does the same as the one above, but it is also listening on the pipe, defined with the parametet -c. If another program writes "ca 1" into the pipe (shell: echo "ca 1" > /path/to/pipe), the application continues with the MJPEG-stream, but starts also a H264-capture 1080p 30fps into the defined file. The capture is stopped with the command "ca 0" via pipe. A new capture overwrites the defined file. To make MJPEG and H264 possible, it is recommended to write the image file only into the ram, as described above.

Usage as MJPEG-streamer with capture and status-output: raspimjpeg -w 320 -h 240 -d 5 -of /path/to/image.jpg -cf /path/to/pipe -vf /path/to/video.h264 -sf /path/to/textfile.txt

Until now, RaspiMJPEG wrote its status into stdout/stderr. With this new command, the status is also written into a textfile (no logging, just the newest status). Possible messages and their meanings are: ready --> MJPEG is streaming, not capturing capture --> MJPEG is streaming and H264 is capturing errror --> An error occured and the application terminated

How to download
I created a git fork from the Pi userland, so use this commands:

Code: Select all

sudo apt-get install cmake
git clone https://github.com/silvanmelchior/userland.git
cd userland
./buildme

The compilation on the Pi itself takes about 20 minutes and afterwards, raspimjpeg is installed in /opt/vc/bin and with

Code: Select all

sudo ln -s /opt/vc/bin/raspimjpeg /usr/bin/raspimjpeg

is it available from the shell without the path.

If you've found a bug or have any questions/comments, feel free to post. And of course do not hesitate do edit my source code for your own projects.

Silvan

Edit: I updated my application, it is now also possible to take images in full resolution and to record timelapses with max. 30fps. Detail in the readme on github.

Edit II: It's now possible to change the camera settings (brightness, white balance, exposure mode, ...) on the fly with a command into the control-pipe. The newest version is available via github, all information are in the readme.

[poing]

It sounds interesting. Would it also be possible to look at the stream and then when sending the signal have raspistill save a high def photo?

[silvanmelchior]

Yes of course, if I have some free time I'm going to add this feature and maybe even the possibility to change the camera-settings while streaming

[poing]

Great! That would be *really* useful (for me )

[jbeale]

This looks quite useful, thank you! I second the request for ability to save a high-res still on the signal. I presume the recommended location of the ever-changing JPEG image is ramdisk, if the process runs continuously. If you store it 5x per second to flash, I assume your flash card would wear out more quickly than it would otherwise.

Right now the raspistill command can write time-lapse stills with sequentially-numbered file names, but using the maximum possible rate ('-tl 0') I find it is limited to about 2 fps regardless of the image resolution. It seems your application can work faster, so this is a big leap forward. Could RaspiMJPEG also write jpeg file sequences? (something like img%02d.jpg to deliver img01.jpg, img02,jpg, im03.jpg ...)

I also have other suggestions, if it wouldn't be too difficult: the option to set the H.264 bitrate, and also set the H.264 video resolution to something lower. Some applications don't need full-HD, but do need long term recording with low bitrates. In theory you can also simply turn down the bitrate even at full HD, but at low bitrates, you generally get a better image with a smaller image going into the compressor because the rescaling process averages out some of the noise, which helps compression.

[silvanmelchior]

Yes, the idea is to use e.g. /dev/shm, otherwise the sd-card can't handle it together with the recording of the 1080p-stream.

I updated the github with my new version, it is now possible to capture images with "im" as pipe-command at the full resolution, the preview just hangs on while capturing and continues afterwards. The filename for the image is set like in raspistill with "image%d.jpg" Additionally it is now possible to set the MJPEG-quality with the parameter -q.

Yes, you're right, I connected the image encoder via image-resizer to the video-output of the camera, so it is possible to write 30 images per second. If you make the quality and resolution low enough, the filesystem can handle it. I added the option to set the output-file as /path/to/output%04d.jpg, like it is possible with RaspiStill, so you can do now "timelapses" with max. 30fps.

I haven't implemented the option to control the video-resolution and bitrate yet, I'm going to do that together with the camera-control-parameters (contrast, white balance, ...). But first I'm going to write a tutorial on how to use my application to control the camera with a website, because my next project is to improve my Quadcopter with the Pi Cam and for that I needed a way to control the cam with a browser. But if its urgent, feel free to modify my source code!

Silvan

[mikerr]

Here's an executable binary for those wanting to save 20 minutes compiling:

Code: Select all

wget https://www.dropbox.com/s/3f47cyh902l4dto/raspimjpeg
chmod a+x raspimjpeg

[jbeale]

Wow, it works! Thank you very much silvanmelchior for this very useful code! And mikerr also, for the compiled version.

It seems the images start getting saved about 0.5 seconds after starting the program. Using a resolution 480x352 I get a solid 30 fps of JPEG images (at least, no skipped frames in 6 seconds). My setup is not overclocked (ARM = 700 MHz).

Code: Select all

raspimjpeg -w 480 -h 352 -d 1 -of /run/shm/img%03d.jpg

Using 640x480 and asking for 30 fps, the actual achieved framerate is about 25 fps. 800x600 and 1024 x 576 works ok, at least for a few seconds, until the ramdisk is full. You do get some more skipped frames when attempting 30 fps at these resolutions.

Note: you want to select your JPEG dimensions with the same 16:9 aspect ratio of the HD video frame, otherwise it will be distorted (non-square pixels). So my example dimensions above like 640x480 expecting a 4:3 frame are somewhat distorted.

[shuckle]

Using 640x480 and asking for 30 fps, the actual achieved framerate is about 25 fps. 800x600 and 1024 x 576 works ok, at least for a few seconds, until the ramdisk is full. You do get some more skipped frames when attempting 30 fps at these resolutions.

What is the limiting factor? CPU 100% ? when doing 640x480 with 25 fps? Oveclocking would then help a lot.
I need to try this. Seems like finally a solution to get mjpeg streaming from the camera.

[jamesh]

I've been taking a quick look at the MJPEG codec that runs on the GPU - its pretty old code, and needs some tweaks, and as yet isn't working, but if I do get it working it should allow the encoder MMAL component to output MPJEG directly.

No idea if that would help here!

Can anyone explain what the use case for MJPEG is - it's an old format that isn't nearly as good as H264 in quality or size so I am interested to know why people want it.

[silvanmelchior]

MJPEG allows to stream the camera to nearly every browser, including them on mobilephones. H264-livestreams in HTML5 work with m3u-playlists and only the newest browsers support it and there is a big delay (more than a second), where as mjpeg has a very small delay because it always loads the last captured frame, as you can see on my video.

The JPEG-encoder is the same as raspistill uses and is programmed for still images I guess, so I "missuse" it because I feed it with a videostream and write all output pictures to the same file. But it works pretty well so I don't need the mjpeg-encoder on the gpu .

[shuckle]

In addition to what silvanmelchior said:
I am using zoneminder with my cameras and it can not handle H264, but it needs MJPEG.
And I have seen questions of how to use zoneminder with Rpi camera, but no answers...so I am not the only one using it.

[shuckle]

I've been taking a quick look at the MJPEG codec that runs on the GPU - its pretty old code, and needs some tweaks, and as yet isn't working, but if I do get it working it should allow the encoder MMAL component to output MPJEG directly.

But omxplayer can play mjpeg streams GPU accelerated, so I guess MJPEG code works ?

[jamesh]

I've been taking a quick look at the MJPEG codec that runs on the GPU - its pretty old code, and needs some tweaks, and as yet isn't working, but if I do get it working it should allow the encoder MMAL component to output MPJEG directly.

But omxplayer can play mjpeg streams GPU accelerated, so I guess MJPEG code works ?

The decoder works fine - the encoder doesn't. I modified the mmal layers and included the encoder in the firmware build (after altering so it's compiles to the new codec API). But it doesn't work...I'll need to talk to the codec guys to see what I missed.

[jbeale]

Using 640x480 and asking for 30 fps, the actual achieved framerate is about 25 fps. 800x600 and 1024 x 576 works ok, at least for a few seconds, until the ramdisk is full. You do get some more skipped frames when attempting 30 fps at these resolutions.

What is the limiting factor? CPU 100% ? when doing 640x480 with 25 fps? Oveclocking would then help a lot.
I need to try this. Seems like finally a solution to get mjpeg streaming from the camera.

Code: Select all

raspimjpeg -w 640 -h 480 -d 1 -of /run/shm/image.jpg

shows only 24% CPU utilization, so I imagine it is a timing-synchronization thing where the next camera frame sometimes comes slightly too soon for the file I/O process so it has to wait and do nothing until the next frame after that. (?)

[Nu2RPi]

Firstly, hi to all, and secondly I am a RPi Noob, and its been long time since I last used a BSD/linux style system.
RaspiMJPEG looks awesome, however, I am struggling to get it working.
At present i'm getting the following error when running:

Code: Select all

raspimjpeg -w 640 -h 480 -d 1 -of /run/shm/image.jpg

Error:

mmal: mmal_vc_port_parameter_set: failed to set port parameter 40:2:ENOSYS
Error: Could not set inline flag

Now im sure this is something I've missed, for example, I saw in the readme that I needed a 'ARM cross-compiler' installed, I had already installed 'cmake' and compiled 'RaspiMJPEG' (I thought successfully) before reading the readme .
Anyway, I though id try the precompile version someone posted (many thanks to them) and when I run it shows the same error message.

I was gonna start a fresh and install both 'cmake' and 'https://github.com/raspberrypi/tools/tr ... f-raspbian' however I hadn't a clue how to install the later of these so here I am 'Mr. Clueless' asking for help.
Any help would be greatly appreciated; typically I would search far and wide for a solution before posting, however, RaspiMJPEG appears pretty much bleeding edge and has almost no info on the interweb as of now.

Thx in advance.

[silvanmelchior]

Have you enabled camera-support (http://www.raspberrypi.org/camera)? Are you running the newest version of raspbian? Maybe a soft- or firmwareupdate might help:

Code: Select all

sudo apt-get update
sudo apt-get dist-upgrade
sudo rpi-update

[Nu2RPi]

Have you enabled camera-support (http://www.raspberrypi.org/camera)? Are you running the newest version of raspbian? Maybe a soft- or firmwareupdate might help:

Code: Select all

sudo apt-get update
sudo apt-get dist-upgrade
sudo rpi-update

Well all I can say it thank you so much, it works! Well it says 'MJPEG streaming' which is certainly an improvement. The distro needed upgrading.

Now i'm sorry to ask another stupid question but .....My intention is to use the MJPEG stream with Zoneminder, however, I am a little confused as to how exactly I pipe the stream, simply how to access it remotely. I have some idea but rather than muddy the water I'd really like to hear how you experts would make the stream accessible. Ideally with some examples that a simpleton (like myself) can understand.

[Joe116]

Hello,

for using this with zoneminder you need to stream the jpg with MJPG-stream as example,

http://blog.miguelgrinberg.com/post/how ... spberry-pi
Start with #3 skip #7 and change the path in #8 to the file you wrote with RaspiMJPEG.

I start it like:

Code: Select all

#start cam
nohup raspimjpeg -w 1280 -h 720 -d 1 -of /run/shm/img.jpg &
#start mjpeg-streamer
nohup $(LD_LIBRARY_PATH=/usr/local/lib mjpg_streamer -i "input_file.so -f /run/shm -n img.jpg" -o "output_http.so -w /usr/local/www") &

KR

Joe

[Nu2RPi]

Hello,

for using this with zoneminder you need to stream the jpg with MJPG-stream as example,

http://blog.miguelgrinberg.com/post/how ... spberry-pi
Start with #3 skip #7 and change the path in #8 to the file you wrote with RaspiMJPEG.

I start it like:

Code: Select all

#start cam
nohup $(raspimjpeg -w 1280 -h 720 -d 5 -of /run/shm/img.jpg) &
#start mjpeg-streamer
nohup $(LD_LIBRARY_PATH=/usr/local/lib mjpg_streamer -i "input_file.so -f /run/shm -n img.jpg" -o "output_http.so -w /usr/local/www") &

KR

Joe

Many thanks Joe, I'll give that a go hopefully this evening.

Thanks again, Mark.

[shuckle]

Hello,

for using this with zoneminder you need to stream the jpg with MJPG-stream as example,

http://blog.miguelgrinberg.com/post/how ... spberry-pi
Start with #3 skip #7 and change the path in #8 to the file you wrote with RaspiMJPEG.

I start it like:

Code: Select all

#start cam
nohup raspimjpeg -w 1280 -h 720 -d 1 -of /run/shm/img.jpg &
#start mjpeg-streamer
nohup $(LD_LIBRARY_PATH=/usr/local/lib mjpg_streamer -i "input_file.so -f /run/shm -n img.jpg" -o "output_http.so -w /usr/local/www") &

KR

Joe

What kind of speed (fps) you get with 1280x720 resolution?

[Joe116]

Hello,

the framerate is something between 2 and 5 pictures per second, but the cpu load is below 0.70!

To start the two programs I enter now at the end of /etc/inittab

Code: Select all

T1:234:respawn:raspimjpeg -w 1280 -h 720 -d 5 -of /run/shm/img.jpg

and at the end of /etc/rc.local

Code: Select all

nohup /usr/local/bin/mjpg_streamer -i "/usr/local/lib/input_file.so -f /run/shm -n img.jpg" -o "/usr/local/lib/output_http.so -w /usr/local/www" &

With this the streaming works but I have to test it with zoneminder because my smartphone apps doesn't like the path to the stream

Code: Select all

http://<ip>:8080/?action=stream

KR

Joe

[silvanmelchior]

Just a little information: I've written an installer that installs raspimjpeg and appache with a little website on Raspbian, so all you have to do is to run the installer (shell-script) and you'll be able to see the camera preview on a website (640x360 px, 30 fps). I'm going to write a tutorial and will add a link here as soon as I have some free time.

[jbeale]

Thanks again for your work on the great program! To speed downloads for those getting the github source, you might consider removing at least /userland/host_applications/linux/apps/hello_pi/hello_video/test.h264 which is a 31 MB video file (part of "Big Buck Bunny") and is not relevant to the RaspiMJPEG code. That would shrink the download nearly in half.

[silvanmelchior]

Thanks again for your work on the great program! To speed downloads for those getting the github source, you might consider removing at least /userland/host_applications/linux/apps/hello_pi/hello_video/test.h264 which is a 31 MB video file (part of "Big Buck Bunny") and is not relevant to the RaspiMJPEG code. That would shrink the download nearly in half.

You're welcome . And you're right, I deleted the movie, so nobody needs to waste space...