Streaming mjpg from Raspberry Cam

[TarjeiB]

I managed to get a mjpg-stream from the Raspberry Cam by (ab)using the timelapse function and using mjpg-streamer. The CPU usage is very low (~1%) but it's not at all optimal, the speed is quite bad. Does someone have any idea on how to make a higher FPS solution?

First make sure you have a /tmp or other location that writes to memory instead of thje SD card:

/etc/default/tmpfs:

Code: Select all

# mount /tmp as a tmpfs.  Defaults to no; set to yes to enable (/tmp
# will be part of the root filesystem if disabled).  /tmp may also be
# configured to be a separate mount in /etc/fstab.
RAMTMP=yes

Reboot.

Then run raspistill in timelapse mode with the desired parameters:

Code: Select all

raspistill -tl 500 -t 999999 -vf -w 960 -h 720 -o /tmp/mjpg/test.jpg -n -q 50&

This outputs a JPG picture to /tmp/mjpg/test.jpg every half second. Experiment with the -tl parameter to see how fast you can make it go.

And mjpg_streamer with the desired parameters:

Code: Select all

mjpg_streamer -i 'input_file.so -f /tmp/mjpg -r'

This reads the current image in /tmp/mjpg, outputs it to the mjpg-stream, then deletes it (-r). As soon as raspistill writes a new image, it reads that, outputs it, and deletes it. Etc, etc.

Access the stream by putting

Code: Select all

http://<raspberrypi>:8080/?action=stream

in a compatible browser, like Chrome, and Firefox, or VLC.

This produces as high quality a MJPG-stream you want, but due to the writing and deleting of images, even on ramdisk, it's not giving much of a FPS. Hopefully someone has a better idea.

[pi-man-uk]

on first look - works a treat for me, low cpu usage, v good quality and wide browser support.
well done. many thanks.

[pi-man-uk]

Quick question. As I said, this method works great for my project. One problem at present.

I want the stream to run forever. Currently the -t 999999 switch seems to knock out the raspistill command after a period of time. Is that right and is there a way to have it running without timeout. I tried putting a really big number in there but command didn't seem to run after that.

Thanks for any insight.

[TarjeiB]

The -t parameter is in milliseconds, and there's a limit (I think someone mentioned ~26 days?) Don't know how many 9's that work out to.

The binary shouldn't need -t when it has -tl in my opinion, and the old bug that made -1 work for eternal loop would have been nice to have back

In any case, if you need it to run longer, you can always use a bash script that just loops it forever.

Code: Select all

#!/bin/bash
while :
do
raspistill -tl 500 -t 999999 -vf -w 960 -h 720 -o /tmp/mjpg/test.jpg -n -q 50
sleep 1
done

I haven't tested that code so can't promise it's bug free

*EDIT* Tested and 999999999 works as maximum.

[pi-man-uk]

Thanks T - very clear and helpful... will ssh in and give this a try later... as you say shame the -1 trick doesn't work.. that'd do it easily.

[jamesh]

-t -1 didn't make it run forever, that would just mean the max possible which is the 23 days or whatever.

I'll add a forever option in the next release.

Although it's a one line change in the code to make it run forever, for people who are desperate for it. Perhaps actually making some efforts yourselves might be appropriate...

-t and -tl are different options - you often still want the app the finish, even when in timelapse mode. So useful to have both options.

[ppumkin]

Do you think it will be a good idea to pipe raspivide to ffmpeg and output jpg files. Then ffmpeg writes as fast as it can to the ram /tmp while mjpg reads and deltes them as fast as possible.

something like

Code: Select all

raspivid -o -  \ 
ffmpeg -i - \
    -f image2(?) \
    -c:v mjpeg \
    stream%d.jpg

I tried your method and hardly got 1.5fps - that is with 320x240 or 1080p on lan. So there is something peculiar there. possibly raspistill is slow. I just thought of this at work so will try it at home later.

I will see what performance I get. then will try to overclock see if it helps. boost your technique and compare mine too.

This is prooving to be a bit of a pain to stream to a browser:(

[pi-man-uk]

Thanks. .. trying the -t 999999999 approach right now. Seems to work thanks.

am I right in thinking that is 11 days?

[texy]

Thanks. .. trying the -t 999999999 approach right now. Seems to work thanks.

am I right in thinking that is 11 days?

11.57 days by my calculations

T.

[TarjeiB]

Do you think it will be a good idea to pipe raspivide to ffmpeg and output jpg files. Then ffmpeg writes as fast as it can to the ram /tmp while mjpg reads and deltes them as fast as possible.

something like

Code: Select all

raspivid -o -  \ 
ffmpeg -i - \
    -f image2(?) \
    -c:v mjpeg \
    stream%d.jpg

I tried your method and hardly got 1.5fps - that is with 320x240 or 1080p on lan. So there is something peculiar there. possibly raspistill is slow. I just thought of this at work so will try it at home later.

raspivid may work but I think this will eat CPU like a monster as the stream has to be converted from h.264 to jpg.
In any case, the limitation isn't raspistill, but the process of having a program writing to a file, having another program poll for that file, read it and then delete it, and then having raspistill write a new file. It's just a really bad solution in place of missing stdin/stdout functions.

If someone smart wanted to look at the mjpg-streamer input_file.so filter and make it read directly from stdin, we should be golden raspistill and mjpg_streamer in themselves both seem really fast and use little to no CPU.

[LeoWhite]

-t -1 didn't make it run forever, that would just mean the max possible which is the 23 days or whatever.

I'll add a forever option in the next release.

Although it's a one line change in the code to make it run forever, for people who are desperate for it. Perhaps actually making some efforts yourselves might be appropriate...

I've very quickly added a -notimeout option if you've not already made the above change yourself! patch below. Its only had a few minutes testing, but it certainly running for more than the default 5 seconds. Obviously I should leave my RPi streaming for the next month to see if it gets past the 23 day mark

Code: Select all

diff --git a/host_applications/linux/apps/raspicam/RaspiVid.c b/host_applications/linux/apps/raspicam/RaspiVid.c
index 580ceef..bd6cfa0 100755
--- a/host_applications/linux/apps/raspicam/RaspiVid.c
+++ b/host_applications/linux/apps/raspicam/RaspiVid.c
@@ -103,6 +103,7 @@ int mmal_status_to_int(MMAL_STATUS_T status);
 typedef struct
 {
    int timeout;                        /// Time taken before frame is grabbed and app then shuts down. Units are milliseconds
+   int notimeout;                      /// !0 to disable the timeout
    int width;                          /// Requested width of image
    int height;                         /// requested height of image
    int bitrate;                        /// Requested bitrate
@@ -146,6 +147,7 @@ static void display_valid_parameters(char *app_name);
 #define CommandDemoMode     7
 #define CommandFramerate    8
 #define CommandPreviewEnc   9
+#define CommandNoTimeout   10
 
 static COMMAND_LIST cmdline_commands[] =
 {
@@ -159,6 +161,7 @@ static COMMAND_LIST cmdline_commands[] =
    { CommandDemoMode,"-demo",       "d",  "Run a demo mode (cycle through range of camera options, no capture)", 1},
    { CommandFramerate,"-framerate", "fps","Specify the frames per second to record", 1},
    { CommandPreviewEnc,"-penc",     "e",  "Display preview image *after* encoding (shows compression artifacts)", 0},
+   { CommandNoTimeout,"-notimeout", "nt",  "Disable timeout when capturing video", 0},
 };
 
 static int cmdline_commands_size = sizeof(cmdline_commands) / sizeof(cmdline_commands[0]);
@@ -181,6 +184,7 @@ static void default_status(RASPIVID_STATE *state)
 
    // Now set anything non-zero
    state->timeout = 5000;     // 5s delay before take image
+   state->notimeout = 0;      // Default to obeying the timeout
    state->width = 1920;       // Default to 1080p
    state->height = 1080;
    state->bitrate = 17000000; // This is a decent default bitrate for 1080p
@@ -210,7 +214,7 @@ static void dump_status(RASPIVID_STATE *state)
    }
 
    fprintf(stderr, "Width %d, Height %d, filename %s\n", state->width, state->height, state->filename);
-   fprintf(stderr, "bitrate %d, framerate %d, time delay %d\n", state->bitrate, state->framerate, state->timeout);
+   fprintf(stderr, "bitrate %d, framerate %d, time delay %d, no timeout %d\n", state->bitrate, state->framerate, state->timeout, state->notimeout);
 
    raspipreview_dump_parameters(&state->preview_parameters);
    raspicamcontrol_dump_parameters(&state->camera_parameters);
@@ -363,6 +367,10 @@ static int parse_cmdline(int argc, const char **argv, RASPIVID_STATE *state)
          state->immutableInput = 0;
          break;
 
+      case CommandNoTimeout:
+         state->notimeout = 1;
+         break;
+         
       default:
       {
          // Try parsing for any image specific parameters
@@ -1037,12 +1045,11 @@ int main(int argc, const char **argv)
          {
             // Run for the user specific time..
             int num_iterations = state.timeout / state.demoInterval;
-            int i;
 
             if (state.verbose)
                fprintf(stderr, "Running in demo mode\n");
 
-            for (i=0;i<num_iterations;i++)
+            while (state.notimeout || num_iterations--)
             {
                raspicamcontrol_cycle_test(state.camera_component);
                vcos_sleep(state.demoInterval);
@@ -1084,18 +1091,27 @@ int main(int argc, const char **argv)
                // out of storage space)
                // Going to check every ABORT_INTERVAL milliseconds
 
-               for (wait = 0; wait < state.timeout; wait+= ABORT_INTERVAL)
+               for (wait = 0; wait < state.timeout;)
                {
                   vcos_sleep(ABORT_INTERVAL);
                   if (callback_data.abort)
                      break;
+                     
+                  // Only increase the wait time if we are obeying the timeout
+                  if(!state.notimeout) {
+                  	wait+= ABORT_INTERVAL;
+                  }
                }
 
                if (state.verbose)
                   fprintf(stderr, "Finished capture\n");
             }
-            else
+            else {
+            	// If we are not obeying the timeout then loop forever here
+            	do {
                vcos_sleep(state.timeout);
+              } while(!state.notimeout);
+            }
          }
       }
       else
-- 
1.7.9.5

Leo

[texy]

I believe it's 11.57 days
Texy

[LeoWhite]

I believe it's 11.57 days

Mmhh I just got 24.85 days as the max

Leo

[texy]

I believe it's 11.57 days

Mmhh I just got 24.85 days as the max

Leo

I thought it was 999999999ms, which is 11.57 days

T.

[LeoWhite]

I believe it's 11.57 days

Mmhh I just got 24.85 days as the max

Leo

I thought it was 999999999ms, which is 11.57 days

The timeout is stored as a 32bit integer value, so the maximum possible value should be 2147483647ms. Its just much easier to put in a bunch of 9s than it is to remember that value

Not that I've gone and actually tried that myself but having just looked through the code it should work (There does appear to be an issue where the timeout is read in as an unsigned value, which may explain why the -1 thing was working)

Leo

[TarjeiB]

Digging a bit more here, the input_file.so seems to be using inotify to check if there are new files.
This will never be fast, so without a new plugin we're stuck at horrid framerates.

Unfortunately I'm not at all a coder - I looked through the input_file.c and tried to find examples of how to make C read from stdin, but it's not possible for me to figure out by just reading the code by a long shot.

The most updated mjpg-streamer seems to be the experimental version at github:
https://github.com/codewithpassion/mjpg ... perimental

That is the one I've compiled on the Pi, if someone wants to look at the plugins and see if it's possible to make a input_stdin one it'd be a huge step forward

[AntoineDUBOSTBULARD]

Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB

[TarjeiB]

Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB

I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?

[sharix]

There should be an option to only adjust the lighting levels, awb, etc before the first shot instead of before every shot. someone get to it

[AntoineDUBOSTBULARD]

Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first thought that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB

I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?

TarjeiB, can you tell me the exact commands your are using for achieving such frame rates ?
Mine is :
$ mkdir /tmp/stream
$ raspistill -w 1920 -h 1080 -q 25 -o /tmp/stream/pic.jpg -tl 40 -t 9999999 -th 0:0:0 | ./mjpg_streamer -i "./input_file.so -f /tmp/stream" -o "./output_http.so -w ./www"

Changing the resolution do not change anything

Thanks

ADB

[TarjeiB]

TarjeiB, can you tell me the exact commands your are using for achieving such frame rates ?
Mine is :
$ mkdir /tmp/stream
$ raspistill -w 1920 -h 1080 -q 25 -o /tmp/stream/pic.jpg -tl 40 -t 9999999 -th 0:0:0 | ./mjpg_streamer -i "./input_file.so -f /tmp/stream" -o "./output_http.so -w ./www"
ADB

Code: Select all

$ raspistill -t 999999 -tl 50 -o /tmp/mjpg/test.jpg -n -w 960 -h 720&
$ mjpg_streamer -i 'input_file.so -f /tmp/mjpg -d 0'&

Goes quite fast! Make sure /tmp is a ramdisk. Not sure the pipe is a good idea either, I run them as separate commands.

[sergkit]

Does anybody rich normal fps with streaming RaspberryCam? I have get only 0.2 fps

Code: Select all

raspistill -t 999999 -tl 50 -o /tmp/test.jpg -n -w 960 -h 720

Code: Select all

./mjpg_streamer -i './input_file.so -f /tmp/ -d 0' -o "./output_http.so -w ./www -p 8080"

[jacksonliam]

Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB

I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?

How did you achieve this >~7fps? I've been playing with the code with aim to write an mjpeg streamer input plugn and in a simple capture->encode loop the fastest i seem to get is about 12fps. Resolution makes very little difference here, and it requires around a 100ms delay between captures else the picture is too dark or just black.


For anyone who is interested, here's my work on an mjpeg streamer plugin for raspicam https://github.com/jacksonliam/mjpg-streamer

[TonAu]

I downloaded the experimental mjpgstreamer from github and it works fine for a couple of minutes.

Had to apt-get cmake and libjpeg8-dev

Also, the makefile does not find a directory 'build' in the raspicam plugin and then stops with an error.
That directory is not in the repository. I just did 'mkdir build' there and after that cmake went on and the make finished succesfully.

The webserver starts with 1 fps approximately. But after 800 frames or so the picture slowly turns dark and ends up black.
Then I have to restart mjpg-streamer.
Any clue what that can be?
update:
- a delay of 300 seems to be working better: now for hours already, with 5-6% cpu load. (I specified x and y to 640 480, should make no diff)

[jacksonliam]

Glad your trying it out. I fixed the build folder missing thing. Didn't know you can't have empty github folders.

I think I have libjpeg62-dev. I've not seen the darker images thing and I've run this for ~24hrs on the default 1000 delay. Wouldn't have expected a shorter delay to help... I wonder if this is related:
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=45783
Like raspistill, I keep the camera open the entire time, though it does seem to do all the light level stuff on each shot, I don't think its possible to have it only do it once so you can get faster stills at the same light level - I've had a good look though threads on here and the source and can't find anything that says so.

which firmware are you on? I dunno if a rpi-update would help?

Code: Select all

/opt/vc/bin/vcgencmd version
Jun  6 2013 18:30:04
Copyright (c) 2012 Broadcom
version e6f0abde032163069972925f175c4a3a5617d46d (clean) (release)