TarjeiB
Posts: 157
Joined: Thu Jul 12, 2012 3:33 pm

Streaming mjpg from Raspberry Cam

Mon May 27, 2013 11:19 am

I managed to get a mjpg-stream from the Raspberry Cam by (ab)using the timelapse function and using mjpg-streamer. The CPU usage is very low (~1%) but it's not at all optimal, the speed is quite bad. Does someone have any idea on how to make a higher FPS solution?

First make sure you have a /tmp or other location that writes to memory instead of thje SD card:

/etc/default/tmpfs:

Code: Select all

# mount /tmp as a tmpfs.  Defaults to no; set to yes to enable (/tmp
# will be part of the root filesystem if disabled).  /tmp may also be
# configured to be a separate mount in /etc/fstab.
RAMTMP=yes
Reboot.

Then run raspistill in timelapse mode with the desired parameters:

Code: Select all

raspistill -tl 500 -t 999999 -vf -w 960 -h 720 -o /tmp/mjpg/test.jpg -n -q 50&
This outputs a JPG picture to /tmp/mjpg/test.jpg every half second. Experiment with the -tl parameter to see how fast you can make it go.

And mjpg_streamer with the desired parameters:

Code: Select all

mjpg_streamer -i 'input_file.so -f /tmp/mjpg -r'
This reads the current image in /tmp/mjpg, outputs it to the mjpg-stream, then deletes it (-r). As soon as raspistill writes a new image, it reads that, outputs it, and deletes it. Etc, etc.

Access the stream by putting

Code: Select all

http://<raspberrypi>:8080/?action=stream
in a compatible browser, like Chrome, and Firefox, or VLC.

This produces as high quality a MJPG-stream you want, but due to the writing and deleting of images, even on ramdisk, it's not giving much of a FPS. Hopefully someone has a better idea.

pi-man-uk
Posts: 62
Joined: Tue May 08, 2012 1:41 pm

Re: Streaming mjpg from Raspberry Cam

Mon May 27, 2013 11:53 am

on first look - works a treat for me, low cpu usage, v good quality and wide browser support.
well done. many thanks.

pi-man-uk
Posts: 62
Joined: Tue May 08, 2012 1:41 pm

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 7:37 am

Quick question. As I said, this method works great for my project. One problem at present.

I want the stream to run forever. Currently the -t 999999 switch seems to knock out the raspistill command after a period of time. Is that right and is there a way to have it running without timeout. I tried putting a really big number in there but command didn't seem to run after that.

Thanks for any insight.

TarjeiB
Posts: 157
Joined: Thu Jul 12, 2012 3:33 pm

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 7:53 am

The -t parameter is in milliseconds, and there's a limit (I think someone mentioned ~26 days?) Don't know how many 9's that work out to.

The binary shouldn't need -t when it has -tl in my opinion, and the old bug that made -1 work for eternal loop would have been nice to have back :)

In any case, if you need it to run longer, you can always use a bash script that just loops it forever.

Code: Select all

#!/bin/bash
while :
do
raspistill -tl 500 -t 999999 -vf -w 960 -h 720 -o /tmp/mjpg/test.jpg -n -q 50
sleep 1
done
I haven't tested that code so can't promise it's bug free :)

*EDIT* Tested and 999999999 works as maximum.

pi-man-uk
Posts: 62
Joined: Tue May 08, 2012 1:41 pm

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 8:01 am

Thanks T - very clear and helpful... will ssh in and give this a try later... as you say shame the -1 trick doesn't work.. that'd do it easily.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 32345
Joined: Sat Jul 30, 2011 7:41 pm

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 9:21 am

-t -1 didn't make it run forever, that would just mean the max possible which is the 23 days or whatever.

I'll add a forever option in the next release.

Although it's a one line change in the code to make it run forever, for people who are desperate for it. Perhaps actually making some efforts yourselves might be appropriate...

-t and -tl are different options - you often still want the app the finish, even when in timelapse mode. So useful to have both options.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

ppumkin
Posts: 82
Joined: Tue May 29, 2012 10:22 pm

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 9:42 am

Do you think it will be a good idea to pipe raspivide to ffmpeg and output jpg files. Then ffmpeg writes as fast as it can to the ram /tmp while mjpg reads and deltes them as fast as possible.

something like

Code: Select all

raspivid -o -  \ 
ffmpeg -i - \
    -f image2(?) \
    -c:v mjpeg \
    stream%d.jpg

I tried your method and hardly got 1.5fps - that is with 320x240 or 1080p on lan. So there is something peculiar there. possibly raspistill is slow. I just thought of this at work so will try it at home later.

I will see what performance I get. then will try to overclock see if it helps. boost your technique and compare mine too.

This is prooving to be a bit of a pain to stream to a browser:(

pi-man-uk
Posts: 62
Joined: Tue May 08, 2012 1:41 pm

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 10:13 am

Thanks. .. trying the -t 999999999 approach right now. Seems to work thanks.

am I right in thinking that is 11 days?

texy
Forum Moderator
Forum Moderator
Posts: 5174
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 10:54 am

pi-man-uk wrote:Thanks. .. trying the -t 999999999 approach right now. Seems to work thanks.

am I right in thinking that is 11 days?
11.57 days by my calculations

T.
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555

TarjeiB
Posts: 157
Joined: Thu Jul 12, 2012 3:33 pm

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 11:23 am

ppumkin wrote:Do you think it will be a good idea to pipe raspivide to ffmpeg and output jpg files. Then ffmpeg writes as fast as it can to the ram /tmp while mjpg reads and deltes them as fast as possible.

something like

Code: Select all

raspivid -o -  \ 
ffmpeg -i - \
    -f image2(?) \
    -c:v mjpeg \
    stream%d.jpg
I tried your method and hardly got 1.5fps - that is with 320x240 or 1080p on lan. So there is something peculiar there. possibly raspistill is slow. I just thought of this at work so will try it at home later.
raspivid may work but I think this will eat CPU like a monster as the stream has to be converted from h.264 to jpg.
In any case, the limitation isn't raspistill, but the process of having a program writing to a file, having another program poll for that file, read it and then delete it, and then having raspistill write a new file. It's just a really bad solution in place of missing stdin/stdout functions.

If someone smart wanted to look at the mjpg-streamer input_file.so filter and make it read directly from stdin, we should be golden :) raspistill and mjpg_streamer in themselves both seem really fast and use little to no CPU.

LeoWhite
Posts: 131
Joined: Mon Apr 08, 2013 9:20 am

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 7:48 pm

jamesh wrote:-t -1 didn't make it run forever, that would just mean the max possible which is the 23 days or whatever.

I'll add a forever option in the next release.

Although it's a one line change in the code to make it run forever, for people who are desperate for it. Perhaps actually making some efforts yourselves might be appropriate...
I've very quickly added a -notimeout option if you've not already made the above change yourself! patch below. Its only had a few minutes testing, but it certainly running for more than the default 5 seconds. Obviously I should leave my RPi streaming for the next month to see if it gets past the 23 day mark :D

Code: Select all

diff --git a/host_applications/linux/apps/raspicam/RaspiVid.c b/host_applications/linux/apps/raspicam/RaspiVid.c
index 580ceef..bd6cfa0 100755
--- a/host_applications/linux/apps/raspicam/RaspiVid.c
+++ b/host_applications/linux/apps/raspicam/RaspiVid.c
@@ -103,6 +103,7 @@ int mmal_status_to_int(MMAL_STATUS_T status);
 typedef struct
 {
    int timeout;                        /// Time taken before frame is grabbed and app then shuts down. Units are milliseconds
+   int notimeout;                      /// !0 to disable the timeout
    int width;                          /// Requested width of image
    int height;                         /// requested height of image
    int bitrate;                        /// Requested bitrate
@@ -146,6 +147,7 @@ static void display_valid_parameters(char *app_name);
 #define CommandDemoMode     7
 #define CommandFramerate    8
 #define CommandPreviewEnc   9
+#define CommandNoTimeout   10
 
 static COMMAND_LIST cmdline_commands[] =
 {
@@ -159,6 +161,7 @@ static COMMAND_LIST cmdline_commands[] =
    { CommandDemoMode,"-demo",       "d",  "Run a demo mode (cycle through range of camera options, no capture)", 1},
    { CommandFramerate,"-framerate", "fps","Specify the frames per second to record", 1},
    { CommandPreviewEnc,"-penc",     "e",  "Display preview image *after* encoding (shows compression artifacts)", 0},
+   { CommandNoTimeout,"-notimeout", "nt",  "Disable timeout when capturing video", 0},
 };
 
 static int cmdline_commands_size = sizeof(cmdline_commands) / sizeof(cmdline_commands[0]);
@@ -181,6 +184,7 @@ static void default_status(RASPIVID_STATE *state)
 
    // Now set anything non-zero
    state->timeout = 5000;     // 5s delay before take image
+   state->notimeout = 0;      // Default to obeying the timeout
    state->width = 1920;       // Default to 1080p
    state->height = 1080;
    state->bitrate = 17000000; // This is a decent default bitrate for 1080p
@@ -210,7 +214,7 @@ static void dump_status(RASPIVID_STATE *state)
    }
 
    fprintf(stderr, "Width %d, Height %d, filename %s\n", state->width, state->height, state->filename);
-   fprintf(stderr, "bitrate %d, framerate %d, time delay %d\n", state->bitrate, state->framerate, state->timeout);
+   fprintf(stderr, "bitrate %d, framerate %d, time delay %d, no timeout %d\n", state->bitrate, state->framerate, state->timeout, state->notimeout);
 
    raspipreview_dump_parameters(&state->preview_parameters);
    raspicamcontrol_dump_parameters(&state->camera_parameters);
@@ -363,6 +367,10 @@ static int parse_cmdline(int argc, const char **argv, RASPIVID_STATE *state)
          state->immutableInput = 0;
          break;
 
+      case CommandNoTimeout:
+         state->notimeout = 1;
+         break;
+         
       default:
       {
          // Try parsing for any image specific parameters
@@ -1037,12 +1045,11 @@ int main(int argc, const char **argv)
          {
             // Run for the user specific time..
             int num_iterations = state.timeout / state.demoInterval;
-            int i;
 
             if (state.verbose)
                fprintf(stderr, "Running in demo mode\n");
 
-            for (i=0;i<num_iterations;i++)
+            while (state.notimeout || num_iterations--)
             {
                raspicamcontrol_cycle_test(state.camera_component);
                vcos_sleep(state.demoInterval);
@@ -1084,18 +1091,27 @@ int main(int argc, const char **argv)
                // out of storage space)
                // Going to check every ABORT_INTERVAL milliseconds
 
-               for (wait = 0; wait < state.timeout; wait+= ABORT_INTERVAL)
+               for (wait = 0; wait < state.timeout;)
                {
                   vcos_sleep(ABORT_INTERVAL);
                   if (callback_data.abort)
                      break;
+                     
+                  // Only increase the wait time if we are obeying the timeout
+                  if(!state.notimeout) {
+                  	wait+= ABORT_INTERVAL;
+                  }
                }
 
                if (state.verbose)
                   fprintf(stderr, "Finished capture\n");
             }
-            else
+            else {
+            	// If we are not obeying the timeout then loop forever here
+            	do {
                vcos_sleep(state.timeout);
+              } while(!state.notimeout);
+            }
          }
       }
       else
-- 
1.7.9.5
Leo

texy
Forum Moderator
Forum Moderator
Posts: 5174
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 7:52 pm

I believe it's 11.57 days ;-)
Texy
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555

LeoWhite
Posts: 131
Joined: Mon Apr 08, 2013 9:20 am

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 7:59 pm

texy wrote:I believe it's 11.57 days ;-)
Mmhh I just got 24.85 days as the max :)

Leo

texy
Forum Moderator
Forum Moderator
Posts: 5174
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 8:08 pm

LeoWhite wrote:
texy wrote:I believe it's 11.57 days ;-)
Mmhh I just got 24.85 days as the max :)

Leo
I thought it was 999999999ms, which is 11.57 days :?:

T.
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555

LeoWhite
Posts: 131
Joined: Mon Apr 08, 2013 9:20 am

Re: Streaming mjpg from Raspberry Cam

Tue May 28, 2013 8:20 pm

texy wrote:
LeoWhite wrote:
texy wrote:I believe it's 11.57 days ;-)
Mmhh I just got 24.85 days as the max :)

Leo
I thought it was 999999999ms, which is 11.57 days :?:
The timeout is stored as a 32bit integer value, so the maximum possible value should be 2147483647ms. Its just much easier to put in a bunch of 9s than it is to remember that value :)

Not that I've gone and actually tried that myself but having just looked through the code it should work (There does appear to be an issue where the timeout is read in as an unsigned value, which may explain why the -1 thing was working)

Leo

TarjeiB
Posts: 157
Joined: Thu Jul 12, 2012 3:33 pm

Re: Streaming mjpg from Raspberry Cam

Thu May 30, 2013 10:31 am

Digging a bit more here, the input_file.so seems to be using inotify to check if there are new files.
This will never be fast, so without a new plugin we're stuck at horrid framerates.

Unfortunately I'm not at all a coder - I looked through the input_file.c and tried to find examples of how to make C read from stdin, but it's not possible for me to figure out by just reading the code by a long shot.

The most updated mjpg-streamer seems to be the experimental version at github:
https://github.com/codewithpassion/mjpg ... perimental

That is the one I've compiled on the Pi, if someone wants to look at the plugins and see if it's possible to make a input_stdin one it'd be a huge step forward :)

AntoineDUBOSTBULARD
Posts: 8
Joined: Wed Apr 10, 2013 7:17 am

Re: Streaming mjpg from Raspberry Cam

Thu May 30, 2013 11:48 am

Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB

TarjeiB
Posts: 157
Joined: Thu Jul 12, 2012 3:33 pm

Re: Streaming mjpg from Raspberry Cam

Thu May 30, 2013 9:26 pm

AntoineDUBOSTBULARD wrote:Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB
I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?

sharix
Posts: 200
Joined: Thu Feb 16, 2012 11:29 am
Location: Slovenia

Re: Streaming mjpg from Raspberry Cam

Thu May 30, 2013 10:29 pm

There should be an option to only adjust the lighting levels, awb, etc before the first shot instead of before every shot. someone get to it :D

AntoineDUBOSTBULARD
Posts: 8
Joined: Wed Apr 10, 2013 7:17 am

Re: Streaming mjpg from Raspberry Cam

Fri May 31, 2013 4:58 am

TarjeiB wrote:
AntoineDUBOSTBULARD wrote:Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first thought that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB
I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?
TarjeiB, can you tell me the exact commands your are using for achieving such frame rates ?
Mine is :
$ mkdir /tmp/stream
$ raspistill -w 1920 -h 1080 -q 25 -o /tmp/stream/pic.jpg -tl 40 -t 9999999 -th 0:0:0 | ./mjpg_streamer -i "./input_file.so -f /tmp/stream" -o "./output_http.so -w ./www"

Changing the resolution do not change anything

Thanks

ADB

TarjeiB
Posts: 157
Joined: Thu Jul 12, 2012 3:33 pm

Re: Streaming mjpg from Raspberry Cam

Fri May 31, 2013 1:17 pm

AntoineDUBOSTBULARD wrote: TarjeiB, can you tell me the exact commands your are using for achieving such frame rates ?
Mine is :
$ mkdir /tmp/stream
$ raspistill -w 1920 -h 1080 -q 25 -o /tmp/stream/pic.jpg -tl 40 -t 9999999 -th 0:0:0 | ./mjpg_streamer -i "./input_file.so -f /tmp/stream" -o "./output_http.so -w ./www"
ADB

Code: Select all

$ raspistill -t 999999 -tl 50 -o /tmp/mjpg/test.jpg -n -w 960 -h 720&
$ mjpg_streamer -i 'input_file.so -f /tmp/mjpg -d 0'&
Goes quite fast! Make sure /tmp is a ramdisk. Not sure the pipe is a good idea either, I run them as separate commands.

sergkit
Posts: 1
Joined: Fri Jun 07, 2013 10:05 am

Re: Streaming mjpg from Raspberry Cam

Fri Jun 07, 2013 10:13 am

Does anybody rich normal fps with streaming RaspberryCam? I have get only 0.2 fps

Code: Select all

raspistill -t 999999 -tl 50 -o /tmp/test.jpg -n -w 960 -h 720

Code: Select all

./mjpg_streamer -i './input_file.so -f /tmp/ -d 0' -o "./output_http.so -w ./www -p 8080"

User avatar
jacksonliam
Posts: 181
Joined: Tue Feb 07, 2012 10:09 pm

Re: Streaming mjpg from Raspberry Cam

Fri Jun 07, 2013 4:42 pm

TarjeiB wrote:
AntoineDUBOSTBULARD wrote:Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB
I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?
How did you achieve this >~7fps? I've been playing with the code with aim to write an mjpeg streamer input plugn and in a simple capture->encode loop the fastest i seem to get is about 12fps. Resolution makes very little difference here, and it requires around a 100ms delay between captures else the picture is too dark or just black.


For anyone who is interested, here's my work on an mjpeg streamer plugin for raspicam https://github.com/jacksonliam/mjpg-streamer

TonAu
Posts: 26
Joined: Sat Jul 21, 2012 11:23 am

Re: Streaming mjpg from Raspberry Cam

Sun Jun 09, 2013 11:34 am

I downloaded the experimental mjpgstreamer from github and it works fine for a couple of minutes.

Had to apt-get cmake and libjpeg8-dev

Also, the makefile does not find a directory 'build' in the raspicam plugin and then stops with an error.
That directory is not in the repository. I just did 'mkdir build' there and after that cmake went on and the make finished succesfully.

The webserver starts with 1 fps approximately. But after 800 frames or so the picture slowly turns dark and ends up black.
Then I have to restart mjpg-streamer.
Any clue what that can be?
update:
- a delay of 300 seems to be working better: now for hours already, with 5-6% cpu load. (I specified x and y to 640 480, should make no diff)

User avatar
jacksonliam
Posts: 181
Joined: Tue Feb 07, 2012 10:09 pm

Re: Streaming mjpg from Raspberry Cam

Mon Jun 10, 2013 12:06 pm

Glad your trying it out. I fixed the build folder missing thing. Didn't know you can't have empty github folders.

I think I have libjpeg62-dev. I've not seen the darker images thing and I've run this for ~24hrs on the default 1000 delay. Wouldn't have expected a shorter delay to help... I wonder if this is related:
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=45783
Like raspistill, I keep the camera open the entire time, though it does seem to do all the light level stuff on each shot, I don't think its possible to have it only do it once so you can get faster stills at the same light level - I've had a good look though threads on here and the source and can't find anything that says so.

which firmware are you on? I dunno if a rpi-update would help?

Code: Select all

/opt/vc/bin/vcgencmd version
Jun  6 2013 18:30:04
Copyright (c) 2012 Broadcom
version e6f0abde032163069972925f175c4a3a5617d46d (clean) (release)

Return to “Camera board”