stefan.knight
Posts: 24
Joined: Tue Dec 17, 2013 7:43 pm

Re: RPi Cam Web Interface

Mon Jan 06, 2014 8:03 pm

Hi Silvan,
Glad to see motion included now. Can you explain a bit how motion is used vs. the live view. If I understand correctly, Live view is MJPEG you did with RaspiMJPEG and motion is being done natively with motion-mmal and both are using the H.264 preview?

Also, not sure if you already knew, but the current version of PHP already includes an HTTP server so Apache is not necessary. I tried the PHP HTTP server and it it seems more responsive.

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Mon Jan 06, 2014 10:43 pm

Great work! This is the very first time that motion-detecting with full-HD video is available from the R-Pi's camera. I think this ought to be headline news, it is one of the big use-cases for the R-Pi camera. (I only just installed it, I can't comment yet on long-term reliability.)
stefan.knight wrote:Glad to see motion included now. Can you explain a bit how motion is used vs. the live view.
Looking at the motion config file /etc/motion/motion.conf I see this:

Code: Select all

netcam_url file:///dev/shm/mjpeg/cam.jpg
gap 3
on_event_start echo 'ca 1' > /var/www/FIFO
on_event_end echo 'ca 0' > /var/www/FIFO
which means that the 'motion' process is not doing anything directly with the R-Pi camera (and this is the stock 'motion' package, not motion-mmal). You can see that motion is only looking at the preview MJPEG being generated by the 'raspimjpeg' process, and telling it via commands into the named pipe /var/www/FIFO to start recording the full-HD H.264 video when motion is detected, and stop recording 3 seconds after no motion is detected. The video files are created in /var/www/media and available for preview and download through the web interface.

One addition comes immediately to mind (I think fairly simple): include the filesize in MB along with the file list on the media download page. This gives you an idea for each movie if it is short or long, without having to download it.

EDIT: here is a replacement for /var/www/preview.php which includes the file size in megabytes (MB).
(Edit2: fixed copy/paste issue.)

Code: Select all

<!DOCTYPE html>
<html>
  <head>
    <title>RPi Cam Download</title>
  </head>
  <body>
    <p><a href="index.html">Back</a></p>
    <?php
      if(isset($_GET["delete"])) {
        unlink("media/" . $_GET["delete"]);
      }
      else if(isset($_GET["file"])) {
        echo "<h1>Preview</h1>";
        if(substr($_GET["file"], -3) == "jpg") echo "<img src='media/" . $_GET["file"] . "' width='480' height='360'>";
        else echo "<video width='640' height='360' controls><source src='media/" . $_GET["file"] . "' type='video/mp4'>Your browser does not support the video tag.</video>";
        echo "<p><input type='button' value='Download' onclick='window.open(\"download.php?file=" . $_GET["file"] . "\", \"_blank\");'> ";
        echo "<input type='button' value='Delete' onclick='window.location=\"preview.php?delete=" . $_GET["file"] . "\";'></p>";
      }
    ?>
    <h1>Files</h1>
    <?php
      $files = scandir("media");
      if(count($files) == 2) echo "<p>No videos/images saved</p>";
      foreach($files as $file) {
        $fsz = round ((filesize("media/" . $file)) / (1024 * 1024));
        if(($file != '.') && ($file != '..')) {
          echo "<p><a href='preview.php?file=$file'>$file</a> $fsz MB</p>";
        }
      }
    ?>    
  </body>
</html>
And an example of the output:

Code: Select all

Back

Recorded Files

video_0007_20140106_144822.mp4 17 MB

video_0008_20140106_144914.mp4 5 MB

video_0009_20140106_144922.mp4 7 MB

video_0010_20140106_144944.mp4 7 MB

video_0011_20140106_144958.mp4 2 MB
Last edited by jbeale on Tue Jan 07, 2014 6:01 am, edited 3 times in total.

stefan.knight
Posts: 24
Joined: Tue Dec 17, 2013 7:43 pm

Re: RPi Cam Web Interface

Mon Jan 06, 2014 11:05 pm

ah..wonderful...thank you jbeale for the detailed explanation. I guess if it's recording full-HD would it be possible to have it record on a network share via WiFi or on the SD card then a move to the network share?

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Mon Jan 06, 2014 11:42 pm

stefan.knight wrote:ah..wonderful...thank you jbeale for the detailed explanation. I guess if it's recording full-HD would it be possible to have it record on a network share via WiFi or on the SD card then a move to the network share?
You can record to a network share but I would not recommend doing that via wifi. Even if it works at first, in my experience it is unlikely to work all the time (the wifi adaptors I've used on the R-Pi do not have enough bandwidth to reliably support full-HD). Not sure what happens if your links stalls while recording- does it lock up the whole system? Anyway if you are hardwired to the network (ethernet) it should be no trouble. If you have only a wifi link, it is much better to record it locally to SD card or to USB thumb drive, then transfer the file after recording, when it's not a real-time critical process.

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Tue Jan 07, 2014 12:21 am

Observation: in moderate to dim lighting (typical indoor office) the video output recorded by the R-Pi has some heavy temporal noise filtering going on. Anything in motion is very blurred, like it's averaging together 3 or 4 frames, even while the fixed background remains crisp and detailed. For a security camera you much prefer a noisy but detailed image, over a smooth but blurry one. It would be nice to have control over the amount of noise reduction going on. The downside (apart from noisy textures) would be a big hit on compression efficiency, so I understand why it is the way it is, but it's not optimal for some use cases.

UPDATE: I've had the RPi Web interface with motion detection running overnight (about 7 hours) on three different R-Pis now. The raspimjpeg program has been stable so far on all systems, so that's good. In a room where the lights stayed on, it worked as expected: recorded video when there was motion, and not recording when there was none.

I do notice that with the cameras looking outdoors during the night (when image is nearly or completely black) I still get hundreds of motion detections over the course of the night. The recorded video is generally all dark. I also set the '/etc/motion/motion.conf' configuration file to record stills with a rectangle around the area of motion detected to see what was happening, and during the night the saved images show a frame at the lower right hand corner, just around the time/date stamp that motion provides. This seems to be bug in motion itself- surely the program should run the motion-detection algorithm *before* adding a time/date stamp to the image?

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Tue Jan 07, 2014 6:24 pm

I don't know PHP at all, but I worked out enough syntax to modify the '/var/www/preview.php' script to add "delete all JPEGs" and "delete all MP4s" control buttons. Not great style I'm sure, but it seems to work for me anyway.

Code: Select all

<!DOCTYPE html>
<html>
  <head>
    <title>RPi Cam Download</title>
  </head>
  <body>
    <p><a href="index.html">Back</a></p>
    <?php
      if(isset($_GET["delete"])) {
        unlink("media/" . $_GET["delete"]);
      }
      if(isset($_GET["deljpg"])) {
        $files = scandir("media");
	foreach($files as $file) {
	  $ext = pathinfo($file, PATHINFO_EXTENSION);
	  if($ext == 'jpg') {
            unlink("media/" . $file);
          }
	}
      }
      if(isset($_GET["delmp4"])) {
        $files = scandir("media");
	foreach($files as $file) {
	  $ext = pathinfo($file, PATHINFO_EXTENSION);
	  if($ext == 'mp4') {
            unlink("media/" . $file);
          }
	}
      }
      else if(isset($_GET["file"])) {
        echo "<h1>Preview</h1>";
        if(substr($_GET["file"], -3) == "jpg") echo "<img src='media/" . $_GET["file"] . "' width='480' height='360'>";
        else echo "<video width='640' height='360' controls><source src='media/" . $_GET["file"] . "' type='video/mp4'>Your browser does not support the video tag.</video>";
        echo "<p><input type='button' value='Download' onclick='window.open(\"download.php?file=" . $_GET["file"] . "\", \"_blank\");'> ";
        echo "<input type='button' value='Delete' onclick='window.location=\"preview.php?delete=" . $_GET["file"] . "\";'> ";
      }
        echo "<input type='button' value='Delete JPEGs' onclick='window.location=\"preview.php?deljpg=" . Y . "\";'> ";
        echo "<input type='button' value='Delete MP4s' onclick='window.location=\"preview.php?delmp4=" . Y . "\";'></p>";
    ?>
    <h1>Recorded Files</h1>
    <?php
      $files = scandir("media");
      if(count($files) == 2) echo "<p>No videos/images saved</p>";
      foreach($files as $file) {
        $fsz = round ((filesize("media/" . $file)) / (1024 * 1024)+0.01,2); # filesize to 2 digits, as 3.14 MB
        if(($file != '.') && ($file != '..')) {
          echo "<p><a href='preview.php?file=$file'>$file</a> $fsz MB</p>";
	}
      }
    ?>
  </body>
</html>
I also made the motion detection more responsive by reducing the MJPEG resolution down to 256x144 in /etc/rc.local and increasing the framerate to 4 in /etc/motion/motion.conf so the recording starts with less delay. I still get full-HD video, the lower res only affects the MJPEG preview.
Last edited by jbeale on Wed Jan 08, 2014 5:53 am, edited 1 time in total.

buzzai
Posts: 7
Joined: Sat Oct 12, 2013 2:01 pm

Re: RPi Cam Web Interface

Tue Jan 07, 2014 6:43 pm

jbeale wrote:...the saved images show a frame at the lower right hand corner, just around the time/date stamp that motion provides. This seems to be bug in motion itself- surely the program should run the motion-detection algorithm *before* adding a time/date stamp to the image?
I sometimes see gray blocks in the same lower-right corner which motion interprets as movement. I looks like it has something to do with decoding of the jpeg images...
Maybe the image being read by motion gets overwritten by raspimjpeg when it isn't processed fast enough.
Increasing the load on my RPi should show an increase in these gray blocks causing motion detection...

User avatar
DougieLawson
Posts: 41955
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: RPi Cam Web Interface

Tue Jan 07, 2014 6:44 pm

buzzai wrote:
jbeale wrote:...the saved images show a frame at the lower right hand corner, just around the time/date stamp that motion provides. This seems to be bug in motion itself- surely the program should run the motion-detection algorithm *before* adding a time/date stamp to the image?
I sometimes see gray blocks in the same lower-right corner which motion interprets as movement. I looks like it has something to do with decoding of the jpeg images...
Maybe the image being read by motion gets overwritten by raspimjpeg when it isn't processed fast enough.
Increasing the load on my RPi should show an increase in these gray blocks causing motion detection...
Have you done a sudo rpi-update to ensure you're running the latest & greatest camera drivers and software?
Languages using left-hand whitespace for syntax are ridiculous

DMs sent on Twitter/LinkedIn will be answered next month.
Fake doctors - are all on my foes list.

The use of crystal balls and mind reading is prohibited.

buzzai
Posts: 7
Joined: Sat Oct 12, 2013 2:01 pm

Re: RPi Cam Web Interface

Tue Jan 07, 2014 6:59 pm

DougieLawson wrote:Have you done a sudo rpi-update to ensure you're running the latest & greatest camera drivers and software?
A few days ago, yes.
But I only see these gray block in motion. Never in the mjpeg stream, the recorded video or stills...

Looking at the raspimjpeg code, the mjpeg stream jpeg's are allocated on disk with extension ".part", which get renamed to the mjpeg stream's filename once the image has been completely written to disk:

Code: Select all

static void jpegencoder_buffer_callback (MMAL_PORT_T *port, MMAL_BUFFER_HEADER_T *buffer) {
...
  if(mjpeg_cnt == 0) {
    if(!jpegoutput_file) {
      asprintf(&filename_temp, jpeg_filename, image_cnt);
      asprintf(&filename_temp2, "%s.part", filename_temp);
      jpegoutput_file = fopen(filename_temp2, "wb");
...
    }
    if(buffer->length) {
      mmal_buffer_header_mem_lock(buffer);
      bytes_written = fwrite(buffer->data, 1, buffer->length, jpegoutput_file);
      mmal_buffer_header_mem_unlock(buffer);
    }
    if(bytes_written != buffer->length) error("Could not write all bytes");
  }
  
  if(buffer->flags & MMAL_BUFFER_HEADER_FLAG_FRAME_END) {
...
      asprintf(&filename_temp, jpeg_filename, image_cnt);
      asprintf(&filename_temp2, "%s.part", filename_temp);
      rename(filename_temp2, filename_temp);
...
    }
  }
...
What will happen when motion has just opened one of these jpeg's for reading when raspimjpeg decides to overwrite the file by renaming a new jpeg to the mjpeg stream's filename?

cupcake
Posts: 59
Joined: Sat Jan 21, 2012 9:00 pm

Re: RPi Cam Web Interface

Tue Jan 07, 2014 7:25 pm

Thanks a bunch for posting your work, Silvian. Your implementation is exactly what I need. Low-latency live feed with the option of recording high-quality images/videos and various video picture configuration options. I'll be using this for my rover.

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Tue Jan 07, 2014 7:52 pm

buzzai wrote:I sometimes see gray blocks in the same lower-right corner which motion interprets as movement. I looks like it has something to do with decoding of the jpeg images...
Maybe the image being read by motion gets overwritten by raspimjpeg when it isn't processed fast enough.
Increasing the load on my RPi should show an increase in these gray blocks causing motion detection...
I think I'm getting the same thing, so maybe it is a JPEG decoding artifact, but I see it only when the image is nearly or entirely black. Maybe under those conditions 'motion' automatically turns up the sensitivity, struggling to see something where there is nothing? But I thought I had turned adaptive sensitivity off in the config file? but maybe I don't understand the config file well enough. All 3 R-Pis I'm using for testing this were fully updated including sudo apt-get update, dist-upgrade and rpi-update yesterday (Jan. 6 2014).

User avatar
Mrbcsimpson
Posts: 17
Joined: Tue Jan 07, 2014 9:29 pm
Location: Lincolnshire, UK
Contact: Website

Re: RPi Cam Web Interface

Tue Jan 07, 2014 9:49 pm

I signed up especially to say thank you to the author of this post. What a fantastic little tutorial for a newcomer to the RPi scene. My aim was to have a way to check up on the dog, keep an eye out for intruders.. but mainly to sit and play with a credit card-sized computer late in to the night :D

I appear to have a couple of issues which I put down to trying other tutorials before this one and not reflashing the SD card first:

Multiple videos over a few minutes even with motion sensor function off.
Extremely slow frame rate, presumably down to the low powered nano wifi USB.

Thanks again for this great guide. My second RPi in the house is equally as useful as the first (running XBMC from the NAS to the TV)!

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Tue Jan 07, 2014 10:12 pm

Misc. useful tip: the application starts with a default bitrate of 17 Mb/sec for saving H.264 video. In my experience this is higher than is useful if you've got an indoor or low-light situation, it just creates larger files and extends the "dead time" after each motion recording when the .h264 gets converted to .mp4 using MP4Box, during which time additional motion is not detected. I recommend about 5 Mb/sec if you're indoors, this gives me about as much useful detail and is much quicker to convert to mp4, preview, and download.

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Wed Jan 08, 2014 6:39 am

Mrbcsimpson wrote:Extremely slow frame rate, presumably down to the low powered nano wifi USB.
I get very low framerates too, using the tiny cheap Realtek RTL8188CUS USB-wifi adaptors. If you have a Model B and use ethernet it is a whole lot faster. There are also better quality wifi adaptors, with better data rate but still not as good as ethernet.

Two out of the three R-Pi's I've tried this on give me grey blocks (JPEG artifact?) at the lower right hand corner. They are visible only in still frames output from motion, not in the MJPEG preview or the mp4 video. At first I thought this was only when it was dark, but it can happen with a lighter image also. I tried using a smaller MJPEG image size, 256x144 as you see here but it still happens. My original unit was overclocked and I reverted back to 700 MHz default but it still happens. When this block appears it triggers the motion algorithm and give me hundreds of false-detects in the space of several hours, making the whole thing unusable. It seems to be related to 'motion' somehow, since the MJPEG preview never shows it.
GreyBlock-20140107_222727.jpg
GreyBlock-20140107_222727.jpg (2.36 KiB) Viewed 19440 times
GreyBlock2-20140107_230355.jpg
GreyBlock2-20140107_230355.jpg (4.58 KiB) Viewed 19430 times
Interestingly, a third R-Pi does not show this artifact in almost a full day of using the motion detector. All three R-Pi's are all updated to the same software. The R-Pi that works OK is a Model A. I have one each Model A and Model B that have the grey blocks.
UPDATE: the third R-Pi does indeed show grey blocks, it just happens less often.
Last edited by jbeale on Wed Jan 08, 2014 7:13 pm, edited 1 time in total.

buzzai
Posts: 7
Joined: Sat Oct 12, 2013 2:01 pm

Re: RPi Cam Web Interface

Wed Jan 08, 2014 8:14 am

FWIW, I observed exactly the same gray blocks in the lower right corner.
On my RPi they're also visible in the video stream generated by motion. Motion nicely 'labels' the gray area as motion detected area, so this definately triggers the motion detection, and not the timestamp.

jbeale, as you've got quite some pi's up and running, could you test if this behaviour changes under heavy load (e.g. compiling something in the background) or when raising motion's priority to realtime:

Code: Select all

sudo chrt -a -r -p 99 `pgrep motion`
Thanks!

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Wed Jan 08, 2014 3:59 pm

buzzai wrote:FWIW, I observed exactly the same gray blocks in the lower right corner.
On my RPi they're also visible in the video stream generated by motion. Motion nicely 'labels' the gray area as motion detected area, so this definately triggers the motion detection, and not the timestamp.

jbeale, as you've got quite some pi's up and running, could you test if this behaviour changes under heavy load (e.g. compiling something in the background) or when raising motion's priority to realtime:

Code: Select all

sudo chrt -a -r -p 99 `pgrep motion`
Thanks!
I tried this command and confirmed motion was running at realtime priority in 'top' but the grey blocks happen just the same.

Now that it is daylight outside I do notice that the camera looking outside is having this problem much less frequently. Could it have to do some timing race condition related to the camera shutter speed (which is faster during daytime) ? When I was working with the "pure python camera interface" I noticed that I would get an error sometimes that was solved by waiting after the camera.capture() call before reading the camera data stream, but it only happened in dark conditions with slow shutter times. See also: http://www.raspberrypi.org/forum/viewto ... 75#p461876

sedonami
Posts: 17
Joined: Wed Jan 08, 2014 5:20 pm

Re: RPi Cam Web Interface

Wed Jan 08, 2014 5:32 pm

I first installed this last night and it worked flawlessly for about 3 minutes before it froze the pi. Since I was playing around with quite a few other solutions I decided to re-flash the os (Raspian) and start fresh.

After a completely fresh installation I re-ran the installation instructions. The RPi Cam Control interface came up with an image for a second and then the pi completely locked up again. Now, even after a reboot (hard reset) I cannot ssh into the the pi. The camera light is lit so my guess is that the raspimpeg was initialized however it appears that something is causing a kernel panic.

Any ideas on how I might go about troubleshooting this? Anyone else experience the same issue?

Here is what I did on a fresh raspian install.

Code: Select all

sudo rapsi-config (enabled camera)

<reboot>

sudo apt-get update
sudo apt-get dist-upgrade
sudo rpi-update
<reboot>

wget -N http://grustu.ch/share/rpi_cam/RPi_Cam_Browser_Control_Installer.sh
chmod u+x RPi_Cam_Browser_Control_Installer.sh
./RPi_Cam_Browser_Control_Installer.sh install

<reboot> 
Thanks in advance!

UPDATE: so, I was able to login to the pi via ssh immediately after a hard reboot and stop the raspimpeg service. Once this was done I ran the following command as I read in other threads that initializing the camera via raspistill needed to take place.

Code: Select all

raspistill -o test.jpg
After doing this and restarting the raspimpeg service it appears to be working beautifully! Very impressed with how this just works, thanks so much!

UPDATE 2 After about 30 minutes of running pi locked up again. Not sure why.
Last edited by sedonami on Wed Jan 08, 2014 8:35 pm, edited 2 times in total.

buzzai
Posts: 7
Joined: Sat Oct 12, 2013 2:01 pm

Re: RPi Cam Web Interface

Wed Jan 08, 2014 6:28 pm

jbeale wrote:Now that it is daylight outside I do notice that the camera looking outside is having this problem much less frequently. Could it have to do some timing race condition related to the camera shutter speed (which is faster during daytime) ?
I would expect a race condition to occur more often at high frame rates/short shutter times, or under high load as I suggested before. But hey, who am I :|

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Wed Jan 08, 2014 7:08 pm

Unless you select "night" mode I believe the camera frame rate is normally fixed at 30 fps (or other setting you select) so a faster shutter means more time to handle image data before the next frame time. At any rate the grey block at lower right corner is quite odd: consider the frame below. It looks like the time/date stamp is partially in front and partially behind the grey block (unless my character set got changed to Arabic?)
20140107_214806.jpg
20140107_214806.jpg (8.18 KiB) Viewed 19349 times
Looks like JPEG data is getting corrupted somewhere and maybe that is the cause of sedonami's lockup as well? Were you using 'motion' or not?

sedonami
Posts: 17
Joined: Wed Jan 08, 2014 5:20 pm

Re: RPi Cam Web Interface

Wed Jan 08, 2014 7:57 pm

jbeale wrote:Unless you select "night" mode I believe the camera frame rate is normally fixed at 30 fps (or other setting you select) so a faster shutter means more time to handle image data before the next frame time. At any rate the grey block at lower right corner is quite odd: consider the frame below. It looks like the time/date stamp is partially in front and partially behind the grey block (unless my character set got changed to Arabic?)
20140107_214806.jpg
Looks like JPEG data is getting corrupted somewhere and maybe that is the cause of sedonami's lockup as well? Were you using 'motion' or not?
jbeale, I've updated my post. After initializing camera with raspistill first my lock issue has gone away. Not entirely sure why that worked but it did. I am just getting started with motion and will report my findings once I have it all sorted.

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

RPi Cam Web Interface - MOTION gets partial JPEG file?

Wed Jan 08, 2014 11:16 pm

How about this for a theory of how the spurious grey blocks at the bottom-right corner that trigger motion get there:

Raspimjpeg writes new image data into "cam.jpg.part" and then when finished, renames that to cam.jpg. The rename operation I thought is an atomic operation, replacing the previous cam.jpg. But perhaps there is some pipelining going on under the surface, where the file isn't quite complete when the rename happens and the local "motion" process grabs it from ramdisk too soon. Or the data has changed but the metadata (file length) hasn't. Meanwhile the network going to the remote browser viewing the MJPEG is too slow to see this unfinished business and always gets the complete file of correct length. Plausible?

How is MJPEG supposed to work anyway? Should there be some kind of handshake between the jpeg file writer and the jpeg file reader? Otherwise wouldn't incomplete file reads happen all the time?

If this is the problem, I suppose you could have some fixed time delay between finishing writing the file, and the rename step. But fixed time delays feel like a hack-fix that may not always work.

Seems like the JPEG gets written like this (from RaspiMJPEG.c )

Code: Select all

static void jpegencoder_buffer_callback (MMAL_PORT_T *port, MMAL_BUFFER_HEADER_T *buffer) {

  int bytes_written = buffer->length;
  char *filename_temp, *filename_temp2;

  if(mjpeg_cnt == 0) {
    if(!jpegoutput_file) {
      asprintf(&filename_temp, jpeg_filename, image_cnt);
      asprintf(&filename_temp2, "%s.part", filename_temp);
      jpegoutput_file = fopen(filename_temp2, "wb");
      free(filename_temp);
      free(filename_temp2);
      if(!jpegoutput_file) error("Could not open mjpeg-destination");
    }
    if(buffer->length) {
      mmal_buffer_header_mem_lock(buffer);
      bytes_written = fwrite(buffer->data, 1, buffer->length, jpegoutput_file);
      mmal_buffer_header_mem_unlock(buffer);
    }
    if(bytes_written != buffer->length) error("Could not write all bytes");
  }
  
  if(buffer->flags & MMAL_BUFFER_HEADER_FLAG_FRAME_END) {
    mjpeg_cnt++;
    if(mjpeg_cnt == divider) {
      fclose(jpegoutput_file);
      jpegoutput_file = NULL;
      asprintf(&filename_temp, jpeg_filename, image_cnt);
      asprintf(&filename_temp2, "%s.part", filename_temp);
      rename(filename_temp2, filename_temp);
      free(filename_temp);
      free(filename_temp2);
      image_cnt++;
      mjpeg_cnt = 0;
    }
  }

  mmal_buffer_header_release(buffer);

  if (port->is_enabled) {
    MMAL_STATUS_T status = MMAL_SUCCESS;
    MMAL_BUFFER_HEADER_T *new_buffer;

    new_buffer = mmal_queue_get(pool_jpegencoder->queue);

    if (new_buffer) status = mmal_port_send_buffer(port, new_buffer);
    if (!new_buffer || status != MMAL_SUCCESS) error("Could not send buffers to port");
  }

}
Not sure I understand it actually... is it guaranteed that one ( MMAL_BUFFER_HEADER_T *buffer ) always contains one complete JPEG image? Or can one JPEG image be split across more than one buffer? Anyway I tried rewriting the code assuming split buffers are possible, but it worked the same.

I also modified this code to put the fclose(jpegoutput_file) immediately after the FRAME_END flag test, and set "-d 5" in the /etc/rc.local invocation of raspimjpeg so that there are 5 frame times between closing the /dev/shm/mjpeg/cam.jpg.part file and the subsequent renaming of it to cam.jpg but sadly this did not help. The grey blocks that trip up "motion" still appear.

I even tried changing the MJPEG directory (referenced in 3 files: /etc/motion/motion.conf, /etc/rc.local, /var/www/cam_pic.php ) to be on flash card /tmp instead of /run/shm but the grey blocks still show up.
Last edited by jbeale on Thu Jan 09, 2014 1:40 am, edited 2 times in total.

sedonami
Posts: 17
Joined: Wed Jan 08, 2014 5:20 pm

Re: RPi Cam Web Interface

Thu Jan 09, 2014 12:51 am

jbeale wrote:Unless you select "night" mode I believe the camera frame rate is normally fixed at 30 fps (or other setting you select) so a faster shutter means more time to handle image data before the next frame time. At any rate the grey block at lower right corner is quite odd: consider the frame below. It looks like the time/date stamp is partially in front and partially behind the grey block (unless my character set got changed to Arabic?)
20140107_214806.jpg
Looks like JPEG data is getting corrupted somewhere and maybe that is the cause of sedonami's lockup as well? Were you using 'motion' or not?
jbeale, have you tried to use a mask file to block that area from being detected?
http://www.lavrsen.dk/foswiki/bin/view/ ... #mask_file

I haven't had a chance to try this yet as my pi is at home, however assuming this works, you should be able to block the gray area of your image from tripping the motion? Not a permanent solution unfortunately.

User avatar
jbeale
Posts: 3898
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Thu Jan 09, 2014 4:10 am

sedonami wrote:jbeale, have you tried to use a mask file to block that area from being detected?
http://www.lavrsen.dk/foswiki/bin/view/ ... #mask_file
Yes! Using the mask to disable detection of the bottom-right grey square is working so far. Perhaps not ideal but it changes "unworkable" into "usable" which is a big advance. I added to /etc/motion/motion.conf :

Code: Select all

mask_file /etc/motion/512-288-mask.pgm
along with putting the pgm file itself in /etc/motion/. The file must match the MJPEG resolution which is 512x288 unless you changed it in /etc/rc.local. This particular mask occupies the entire bottom line and sometimes the grey block even occupies more space; but not too often (so far once in several hours).
512-288-mask.zip
512 x 288 mask file in PGM format for fixing false-detect "motion" problem
(334 Bytes) Downloaded 585 times
Of course the correct way to fix this is to get motion to read in the complete JPEG image each time, which should be possible given that the apache webserver always manages it. I just don't know how to do that yet.

User avatar
Mrbcsimpson
Posts: 17
Joined: Tue Jan 07, 2014 9:29 pm
Location: Lincolnshire, UK
Contact: Website

Re: RPi Cam Web Interface

Thu Jan 09, 2014 5:51 pm

Is anyone aware of how I could change the following?

- Camera sensitivity (I get around 20 videos per 10 mins in an empty, evenly lit room)
- Change the port (I would like to port forward the Pi to view outside my home network)

buzzai
Posts: 7
Joined: Sat Oct 12, 2013 2:01 pm

Re: RPi Cam Web Interface - MOTION gets partial JPEG file?

Thu Jan 09, 2014 6:20 pm

jbeale wrote:Raspimjpeg writes new image data into "cam.jpg.part" and then when finished, renames that to cam.jpg. The rename operation I thought is an atomic operation, replacing the previous cam.jpg. But perhaps there is some pipelining going on under the surface, where the file isn't quite complete when the rename happens and the local "motion" process grabs it from ramdisk too soon....
I had a quick look at the motion code (using jpegutils to decode jpegs) and it seems it mmap's the mjpeg file and reads the frames from there.
Referring to http://stackoverflow.com/questions/1462 ... aced-on-di (see the response to the question) the contents of this mmap'ed file change unnoticed as soon as this file is renamed...
SO it seems like motion is still processing the mjpeg file contents when raspimjpeg decides to rename the file.
As you said, there's no handshake between the two processes, so as long as motions decodes the jpegs faster than raspimjpeg renames them, there's no problem. Any hickup and we get gray blocks (or worse, depending on how error-proof the mjpeg decoding of jpegutils is).

Return to “Camera board”