v2 camera raspivid now captures up to 200fps ...

[HermannSW]

In thread "raspivid: fpsₘₐₓ(v₂)=120, fpsₘₐₓ(v₁)=90 / 800x760: fps(v₂)=120, fps(v₁)=90"
viewtopic.php?f=43&t=201728

I showed that fpsₘₐₓ(v₂)=120. Reason was a capping in (closed source) GPU software.
Yesterday I created new issue "Capping raspivid fps at 120 is too conservative for v2 camera #13":
https://github.com/6by9/userland/issues/13

It turned out that I raised the issue against wrong repo, and even more 6by9 had already fixed the issue.

Now you are just one "sudo rpi-update" away from being able to capture with up to 200fps with raspivid and v2 camera [v1 camera remains capped at 90fps, if you want more you can, see thread High framerate raspivid (initially 640x480 at 180fps)].

Reason for capping at 200fps is a permanent restriction of GPU not being able to go over 202fps:
https://github.com/6by9/userland/issues/14

You can do captures up to 180fps with no or only very few frame skips with mode 7.
In order to analyze frame delta distribution and frame skips you may want to use this tool:
https://github.com/Hermann-SW/userland/ ... ptsanalyze

It analyzes file created with raspivid "--save-pts" option.
In case you work with "standard" raspivid, please pass "1" as startframe:

Code: Select all

pi@raspberrypi2B:~/userland $ tools/ptsanalyze
tools/ptsanalyze file.pts startframe
pi@raspberrypi2B:~/userland $ 

Now you can do such 180fps captures with standard raspivid (after "sudo rpi-update"):
https://www.youtube.com/watch?v=T8VvjJJ ... e=youtu.be

[HermannSW]

Just did a new 180fps 640x480 slowmo of snapping fingers:
https://www.youtube.com/watch?v=I8RxHOv ... e=youtu.be

This is command and 0 frame skips analysis:

Code: Select all

pi@raspberrypi3B:~ $ raspivid -md 7 -fps 180 -pts 180.pts -w 640 -h 480 -o a.h264 -t 4000  -ex off -ss 5000
pi@raspberrypi3B:~ $ userland/tools/ptsanalyze 180.pts 1
creating tstamps.csv
692 frames were captured at 180fps
frame delta time[us] distribution
      1 5514
      1 5516
      1 5517
      1 5519
      1 5520
      1 5524
      1 5527
      9 5528
     39 5529
    244 5530
    326 5531
     46 5532
      7 5533
      4 5534
      1 5535
      1 5537
      1 5540
      1 5541
      1 5544
      1 5546
      1 5547
after skip frame indices (middle column)
0 frame skips (0‰)
pi@raspberrypi3B:~ $ 

The distance the snap finger traverses is >7cm.
But within a single frame it traverses >5cm.
The speed therefore is greater than 9m/s or 32.4km/h(!):

$ echo "0.05/(1/180)" | bc -ql
9.00000000000000000900
$ echo "0.05/(1/180)*3.6" | bc -ql
32.40000000000000003240
$

You can single frame step fore/back in youtue video after stopping with "."/"," keys.

1st frame sample:



Next frame after that:

[HermannSW]

During train ride along river Neckar today, I took several raspivid videos at 180fps. I was able to just run this because I did "sudo rpi-update" and got @6by9's fix for standard raspivid:

Code: Select all

pi@raspberrypi2B:~ $ raspivid -md 7 -w 640 -h 480 -fps 180 -t 40000 -o test.h264 
pi@raspberrypi2B:~ $

I tried to cut out part of the video with ffmpeg and found that none of the samples I found on google searches worked for raspivid generated .h264 files. What worked was "-vframes 360" to restrict the video output size to 360 frames (2s). What did not work was "-ss" option for setting start in input video.

I found a simple solution: just convert the .h264 video to .mkv without any additional option (mkvmerge -o bridge.mkv bridge.h264). Running mkvmerge gives this warning:

Code: Select all

...
Warning: 'bridge.h264' track 0: This AVC/h.264 track's timing information indicates that it uses a variable frame rate. However, no default duration nor an external timecode file has been provided for it, nor does the source container provide timecodes. The resulting timecodes may not be useful.
...

But now "ffmpeg -ss 190 -i bridge.mkv -vframes 360 -c copy out.mkv" works like a charme.
Is running mkvmerge really needed to cut parts from raspivid recorded .h264 video?

... played 180/25=7.2 times slowed down. You see pylon footbridge between Haßmersheim city and train station over river Neckar (Germany). Video was taken from driving train through window, camera hold by soldering third hand.
https://www.youtube.com/watch?v=kGrx1gD ... e=youtu.be

I converted the .mkv video to animated .gif with gifenc.sh as well, warning: size 56MB:
https://stamm-wilbrandt.de/en/forum/bridge.anim.gif

[HermannSW]

Because v4l2src and rpicamsrc were capping maximal framerate at 90fps, and I wanted to use gstreamer pipeline to add millisecond resolution timestamp overlays to the frames (nice, with 0.005s deltas), I did a video pipeing 200fps raspivid output to gstreamer pipeline and got a first good 200fps video:
viewtopic.php?f=43&t=209182&p=1293707#p1293511


Today I was looking for a "fast" gadget I can use for 200fps video and thought on spinning coin.
Because I forgot to take 3rd hand with me video is taken with camera rotated 90° (480x640).
I changed focus of the M12 monted v2 camera lens so that the area directly in front of the camera is sharp.
And then I did capture 200fps with timestamps, did send spinning coin in front of camera.
And in the video taken one traversal of the spinning coin was captured with this command:

Code: Select all

raspivid -md 7 -w 640 -h 480 -t 17000 -fps 200 -pts t.pts -o - | gst-launch-1.0 -v fdsrc ! video/x-h264,width=640,height=480,framerate=200/1 ! h264parse ! avdec_h264 ! timeoverlay time-mode="buffer-time" ! avimux ! filesink location=c2.avi

Here you can see the (fast) traversal of 50 (Euro) cent in front of the v2 camera at 1fps with timestamps.
I did rotate the video for correct view in post processing for animated .gif creation:


You can find unrotated video with few more frames on youtube, 1fps there as well:
https://www.youtube.com/watch?v=OFzb4U6 ... e=youtu.be


For me this is really nice demonstration of (30$) v2 camera 200fps capabilities.
If you have a bigger budget (100k$) you may prefer 1000fps video with Phantom HD camera
https://www.youtube.com/watch?v=cfe7BR9tOD0

[HermannSW]

It turned out to be a not so good idea to add tmeoverlay timestamps while recording at very high framerates. In one video I took at 180fps that reduced the actual framerate in generated video to 120fps.

These steps seem to be the correct for adding timeoverlay timestamps to raspivid high framerate videos:

1. record 180fps VGA video and generate timestamp file:

   Code: Select all

   $ raspivid -md 7 -w 640 -h 480 -t 8000 -fps 180 -pts t.pts -o t.h264

2. combine generated t.h264 and t.pts into Matroska video with timestamps:

   Code: Select all

   $ mkvmerge -o t.mkv --timecodes 0:t.pts t.h264

3. post process that video for (millisecond resolution) timeoverlay addition with gstreamer pipeline:

   Code: Select all

   $ gst-launch-1.0 -v filesrc location=t.mkv ! matroskademux ! h264parse ! avdec_h264 ! timeoverlay time-mode="buffer-time" ! x264enc ! matroskamux ! filesink location=tt.mkv

   (adding "videoflip method=clockwise !" in center of command allows to rotate the video as well)

The generated tt.mkv video is still the video recorded, but with millisecond exact playing of frames and seeing those.

From a video I took today from driving train, again the footbridge shown earlier in this thread. Today I did not have soldering 3rd hand and therefore had to hold the camera myself. Not easy to keep camera in the one position needed. In addition the evening sun was not helpful, shining nearly direct into camera.

These are the 4 relevant timestamps from t.pts file:

Code: Select all

...
8467.329
8472.861
8483.921
8489.453
...

I chose those because it shows the rounding at microsecond level, and a frame skip. I recorded 45s VGA video at 180fps, 7994 frames, and it contained 108 frame skips. First timestamp is 0.000, 7994th timestamp is 44858.601. That makes an average frame delay of 44858.601/7993 = 5.6ms, and an average framerate of 178fps.

Now you can see that the above 4 timestamps are exactly reflected in these 4 consecutive frames from the video, just rounded to millisecond resolution:

8467.329


8472.861


8483.921


8489.453



Yesterday I had fun with two mouse traps again.

From the 22MB animated .gif in this tweet, the two mouse traps did some fun separate flight maneuvers after triggering each other:
https://twitter.com/HermannSW/status/979104882107011072



From youtube video where I cheated and made timestamps 10 times as big before doing further post processing, the two mouse traps did capture each other and do a joint flight maneuver then:
https://www.youtube.com/watch?v=sPv3gYQ ... e=youtu.be




P.S:
I created this collection in order to demonstrate what happens over sequence of consecutive frames:



P.P.S:
See followup thread "Howto upload raspivid high framerate videos to youtube":
viewtopic.php?f=43&t=209794

[HermannSW]

If you have a bigger budget (100k$) you may prefer 1000fps video with Phantom HD camera
https://www.youtube.com/watch?v=cfe7BR9tOD0

No need for big budget, 25$ v2 camera can do 1000fps as well !
viewtopic.php?f=43&t=212518

[HermannSW]

I did a first pull request against Raspberry documentation wrt 200fps maximal framerate for v2 camera mode 7:
https://github.com/raspberrypi/documentation/pull/908

Today it got merged after being enhanced.
I don't fully understand the footnote since I was able to capture with 180fps without "--ex off".
Anyway Raspberry camera documentation does mention 200fps maximal framerate now!
https://www.raspberrypi.org/documentati ... /camera.md

[HermannSW]

We visited youth hostels for many years with our children, but in Germany there is no age restriction to visit youth hostels. Now that all three of our children are adult, we do still make short trips to youth hostels, this time youth hostel in Hochspeyer/Germany, in Palatinate forest.

We arrived yesterday and during dinner I was very impressed of the more than 3m wide and more than 1m high fish tank in the wall to next room. It was totally clear to me that this fish tank deserves high framerate video with raspivid and Raspberry v2 camera. After I got a power extension from reception this was the setup:



I used the Small and simple horizontal v2 camera with M12 mount holder built from Lego bricks with slight modification allowing to direct the camera a bit upwards (on top of reversed white table bin).

I did want to compare v2 camera 640x480 mode 7 raspivid videos taken with 60fps and 180fps. I used these commands to capture the videos:

Code: Select all

$ raspivid -md 7 -w 640 -h 480 -fps 60 -o test.60.h264
$ raspivid -md 7 -w 640 -h 480 -fps 180 -o test.180.h264 
$

This is the 1st video taken at 60pfs, played at 25fps in youtube, 2.4 times slower than real. Colors are fine:
https://www.youtube.com/watch?v=L8mQ9GHAUhs



This is the 2nd video taken at 180pfs, played at 25fps in youtube, 7.2 times slower than real. Too dark:
https://www.youtube.com/watch?v=_CeqVNyt66Y



It was no option to increase lighting of the scene (fish tank), as I would normally do for high framerate recordings ("double framerate, double lighting").Instead I used gstreamer to increase brightness of the 2nd video, and allow the upload to youtube recognize the framerate as 30fps (youtube supports 25fps, 30fps and 60fps), with this command:

Code: Select all

gst-launch-1.0 -v filesrc location=test.180.h264 ! decodebin ! videorate ! video/x-raw,framerate=30/1 ! videoconvert ! videobalance brightness=0.4 ! x264enc ! filesink location=test.180.b.h264

This is the 2nd video taken at 180pfs, with increased brightness, played at 30fps in youtube, 6 times slower than real. Colors are much better. What I learned from this is that "too dark" recording can be fixed afterwards:
https://www.youtube.com/watch?v=zQ86pVRosQY