almost_love_ffmpeg
Posts: 3
Joined: Fri Aug 05, 2022 1:07 am

ffmpeg h264_v4l2m2m "VIDIOC_STREAMON failed on output context"

Fri Aug 05, 2022 1:44 am

I have a video (which was recorded on an iphone) and a jpg image. And I have an ffmpeg command to vstack the image on top of the video with some other filters to scale and crop it all together right (not shown for simplicity). My pi 4B is running 64 bit ubuntu server 22.04 LTS and has ffmpeg 4.4 installed.
This is the command:

Code: Select all

ffmpeg -i video.mp4 -i image.jpg -c:v h264_v4l2m2m -y -filter_complex "[0:v]format=yuv420p,fps=30[v0];[1:v]format=yuv420p[v1];[v1][v0]vstack,fps=30" out.mp4
And the full error output i get is

Code: Select all

ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: isommp41mp42
    creation_time   : 2022-08-04T21:12:19.000000Z
  Duration: 00:00:02.50, start: 0.000000, bitrate: 14744 kb/s
  Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 94 kb/s (default)
    Metadata:
      creation_time   : 2022-08-04T21:12:19.000000Z
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
  Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1080x1920, 14640 kb/s, 30 fps, 30 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2022-08-04T21:12:19.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
Input #1, image2, from 'image.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 909 kb/s
  Stream #1:0: Video: mjpeg (Baseline), gray(bt470bg/unknown/unknown), 1080x152 [SAR 1:1 DAR 135:19], 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:1 (h264) -> format (graph 0)
  Stream #1:0 (mjpeg) -> format (graph 0)
  fps (graph 0) -> Stream #0:0 (h264_v4l2m2m)
  Stream #0:0 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_v4l2m2m @ 0xaaaad933e0b0] Using device /dev/video11
[h264_v4l2m2m @ 0xaaaad933e0b0] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode
[h264_v4l2m2m @ 0xaaaad933e0b0] requesting formats: output=YU12 capture=H264
[h264_v4l2m2m @ 0xaaaad933e0b0] Failed to set gop size: Invalid argument
Output #0, mp4, to 'out.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: isommp41mp42
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(progressive), 1080x2072 [SAR 1:1 DAR 135:259], q=2-31, 200 kb/s, 30 fps, 15360 tbn (default)
    Metadata:
      encoder         : Lavc58.134.100 h264_v4l2m2m
  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 69 kb/s (default)
    Metadata:
      creation_time   : 2022-08-04T21:12:19.000000Z
      handler_name    : Core Media Audio
      vendor_id       : [0][0][0][0]
      encoder         : Lavc58.134.100 aac
[h264_v4l2m2m @ 0xaaaad933e0b0] VIDIOC_STREAMON failed on output context
Video encoding failed
[aac @ 0xaaaad933f2f0] Qavg: 3216.999
[aac @ 0xaaaad933f2f0] 2 frames left in the queue on closing
Conversion failed!
This works fine on libx264 and it works fine with h264_nvenc on my main PC but i would prefer using h264_v4l2m2m. It also does not happen to all videos just some. Additionally it encodes fine if i do not vstack the image onto it. its only when vstacking the image on it does this issue occur. This is for an automated webserver and i would really prefer to make use of the 4B's hardware accelerated ffmpeg encoding.
If you want to test this yourself i have uploaded the uncompressed files here https://cdn.discordapp.com/attachments/ ... /files.zip

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 13123
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ffmpeg h264_v4l2m2m "VIDIOC_STREAMON failed on output context"

Fri Aug 05, 2022 9:31 am

Code: Select all

Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(progressive), 1080x2072 [SAR 1:1 DAR 135:259], q=2-31, 200 kb/s, 30 fps, 15360 tbn (default)
 
If that is what is being passed to h264_v4l2m2m, then it exceeds level 4.0 on the number of macroblocks permitted per frame.
https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels
Max 8192 macroblocks/frame for levels 4.0 and 4.1. 1080 x 2072 is 130 x 68 macroblocks, or 8840 macroblocks/frame. The hardware encoder will reject that inconsistency.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

almost_love_ffmpeg
Posts: 3
Joined: Fri Aug 05, 2022 1:07 am

Re: ffmpeg h264_v4l2m2m "VIDIOC_STREAMON failed on output context"

Sun Aug 07, 2022 7:15 am

i see now that makes sense. it was the caption that increased the resolution just enough that made it fail. i also tried scaling the output to less than 1080p width and it also worked. does that mean h264_v4l2m2m is only limited to level 4? could it do level 5 somehow or is that just a hard limitation? lastly how did u calculate that the resolution of 1080x2072 would have 68x130 macroblocks? how can i calculate how many macroblocks are per frame? my plan is if the macroblocks are above the threshold of 8192 (level 4's limit) then start scaling down the video resolution until it meets the required threshold.

almost_love_ffmpeg
Posts: 3
Joined: Fri Aug 05, 2022 1:07 am

Re: ffmpeg h264_v4l2m2m "VIDIOC_STREAMON failed on output context"

Sun Aug 07, 2022 7:32 am

nevermind after some research i learned a macroblock is 16x16 and i just take (w/16)*(h/16). i appreciate your insight.

Return to “Graphics, sound and multimedia”