B101 hdmi input with i2s sound

[alower]

This produces the file with perfect audio but, rather oddly, the video isn't there and instead, there's a random image from my laptop's desktop in it's place!?

I still get the same "[video4linux2,v4l2 @ 0x11a8a00] Dequeued v4l2 buffer contains 4177920 bytes, but 4147200 were expected. Flags: 0x00002001." error.

avconv (ffmpeg) is being fussy over buffer size.
Most of the graphics processing on the Pi wants the height aligned to a multiple of 16. 1080 is not a multiple of 16, therefore it is rounded up to 1088. The V4L2 format says 1920x1080x16bpp = 4147200, but with sizeimage=4177920 (1920x1088*16bpp).
It seems like avconv is checking the buffer size against what it thinks the buffer size is, and whinges if they don't match (even though it doesn't matter as long as it is "big enough")

https://github.com/FFmpeg/FFmpeg/blob/m ... 4l2.c#L541 would appear to say that it throws the buffer away under these circumstances.
GStreamer (more sensibly) only complains should the buffer be too small - https://github.com/GStreamer/gst-plugin ... or.c#L1366

(except you now appear to have deleted the post I was replying to).

Thank you. So, if it's throwing it away, would that mean I have to use something else other the ffmpeg? You mention Gstreamer, is that the alternative you suggest?

Sorry, the short post I deleted turned out to be nothing.

I'm not at the Pi now but I did see a rate of 48k when using that command, I think. I also feel like I remember seeing 'on' next to audio_present, if that sounds right. Can confirm later.

[alower]

I rebooted this evening (using the same HDMI cables etc) and ran the v4l2-ctl commands as usual.

Now, whether I use "v4l2-ctl --set-edid=file=1080P30EDID.txt --fix-edid-checksums" or "v4l2-ctl --set-edid=file=1080P3=50EDID.txt --fix-edid-checksums" I get 1280x720, where as earlier, I got 1920x1080. Nothing has changed at all.

I ran with this as 720 is a multiple of 16 (going off what you were talking about in your last post) and I can get video and audio into an output file but now I can only get 3fps

This is what I'm running, based on the information I've found from various places and below is it's output. I'm going crazy, 6by9.

Code: Select all

arecord -d 0 -r 48000 -c 2 -f S16_LE -t wav -D sysdefault:CARD=tc358743 | avconv -t 0 -i pipe:0 -f v4l2 -i /dev/video0 -r 30 -c:v libx264 -acodec aac -ar 48000 -vcodec flv -strict experimental -y -f flv test.flv

Code: Select all

Recording WAVE 'stdin' : Signed 16 bit Little Endian, Rate 48000 Hz, Stereo
ffmpeg version 3.2.10-1~deb9u1+rpt2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  configuration: --prefix=/usr --extra-version='1~deb9u1+rpt2' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx-rpi --enable-mmal --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --arch=armhf --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, wav, from 'pipe:0':
  Duration: N/A, bitrate: 1536 kb/s
    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
[video4linux2,v4l2 @ 0x172e110] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0x172e110] Time per frame unknown
Input #1, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 927.631204, bitrate: N/A
    Stream #1:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 120 tbr, 1000k tbn, 1000k tbc
Output #0, flv, to 'test.flv':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: flv1 (flv) ([2][0][0][0] / 0x0002), yuv420p, 1280x720, q=2-31, 200 kb/s, 120 fps, 1k tbn, 120 tbc
    Metadata:
      encoder         : Lavc57.64.101 flv
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.64.101 aac
Stream mapping:
  Stream #1:0 -> #0:0 (rawvideo (native) -> flv1 (flv))
  Stream #0:0 -> #0:1 (pcm_s16le (native) -> aac (native))
[wav @ 0x1728730] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
[video4linux2,v4l2 @ 0x172e110] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
overrun!!! (at least 9513.101 ms long)44kB time=00:00:00.97 bitrate=2890.9kbits/s speed=0.0987x
overrun!!! (at least 25.077 ms long) 363kB time=00:00:01.84 bitrate=1614.1kbits/s speed=0.162x
overrun!!! (at least 254.192 ms long)421kB time=00:00:02.97 bitrate=1159.1kbits/s speed=0.226x
overrun!!! (at least 93.349 ms long) 441kB time=00:00:03.86 bitrate= 935.3kbits/s speed=0.263x
overrun!!! (at least 198.942 ms long)461kB time=00:00:04.97 bitrate= 758.6kbits/s speed=0.303x
overrun!!! (at least 155.909 ms long)526kB time=00:00:06.14 bitrate= 701.0kbits/s speed=0.337x
frame=   54 fps=2.8 q=31.0 Lsize=     534kB time=00:00:06.67 bitrate= 655.6kbits/s speed=0.352x

[alower]

I can feel your excitement at the idea of dealing with me again today, so here's some more information for the whole 3fps issue I mentioned in the last reply.

If I run "avconv -t 0 -i /dev/video0 -f null /dev/null" the console shows 60fps. It's my understanding that this isn't encoding anything.

If I run any of these below (found out it doesn't have to be flv, should that help anything), I get around 3-4 fps.

- avconv -t 0 -i /dev/video0 -c:v libx264 -strict experimental -y -f flv test.flv
- avconv -t 0 -i /dev/video0 -c:v flv -strict experimental -y -f flv test.flv

This next command below shows as getting up to 45fps but the saved video still looks pretty awful and the fps must still be around 5, not 45.

- avconv -t 0 -i /dev/video0 -c:v libx264 -strict experimental -y -f h264 test.h264

Output for the libx264 flv test is:

Code: Select all

ffmpeg version 3.2.10-1~deb9u1+rpt2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  configuration: --prefix=/usr --extra-version='1~deb9u1+rpt2' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx-rpi --enable-mmal --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --arch=armhf --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[video4linux2,v4l2 @ 0x527620] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0x527620] Time per frame unknown
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 6071.912862, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 120 tbr, 1000k tbn, 1000k tbc
No pixel format specified, yuv422p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x52a870] using cpu capabilities: ARMv6 NEON
[libx264 @ 0x52a870] profile High 4:2:2, level 3.2, 4:2:2 8-bit
[libx264 @ 0x52a870] 264 - core 148 r2748 97eaef2 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
Output #0, flv, to 'newtest.flv':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv422p, 1280x720, q=-1--1, 60 fps, 1k tbn, 60 tbc
    Metadata:
      encoder         : Lavc57.64.101 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
frame=  158 fps=4.1 q=-1.0 Lsize=     878kB time=00:00:30.70 bitrate= 234.2kbits/s speed=0.803x
video:874kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.385400%
[libx264 @ 0x52a870] frame I:1     Avg QP:20.00  size: 68854
[libx264 @ 0x52a870] frame P:157   Avg QP: 8.93  size:  5261
[libx264 @ 0x52a870] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x52a870] mb P  I16..4:  1.4%  0.0%  0.0%  P16..4:  7.7%  0.0%  0.0%  0.0%  0.0%    skip:90.9%
[libx264 @ 0x52a870] coded y,uvDC,uvAC intra: 28.1% 13.8% 13.4% inter: 3.4% 2.7% 2.3%
[libx264 @ 0x52a870] i16 v,h,dc,p: 76% 22%  1%  1%
[libx264 @ 0x52a870] i8c dc,h,v,p: 88%  4%  7%  0%
[libx264 @ 0x52a870] kb/s:231.42

Output for the h264 test is:

Code: Select all

ffmpeg version 3.2.10-1~deb9u1+rpt2 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  configuration: --prefix=/usr --extra-version='1~deb9u1+rpt2' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx-rpi --enable-mmal --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --arch=armhf --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[video4linux2,v4l2 @ 0x16d4600] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
[video4linux2,v4l2 @ 0x16d4600] Time per frame unknown
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 5015.769341, bitrate: N/A
    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 59.94 tbr, 1000k tbn, 1000k tbc
No pixel format specified, yuv422p for H.264 encoding chosen.
Use -pix_fmt yuv420p for compatibility with outdated media players.
[libx264 @ 0x16d77e0] using cpu capabilities: ARMv6 NEON
[libx264 @ 0x16d77e0] profile High 4:2:2, level 3.2, 4:2:2 8-bit
Output #0, h264, to 'newtest.h264':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: h264 (libx264), yuv422p, 1280x720, q=-1--1, 59.94 fps, 59.94 tbn, 59.94 tbc
    Metadata:
      encoder         : Lavc57.64.101 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
frame= 1813 fps= 37 q=-1.0 Lsize=    3218kB time=00:00:30.24 bitrate= 871.5kbits/s dup=1700 drop=0 speed=0.617x
video:3218kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
[libx264 @ 0x16d77e0] frame I:8     Avg QP:14.00  size:170114
[libx264 @ 0x16d77e0] frame P:1805  Avg QP:15.46  size:  1071
[libx264 @ 0x16d77e0] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x16d77e0] mb P  I16..4:  0.4%  0.0%  0.0%  P16..4:  2.0%  0.0%  0.0%  0.0%  0.0%    skip:97.6%
[libx264 @ 0x16d77e0] coded y,uvDC,uvAC intra: 43.4% 28.3% 19.2% inter: 1.1% 0.8% 0.2%
[libx264 @ 0x16d77e0] i16 v,h,dc,p: 61% 26%  8%  5%
[libx264 @ 0x16d77e0] i8c dc,h,v,p: 68% 12% 17%  4%
[libx264 @ 0x16d77e0] kb/s:871.45

[6by9]

libx264 is a software video encoder. There's very little chance that will ever keep up at anything more than standard def.

avconv and ffmpeg have a funny history, but assuming it is actually using ffmpeg that you're using then you may be able to use h264_omx or h264_v4l2m2m to use the hardware encoder.
h264_omx may have issues as I don't believe it supports the UYVY format that the B101 produces (a failing of ffmpeg, not the IL implementation). h264_v4l2m2m should accept UYVY, but I had to patch FFmpeg last time I tried it.

[alower]

Thanks again, I had just managed to find me way to hardware encoding just before your reply. I (we) (you) have basically done it! Not using hardware encoder, though. I'd really like to get that working if I could as doing other tasks gets a bit tough with software encoder.

So, good news. Got myself a Pi 4 1GB. Smells slightly like it's burning, so any thoughts on that would be appreciated and it's now running Buster.

Had trouble getting avconv, for some reason, so I'm using ffmpeg.

Code: Select all

arecord -d 0 -r 48000 -c 2 -f S16_LE -t wav -D sysdefault:CARD=tc358743 | 
ffmpeg -f rawvideo -vcodec rawvideo -re -pix_fmt uyvy422 -s:v 1280x720 -i /dev/video0 -i pipe:0 -c:v libx264 -crf 23 -f flv -preset ultrafast -qp 0 -c:a aac test.flv

Here's the video from that command: https://drive.google.com/open?id=1w0vgI ... d58ED16I5P

After an uncomfortable amount of trial and error, this works great. You can see that 'rawvideo' is being used for the input format and codec, that seemed to do a good amount of the work for me. The CPU is around 50% whilst it's doing its thing.

Now, when I run the below, I get this: https://drive.google.com/open?id=17fdEC ... fBLAI3ggxO

Code: Select all

arecord -d 0 -r 48000 -c 2 -f S16_LE -t wav -D sysdefault:CARD=tc358743 | 
ffmpeg -f rawvideo -vcodec rawvideo -re -pix_fmt uyvy422 -s:v 1280x720 -i /dev/video0 -i pipe:0 -c:v h264_omx -crf 23 -f flv -preset ultrafast -qp 0 -c:a aac 6by9_hw.flv

You can see the hw encoded version is the same, just worse quality. CPU was about 14% (I have my GPU set to 256MB if that matters).

Oh and I did try h264_v4l2m2m: https://drive.google.com/open?id=1ZBxiw ... VUSgkdsJcz

Any help, much appreciated as always.

[alower]

Interesting thing is now happening.

I plugged my monitor in to my laptop, via the HDMI splitter that is also feeding the video off to the Pi. the v4l2-ctl query now picks up 1080, whereas it was 720 before.

Now my commands don't do anything. Fps stats at 0.0, my output file is 0kb. I changed the screensize in the commands to 1920x1080.

Is this the whole 1088 issue, do you think?

[alower]

Now it gets even weirder. Running the same queries as above now doesn't work. I get 2fps and a 0kb file.

[panayiotis95]

how to get the sound to work on raspberry pi 4
arecord produces nothing

[idcidc]

Hi, All!
I've successfully done all THESE:

steps on my PI3B+and confirm, that IT WORKS!:)
Namely it looks like it works fine, BUT I observe strange behaviour of I2S output and NO SOUND:
1. Video from my laptop's hdmi output displays fine e.g. 720p50.
2. Sound device via HDMI is present in volume control, sound protperties etc. of my laptop as Toshiba-H2C device.
3.When trying to change rate (48000,44100,32000) via MS Windows control panel I see that

Code: Select all

v4l2-ctl -l

shows me correct rate that TC358743 chip sees on it's input.
The problem is that on Oversample Frequency output I see strange value 1.652MHz and about 6.5kHz on WFS (left/right selection signal, which should be exactly equal to rate e.g.48kHz) Waveform and frequency are controlled by oscilloscope and clear square pulses are present.
Also no data on serial data output of I2S is present.
Any ideas whats went wrong?

[idcidc]

It looks I've found an answer and can reply to myself

viewtopic.php?f=38&t=279108&p=1690386&h ... i#p1690386

viewtopic.php?f=38&t=278929&p=1689342#p1689342

In brief: 6by9 reports that

And I'll add that my investigations of these Chinese versions of the boards say that I2S audio does not work on them.
It does work fine on the Auvidea B10x boards, so there's obviously something wrong in the board design rather than the software.