RTMP Stream using GStreamer and a B101

[cyphercolt]

Following a lot of helpful advice from the HDMI to CSI-2 via TC358743 on kernel 4.1 thread, I've managed to get Video and Audio from any HDMI out device to my Pi.
Big thanks to 6by9 for helping me get this project this far!

I'm trying to set up my Pi to Stream to Twitch.tv without having to buy a second PC with a capture card just to stream. Especially when the Pi setup with the B101 and an SD Card is around the same price as a good capture card.

This is my current pipeline, that is still giving me some buffering issues when I stream to RTMP, but is pretty good when I stream to a file.

1. gst-launch-1.0 flvmux streamable=true name=mux
2. ! rtmpsink location="rtmp://live.justin.tv/app/STREAMKEY" v4l2src io-mode=4
3. ! "video/x-raw,framerate=60/1,format=UYVY"
4. ! v4l2h264enc output-io-mode=5 extra-controls="controls,video_bitrate=2000;"
5. ! video/x-h264,profile=high
6. ! h264parse
7. ! queue
8. ! mux. alsasrc device=hw:1
9. ! audio/x-raw,rate=48000,channels=2
10. ! audioconvert
11. ! avenc_aac bitrate=128000
12. ! aacparse
13. ! queue ! mux.

At the moment, whenever I start the stream and don't do anything, it's fine. As soon as pixels start to change I get this message which appears as choppyness and buffering on Twitch:
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough
Additional debug info:
gstaudiobasesrc.c(849): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 106560 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.


From testing for a few hours, I noticed that Twitch is sending me the video at over 20000 kbps even though I set the bitrate down to 2000. I am positive this is where my issue is coming from. - I noticed no matter what I do in the extra-controls section for v4l2h264enc, nothing changes at all. It's as if it's ignoring everything in there.

Extra Info:
Hardware:
Pi 3B - Kernel 4.19
Auvidea B101 rev4

GStreamer:
gst-launch-1.0 --version
gst-launch-1.0 version 1.14.4
GStreamer 1.14.4
Unknown package origin

[cyphercolt]

I had a thought this morning - Is it possible to cap the Bitrate on the input itself? Potentially through the EDID?

I'll look my EDID editor when I get home anyway to see if there's that option (Using the AW EDID Editor, nice UI tool)
Then I wouldn't need to convert.

[mextril]

Did you found solution to control the video bit rate?

[6by9]

Following mextril's other thread, there is a bug fix for setting the encode bitrate on v4l2h264enc as https://github.com/raspberrypi/linux/pull/2961
It should hopefully be merged in the next couple of days.

[cyphercolt]

Following mextril's other thread, there is a bug fix for setting the encode bitrate on v4l2h264enc as https://github.com/raspberrypi/linux/pull/2961
It should hopefully be merged in the next couple of days.

Oh wow I had put this project on hold for a while, just came back to check in and saw this. I'll look into it and follow up if it helped!

Thanks!

[alower]

Following a lot of helpful advice from the HDMI to CSI-2 via TC358743 on kernel 4.1 thread, I've managed to get Video and Audio from any HDMI out device to my Pi.
Big thanks to 6by9 for helping me get this project this far!

I'm trying to set up my Pi to Stream to Twitch.tv without having to buy a second PC with a capture card just to stream. Especially when the Pi setup with the B101 and an SD Card is around the same price as a good capture card.

This is my current pipeline, that is still giving me some buffering issues when I stream to RTMP, but is pretty good when I stream to a file.

1. gst-launch-1.0 flvmux streamable=true name=mux
2. ! rtmpsink location="rtmp://live.justin.tv/app/STREAMKEY" v4l2src io-mode=4
3. ! "video/x-raw,framerate=60/1,format=UYVY"
4. ! v4l2h264enc output-io-mode=5 extra-controls="controls,video_bitrate=2000;"
5. ! video/x-h264,profile=high
6. ! h264parse
7. ! queue
8. ! mux. alsasrc device=hw:1
9. ! audio/x-raw,rate=48000,channels=2
10. ! audioconvert
11. ! avenc_aac bitrate=128000
12. ! aacparse
13. ! queue ! mux.

At the moment, whenever I start the stream and don't do anything, it's fine. As soon as pixels start to change I get this message which appears as choppyness and buffering on Twitch:
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough
Additional debug info:
gstaudiobasesrc.c(849): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 106560 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.


From testing for a few hours, I noticed that Twitch is sending me the video at over 20000 kbps even though I set the bitrate down to 2000. I am positive this is where my issue is coming from. - I noticed no matter what I do in the extra-controls section for v4l2h264enc, nothing changes at all. It's as if it's ignoring everything in there.

Extra Info:
Hardware:
Pi 3B - Kernel 4.19
Auvidea B101 rev4

GStreamer:
gst-launch-1.0 --version
gst-launch-1.0 version 1.14.4
GStreamer 1.14.4
Unknown package origin

Hey cyphercolt, how did you get audio to work? I'm trying to put together the same thing as you. Got a basic node app sending video to Twitch but no audio.

I'm not familiar at all with Gstreamer and I'm not able to follow the post above. How does this pipeline that you mention work? Any help would be greatly appreciated.

Cheers

[NKnuelle]

Hey everyone,

I am trying to figure out how to get audio working with RTMP, too.

Currently I am using this pipeline which is very similar:

Code: Select all

gst-launch-1.0 flvmux streamable=true name=mux 
! rtmpsink location="rtmp://0.0.0.0:1935/live" v4l2src io-mode=4 
! "video/x-raw,framerate=50/1,format=UYVY" 
! v4l2h264enc output-io-mode=5  extra-controls="controls,video_bitrate=10M;" 
! video/x-h264,profile=high 
! h264parse 
! queue 
! mux. alsasrc device=hw:1 
! audio/x-raw,rate=48000,channels=2
! audioconvert 
! voaacenc bitrate=48000 ! aacparse 
! queue
! mux.

The RTMP stream is send to nginx running on the raspberry pi. To test it I view it inside vlc player over the network. If I access the stream there is only buffering but no audio or video.

If I just stream video everything is working fine (without audio).

Any thoughts what could be wrong here?

[NKnuelle]

Hi again,

I have a big issue using GStreamer-1.0 with this command:
The RTMP stream stops after some minutes (about 10 minutes, not always the same). I don't get any error messages using this comnmand:

Code: Select all

GST_DEBUG=2 gst-launch-1.0 flvmux streamable=true name=mux ! rtmpsink location="rtmp://0.0.0.0/live/stream" v4l2src io-mode=4 ! "video/x-raw,framerate=50/1,format=UYVY" ! v4l2h264enc output-io-mode=5 extra-controls="controls,video_bitrate=6M;" ! video/x-h264,profile=high ! h264parse ! queue ! mux.

The stream just suddenly stops and I mentioned that the cpu usage went down from ~10% to ~0%. If I stop the gst-launch-1.0 command using ctrl-c and start it again the RTMP stream does not start again. I need to restart the raspberry to get it working again.

Some debugging information from dmesg:

Code: Select all

[  +0,004580] ------------[ cut here ]------------
[  +0,000050] WARNING: CPU: 1 PID: 1575 at drivers/media/common/videobuf2/videobuf2-core.c:1688 __vb2_queue_cancel+0x158
[  +0,000010] Modules linked in: bnep hci_uart btbcm serdev bluetooth ecdh_generic 8021q garp stp llc tc358743 brcmfmac
[  +0,000164] CPU: 1 PID: 1575 Comm: v4l2h264enc0:sr Tainted: G        WC        4.19.97-v7l+ #1294
[  +0,000008] Hardware name: BCM2835
[  +0,000033] [<c0212e04>] (unwind_backtrace) from [<c020d5e0>] (show_stack+0x20/0x24)
[  +0,000020] [<c020d5e0>] (show_stack) from [<c09b15c8>] (dump_stack+0xe0/0x124)
[  +0,000021] [<c09b15c8>] (dump_stack) from [<c0222544>] (__warn+0x104/0x11c)
[  +0,000017] [<c0222544>] (__warn) from [<c0222694>] (warn_slowpath_null+0x50/0x58)
[  +0,000034] [<c0222694>] (warn_slowpath_null) from [<bf224104>] (__vb2_queue_cancel+0x158/0x1c8 [videobuf2_common])
[  +0,000052] [<bf224104>] (__vb2_queue_cancel [videobuf2_common]) from [<bf224bdc>] (vb2_core_queue_release+0x28/0x48 [
[  +0,000042] [<bf224bdc>] (vb2_core_queue_release [videobuf2_common]) from [<bf2465d8>] (vb2_queue_release+0x18/0x1c [v
[  +0,000039] [<bf2465d8>] (vb2_queue_release [videobuf2_v4l2]) from [<bf3065a8>] (v4l2_m2m_ctx_release+0x24/0x38 [v4l2_
[  +0,000038] [<bf3065a8>] (v4l2_m2m_ctx_release [v4l2_mem2mem]) from [<bf346d9c>] (bcm2835_codec_release+0x64/0xcc [bcm
[  +0,000087] [<bf346d9c>] (bcm2835_codec_release [bcm2835_codec]) from [<bf1735b8>] (v4l2_release+0x48/0x8c [videodev])
[  +0,000080] [<bf1735b8>] (v4l2_release [videodev]) from [<c03d66fc>] (__fput+0xa4/0x1e8)
[  +0,000023] [<c03d66fc>] (__fput) from [<c03d68b0>] (____fput+0x18/0x1c)
[  +0,000020] [<c03d68b0>] (____fput) from [<c02426dc>] (task_work_run+0xbc/0xe0)
[  +0,000019] [<c02426dc>] (task_work_run) from [<c0226bd0>] (do_exit+0x44c/0xc18)
[  +0,000017] [<c0226bd0>] (do_exit) from [<c0227434>] (do_group_exit+0x4c/0xe4)
[  +0,000021] [<c0227434>] (do_group_exit) from [<c0233780>] (get_signal+0x1a4/0x8cc)
[  +0,000016] [<c0233780>] (get_signal) from [<c020c73c>] (do_signal+0x84/0x474)
[  +0,000015] [<c020c73c>] (do_signal) from [<c020cd38>] (do_work_pending+0xdc/0xf4)
[  +0,000014] [<c020cd38>] (do_work_pending) from [<c0201034>] (slow_work_pending+0xc/0x20)
[  +0,000009] Exception stack(0xd0a47fb0 to 0xd0a47ff8)
[  +0,000012] 7fa0:                                     b2600e38 00000002 00000000 00000000
[  +0,000013] 7fc0: 00000008 00026ee8 00000000 00000150 2a4ae5ff 00000000 b6f31660 00026ee8
[  +0,000011] 7fe0: 00000002 b2ffe980 00000000 b6beede0 80000010 b2600e38
[  +0,000010] ---[ end trace c09d1f5fe6a145c0 ]---
[  +0,000014] videobuf2_common: driver bug: stop_streaming operation is leaving buf 47a900c0 in active state
[  +0,000014] videobuf2_common: driver bug: stop_streaming operation is leaving buf 45e567d7 in active state
[  +0,000013] videobuf2_common: driver bug: stop_streaming operation is leaving buf 19a81e76 in active state
[  +0,000012] videobuf2_common: driver bug: stop_streaming operation is leaving buf 48b508a7 in active state
[  +0,002475] ------------[ cut here ]------------
[  +0,000036] WARNING: CPU: 1 PID: 1575 at drivers/media/common/videobuf2/videobuf2-core.c:1688 __vb2_queue_cancel+0x158
[  +0,000009] Modules linked in: bnep hci_uart btbcm serdev bluetooth ecdh_generic 8021q garp stp llc tc358743 brcmfmac
[  +0,000150] CPU: 1 PID: 1575 Comm: v4l2h264enc0:sr Tainted: G        WC        4.19.97-v7l+ #1294
[  +0,000008] Hardware name: BCM2835
[  +0,000021] [<c0212e04>] (unwind_backtrace) from [<c020d5e0>] (show_stack+0x20/0x24)
[  +0,000021] [<c020d5e0>] (show_stack) from [<c09b15c8>] (dump_stack+0xe0/0x124)
[  +0,000017] [<c09b15c8>] (dump_stack) from [<c0222544>] (__warn+0x104/0x11c)
[  +0,000016] [<c0222544>] (__warn) from [<c0222694>] (warn_slowpath_null+0x50/0x58)
[  +0,000030] [<c0222694>] (warn_slowpath_null) from [<bf224104>] (__vb2_queue_cancel+0x158/0x1c8 [videobuf2_common])
[  +0,000045] [<bf224104>] (__vb2_queue_cancel [videobuf2_common]) from [<bf224bdc>] (vb2_core_queue_release+0x28/0x48 [
[  +0,000039] [<bf224bdc>] (vb2_core_queue_release [videobuf2_common]) from [<bf2465d8>] (vb2_queue_release+0x18/0x1c [v
[  +0,000035] [<bf2465d8>] (vb2_queue_release [videobuf2_v4l2]) from [<bf3065b0>] (v4l2_m2m_ctx_release+0x2c/0x38 [v4l2_
[  +0,000035] [<bf3065b0>] (v4l2_m2m_ctx_release [v4l2_mem2mem]) from [<bf346d9c>] (bcm2835_codec_release+0x64/0xcc [bcm
[  +0,000085] [<bf346d9c>] (bcm2835_codec_release [bcm2835_codec]) from [<bf1735b8>] (v4l2_release+0x48/0x8c [videodev])
[  +0,000075] [<bf1735b8>] (v4l2_release [videodev]) from [<c03d66fc>] (__fput+0xa4/0x1e8)
[  +0,000020] [<c03d66fc>] (__fput) from [<c03d68b0>] (____fput+0x18/0x1c)
[  +0,000019] [<c03d68b0>] (____fput) from [<c02426dc>] (task_work_run+0xbc/0xe0)
[  +0,000018] [<c02426dc>] (task_work_run) from [<c0226bd0>] (do_exit+0x44c/0xc18)
[  +0,000017] [<c0226bd0>] (do_exit) from [<c0227434>] (do_group_exit+0x4c/0xe4)
[  +0,000017] [<c0227434>] (do_group_exit) from [<c0233780>] (get_signal+0x1a4/0x8cc)
[  +0,000015] [<c0233780>] (get_signal) from [<c020c73c>] (do_signal+0x84/0x474)
[  +0,000013] [<c020c73c>] (do_signal) from [<c020cd38>] (do_work_pending+0xdc/0xf4)
[  +0,000014] [<c020cd38>] (do_work_pending) from [<c0201034>] (slow_work_pending+0xc/0x20)
[  +0,000010] Exception stack(0xd0a47fb0 to 0xd0a47ff8)
[  +0,000011] 7fa0:                                     b2600e38 00000002 00000000 00000000
[  +0,000012] 7fc0: 00000008 00026ee8 00000000 00000150 2a4ae5ff 00000000 b6f31660 00026ee8
[  +0,000011] 7fe0: 00000002 b2ffe980 00000000 b6beede0 80000010 b2600e38
[  +0,000010] ---[ end trace c09d1f5fe6a145c1 ]---
[  +0,000011] videobuf2_common: driver bug: stop_streaming operation is leaving buf f12221af in active state

Did anybody experience these problems, too?