Real Time Transcoding of DVB TV Streams

[gkreidl]

This has been treated here in different threads already (more problems than success usually). I've successfully implemented real time transcoding of TV (mpeg TS over http) streams in my SAT-TV-Server application (German version only right now). I'm using (a slightly modified version of) Sebastian Droege's http-launch with matching gstreamer1-0 toolchains.

Meanwhile I can transcode all types of TV streams. Here's a snapshot of a 720p50 video stream, transcoded to 768x432p50 with a video bitrate of 640 KBit (and audio also transcoded to 32 KBit).



I have been asked by a number of people if I could make the transcoding software available as a separate software package, usable with all kinds of TV server backends. The result is my transcoding software rtranscode, which has reached version 4.0 by now.

The next posting gives you access to my rtranscode package V. 4.0

[gkreidl]

Edit April, 3rd, 2018: New version rtranscode V. 4.0 TV package available

Edit April, 8th, 2018: I've uploaded a new package (same name) containing a bug fix. If you have installed rtranscode V 4 before, just download and install it again.

Installation (from a terminal):

Code: Select all

wget http://steinerdatenbank.de/software/rtranscode4.tar.gz
tar -xzf rtranscode4
cd rtranscode4
sudo ./install

This will install a few binaries and scripts in/usr/local/bin and create the folder /usr/local/share/rtranscode containing a configuration file and and a default (empty) channel database. An existing channel database (from rtranscode 3) will not be overwritten and you can continue to use it, although the database format has slightly changed.

The rtranscode4 directory contains the new and extended (75 pages) manual as PDF file, which is also available online. It contains a comprehensive overview, a very detailed program reference and a few tutorials. Read it, please, before starting to ask questions here.

New or extended features of rtranscode:

• Works on both Raspbian Jessie and Stretch
  Extended range of video modes
  Deinterlacing using different methods, including doubling the frame rate for SD streams
  Support for different scalers
  Support for multiple audio streams in the channel database
  Support for unmodified streams (repackaging of original video and one audio stream)
  Support for user scripts for backend tuning control
  Lots of small improvements

Additional tools:

rtranscode web interface
A standalone web server, written in Python, which contains its own rtranscode module. It supports user management for different kinds of users. You have to create a channel database, befor you can use it.

kweb GUI
a script which creates a local GUI for rtranscode, running in kweb.

Server backends using either GnuTV or MuMUDVB
A number of scripts which create all the configuration files, a kweb GUI, a rtranscode channel database and a user script for rtranscode for both backends. Automatic tuning is supported by both rtranscode and the rtranscode web interface. The web interface also supports manual tuning and gives access to the original streams. You will find a detailed description in the tutorials part of the manual.

Additional (optional) packages supplied with this release:

gst-omx 1.10.5 for Stretch
You can use this instead of the 1.10.4 version supplied by the Foundation.

Latest MuMuDVB release (mumudvb_2.1.0-6-6ae3baa)

The sources of rtranscode, rtc4.so, http-launch and my changes to the gst-omx 1.10.5 module can be found in the "sources" directory.

Jessie vs. Stretch

The omx-gst 1.0.0.1 package developped for Wheezy and continued in Jessie contained a large number of patches or additions which have never found their way into upstream development and are not available in the current 1.10.4 (or 1.10.5) release. The most important difference is the use of the GPU for scaling the video image size. As a result rtranscode will perform much better on Jessie, especially in the following cases:
Scaling of 720p progressive streams.
1080i streams require no deinterlacing, if the output size is below 540p.
And the scaling quality is better than software scaling, which has to be used on Stretch.

Caveats

Although I have been using it (different versions) for a number of years and quite a lot of users have been successfully using it with different backends, I still consider rtranscode to be experimental software for a number of reasons:

rtranscode depends on a large umber of software packages (especially from the gstreamer family) for which I'm not responsible in any way. These packages may (and do) contain bugs, have memory leaks or might simply not work correctly in some circumstances. That's outside of my control.

I have no chance to test rtranscode with all possible kinds of server backends (hardware and/or software) and cannot guarantee that it will work in every combination.

[gkreidl]

Originally this post contained the whole README file of the transcoder package, but in version 4 there is no README file any more, as it contains a full 75 pages manual in PDF format.

After installing rtranscode, you can run
rtranscode -h (or --help)
to get a list of possible usages and command line options:

Usage: rtranscode [options] uri videomode audiomode:audiopid
uri = URL of your local TV http or udp server stream (MPEG TS only)
videomode must be one of the following:
sd1 for MPEG 720x576i SD video
sd2 for H264 720x576i SD video
sd3 for MPEG 720x480i SD video
sd4 for H264 720x480i SD video
sd5 for MPEG 704x576i SD video
sd6 for MPEG 544x576i SD video
sd7 for MPEG 480x576i SD video
sd8 for MPEG 352x576i SD video
hd1 for H264 1280x720p HD video
hd2 for H264 1920x1080i HD video
hd3 for MPEG 1280x720p HD video
hd4 for MPEG 1920x1080i HD video
hd5 for H264 1440x1080i HD video
hd6 for H264 1280x1080i HD video
audiomode must be either 'mpeg', 'ac3' or 'aac'
audiopid must match your TS stream (decimal or hex starting with 0x)

Alternate methods:
rtranscode [options] -f=file_path
read data from file
rtranscode [options] -n=channel_name
select a channel by name from channels file
rtranscode [options] -g=URI
analyze stream URI and return name and arguments
rtranscode [options] -t=URI
analyze stream URI and start transcoding, if possible
rtranscode [options] -i=infile -o=outfile
analyse URIs from infile and append to channels file outfile (created, if missing)
rtranscode [options]
If called without further argments, transcode will try to read
from a channels file and offer a simple selection menu.

Options:
-c=config file
(default = /usr/local/share/rtranscode/rt_config.py)
-d=channels file
(default = /usr/local/share/rtranscode/channels.dat)
-a=Audio bitrate number (0..12), default = 0 (original)
-v=video bitrate number (0..24), default = 2 (medium)
-s=SD video size number (0..7), default = 7 (576p)
-h=HD video size number (0..8), default = 4 (512p)
-e=deinterlace (0..7), default = 0 (off)
-x=videoscaler quality (0..2), default = 0 (OMX)
-path=path output path for stream URL, must start with a "/" and end with ".mkv", default = "/xyz.mkv"
-port=port output port for stream URL, must be greater than 1024, default = "9080"
-u=audio_output_format must be either "aac", "ac3" or "both", default = both
-r=delay, start the original stream before running the transcoder, (0..10), default = 0
-l=record_path, full path to recording directory, default = current directory
-z=user_script_path, full path to a user script for tuning commands, default: /usr/local/share/rtranscode/tuner.py
Options without arguments:
-sm print the http-launch command line with all arguments instead of starting the transcoder
-q run both rtranscode and http-launch in silent mode, prevents menu mode from working!
-h or --help, show usage

Once you have created a channel database, you can run rtranscode in interactive mode, using a curses menu:


rtranscode can also be controlled using a GUI running in kweb:


The rtranscode web server gives you access to transcoding using any browser on your network or even from the internet:


It offers a number of configuration options:


Less privileged user will see a much simpler web page:




The rtranscode V. 4 package contains a number of tools to create your own TV server backend, using either GnuTV or MuMuDVB.

GnuTV GUI, running in kweb:


MuMuDVB GUI, running in kweb, serving complete transponders:


For both backends, tuning and access to the original streams is also supported by the rtranscode web server.

For more details, check the PDF manual, which is also available online.

[mishu77]

Hi,

Do you think is possible to add flvmux and rtmp output to http-launch?

Thank you.

[gkreidl]

Hi,

Do you think is possible to add flvmux and rtmp output to http-launch?

Thank you.

rtmp: no, not without rewriting it completely.

A gstreamer tool chain which replaces matroskamux by flvmux might work, but the problem is the audio transcoding: flvmux does not support aac and there is no mpeg audio encoder in gstreamer1.0.

[c-o-m-m-a-n-d-e-r]

wow cool, i think its exactly what i was looking for ... i will test it with tvheadend as server backend

Thanks!

[gkreidl]

wow cool, i think its exactly what i was looking for ... i will test it with tvheadend as server backend

Thanks!

tvheadend uses it's own streaming protocol. You must make sure that it streams pure TS streams.

[c-o-m-m-a-n-d-e-r]

yep, i know ... i use htsp in combination with Kodi ...
For Streaming / Transcoding i use http mpeg-ts output (pass) ...

Thanks / Danke

[c-o-m-m-a-n-d-e-r]

Just a short Information : After some mailing with gkreidl i found the trick.

HD transcoding needs a Rpi2 ... Rpi1 isn't working ...

I test with "Das Erste HD" with transcoding to 1280x720, 1,5 MBit/s and 128 Kbit/s Audio ...
Rpi2 is round about at 85% (with gpu_freq @ 500)

Big Thanks!

[stderr]

I've created a small transcoder package for real time transcoding of

I'd be interested in knowing if anyone has transcoding or viewing of full HD from the antenna of North American digital TV working. Specifically I'm talking about the HDHomerun. I've heard some folks make some claims about doing this off cable but the antenna is supposed to be more demanding.

[Bastinazo]

I have installed your transcoder package tool manually cause I use raspbian wheezy for transcoding. However "http-launch" shows me this error


"** (http-launch:3193): WARNING **: Cannot autolaunch D-Bus without X11 $DISPLAY"

Is there any other option for launching transcoding.py in raspbian wheezy ?


Thxs in advance !!

[gkreidl]

I have installed your transcoder package tool manually cause I use raspbian wheezy for transcoding. However "http-launch" shows me this error


"** (http-launch:3193): WARNING **: Cannot autolaunch D-Bus without X11 $DISPLAY"

Is there any other option for launching transcoding.py in raspbian wheezy ?


Thxs in advance !!

The message comes from gstreamer. I cannot tell you if will run completely headless, although I have often used it from an SSH terminal (but X-Server was running on the Pi.)
I've done transcoding on Wheezy, but H264 sources did not work because of a bug in gstreamer1.2 which has been fixed in Jessie (version 1.4 now). I could transcode MPEG streams (you need the mpeg codecs).
The bug in the gstreamer-omx Module of Jessie (memory leak) has been fixed recently.

[henkedk]

Seems to be some nice stuff, but how would one do it if the purpose is to be to actually get the stream back to a PVR client remotely, which is also able to change channels, see EPG etc just as if it connected directly to TvHeadend?

What I'm trying to do is using a RPI2 as a TV-server (USB DVB-C) (jessie without any GUI) and use a OpenElec/Kodi remote to get access to the channels I got at home. Whilst SD channels work fine, HD don't, they stutter. I tried compiling TvHeadend with transcoding and adding the MPEG2 license, but the RPI2 doesn't seem to have the power to user the transcoding (no matter which version I choose), since all I get is stutter on both HD and SD.

[1kane]

Hay gkreidl,
im using succesfully ur transcoding script from

http://steinerdatenbank.de/software/transcoder.tar.gz

but on interlaced channels i get really ugly pictures that look like bad downscaled interlace. bc that final interlace video isnt "real interlace" (like odd lines are double the size) im not able to deinterlace it at the reciving device.

i saw that deinterlacing is possible with gstramer, but im not able to get it working (tried to place the "! deinterlace" pipe at every known position in the config file). do you know how to modify the settings to get the deinterlace working on the PI?
that would be great

[gkreidl]

Hay gkreidl,
im using succesfully ur transcoding script from

http://steinerdatenbank.de/software/transcoder.tar.gz

but on interlaced channels i get really ugly pictures that look like bad downscaled interlace. bc that final interlace video isnt "real interlace" (like odd lines are double the size) im not able to deinterlace it at the reciving device.

i saw that deinterlacing is possible with gstramer, but im not able to get it working (tried to place the "! deinterlace" pipe at every known position in the config file). do you know how to modify the settings to get the deinterlace working on the PI?
that would be great

Deinterlacing is really a problem I have not solved yet. If the image size is larger than half of the original size it's getting ugly. Software deinterlacing with gstreamer didn't work either (and I'm not sure if that wouldn't be too slow). Unfortunately the omx module doesn't support any interlace function on the GPU.

[1kane]

thanks for your fast reply.
thats very sad, because the rest is working very well.
so my resolution will be to set the size of interlaced material at 288p, than it looks proper. and if the stream is progessiv than i gonna set higher resolution.

and thank you again for your transcoding script! its the onliest raspberry hardwareVideo transcoder (that streams to network) i got working!

[gkreidl]

thanks for your fast reply.
thats very sad, because the rest is working very well.
so my resolution will be to set the size of interlaced material at 288p, than it looks proper. and if the stream is progessiv than i gonna set higher resolution.

and thank you again for your transcoding script! its the onliest raspberry hardwareVideo transcoder (that streams to network) i got working!

For SD channels I use 360x288, for 1080i channels 768*432 (or 910*512). 720p50 channels can be send with any resolution.

[kylemassey86]

This may seem stupid, but I've gotten all the way through configuring and I'm getting the "Listening on http://... message, but how do I actually watch the transcoded stream??

[gkreidl]

This may seem stupid, but I've gotten all the way through configuring and I'm getting the "Listening on http://... message, but how do I actually watch the transcoded stream??

On the Pi:
omxplayer http....
on other computers, use VLC and enter the network stream address

[spyer]

Hello gkreidl,

what does that mean if I try to run transcode.py?

Code: Select all

pi@raspberrypi:~ $ transcode.py
dbox (0)
Enter a channel number or 'q' to quit: 0
http-launch: error while loading shared libraries: libgstreamer-1.0.so.0: cannot open shared object file: No such file or directory

my steps:

Code: Select all

wget http://steinerdatenbank.de/software/transcoder.tar.gz
tar -xzf transcoder.tar.gz
cd transcoder
sudo ./install
sudo ./install_omx

edit/add:

Code: Select all

sudo nano /usr/local/share/transcode/channels.dat
--->>add dbox=http://spyer:[email protected]:9981/stream/service/3cf5c667af85ef993947b41ce897758b? sd1 mpeg 2652
-->>source: TVHeadend from dbox2
profil: pass, Mpeg-TS Pass.trough

VLC-tvheadend sad under codec: 
Stream 0
Typ: Video
Originale ID: 2651
Codec: MPEG-1/2 Video (mpgv)
Auflösung: 720x576
Bildschirmauflösung: 720x576
Bildwiederholrate: 25
Decodiertes Format: Planar 4:2:0 YUV

Stream 1
Typ: Audio
Originale ID: 2652 <---[b]AudioPID???[/b]
Codec: MPEG Audio layer 1/2 (mpga)
Sprache: Deutsch
Kanäle: Stereo
Abtastrate: 48000 Hz
Bitrate: 192 kB/s

what I am doing wrong or what is missing?

Best wishes
spyer

PS: I ordered the mpeg-codec activator and wait for the activation to install this in confog.txt. Is this the reason for "No such file or directory" ?

[gkreidl]

1) For MPEG transcoding you need the MPEG codec license. (used by the omx decoder)
2) Check if all gstreamer-1.0 libtraries are isntalled. My preferred way is using synaptic and searching for gstreamer.

[spyer]

Good Morning,

do you have any idea how I can install it without synaptic?

edit: i try know:
sudo apt-get install gstreamer1.0*

[gkreidl]

Good Morning,

do you have any idea how I can install it without synaptic?

edit: i try know:
sudo apt-get install gstreamer1.0*

gstreamer1.0 has quite a lot of packages- Some of them are installed by default (to support epiphany browser), but not all.

If you don't want to install synaptic, you can also use PiPackages. Search for gstreamer1.0-plugins.

[spyer]

hello,

the transcoding works, but with freezes after few minutes.
Unfortunately I can not setup higher bitrate, keeping the transcoding power.

What cann I do, to make it usable?

Spyer

[gkreidl]

hello,

the transcoding works, but with freezes after few minutes.
Unfortunately I can not setup higher bitrate, keeping the transcoding power.

What cann I do, to make it usable?

Spyer

I need more details what you are trying to do.