US ATSC TV Partial Success

[cmason]

I've had some success playing OTA HDTV (over the air) here in the States on my Raspberry PI. I'm in Portland, OR. I wanted to share some tips for folks because it seems like most people doing this stuff are in the UK or Europe (see how I made that distinction and I think perhaps the DVB hardware is less well supported for US stuff.

The hardware is an Elgato EyeTV hybrid. Mine is a few years old and doesn't look like the one pictured, so the hardware may be different (although I kind of doubt it). It must be plugged into a powered USB hub. The hardware seems to really not like being hot plugged (in fact I'm pretty sure that unplugging or replugging it restarts the PI). I always make sure it's plugged in (to the powered hub) before boot. I'm using raspbian from 18 Sep 2012 (with some updates). This is a 512MB pi (although I doubt it matters).

lsusb identified it immediately:

lsusb
Bus 001 Device 006: ID 0fd9:0008 Elgato Systems GmbH

I googled the device Id's (0fd9:0008) and discovered that the device is actually a Hauppauge HVR950Q based around the Auvitek AU8522 QAM/8VSB Frontend chip. The hardware pages on linux tv are somewhat helpful but googling the device ids was more helpful.

I compiled a kernel, enabling the au8522 module. This is involved and tedious, but described well elsewhere so I won't repeat. I bought an MPEG2 license; this was super easy and cheap!

I installed some stuff:

sudo apt-get install w_scan dvb-apps

I grabbed a bunch of firmware files:

cd /lib/firmware/
sudo wget goo.gl/xZSPH -O dvb_firmware.tar.gz
sudo tar xzf dvb_firmware.tar.gz

(thanks to Quonith on stmlabs forum, below, for this.)

I may have had some other stuff installed previously (I have compiled xbmc locally before).

I added myself to the video group:

sudo gpasswd -a cmason video

substitute your username (ie pi).

I rebooted. I watched the output in dmesg:

usb 1-1.3.3: new high-speed USB device number 6 using dwc_otg
usb 1-1.3.3: New USB device found, idVendor=0fd9, idProduct=0008
usb 1-1.3.3: New USB device strings: Mfr=1, Product=2, SerialNumber=10
usb 1-1.3.3: Product: EyeTV Hybrid
usb 1-1.3.3: Manufacturer: Elgato
usb 1-1.3.3: SerialNumber: 4031657106
Linux video capture interface: v2.00
au0828 driver loaded
au0828: i2c bus registered
tveeprom 0-0050: Hauppauge model 72231, rev B3F0, serial# 5125266
tveeprom 0-0050: MAC address is 00:0d:fe:4e:34:92
tveeprom 0-0050: tuner model is Xceive XC5000 (idx 150, type 76)
tveeprom 0-0050: TV standards NTSC(M) ATSC/DVB Digital (eeprom 0x88)
tveeprom 0-0050: audio processor is AU8522 (idx 44)
tveeprom 0-0050: decoder processor is AU8522 (idx 42)
tveeprom 0-0050: has no radio, has IR receiver, has no IR transmitter
hauppauge_eeprom: hauppauge eeprom: model=72231
au8522 0-0047: creating new instance
au8522_decoder creating new instance...
i2c-core: driver [tuner] using legacy suspend method
i2c-core: driver [tuner] using legacy resume method
tuner 0-0061: Tuner -1 found with type(s) Radio TV.
xc5000 0-0061: creating new instance
xc5000: Successfully identified at address 0x61
xc5000: Firmware has not been loaded previously
au8522 0-0047: attaching existing instance
xc5000 0-0061: attaching existing instance
xc5000: Successfully identified at address 0x61
xc5000: Firmware has not been loaded previously
DVB: registering new adapter (au0828)
DVB: registering adapter 0 frontend 0 (Auvitek AU8522 QAM/8VSB Frontend)...
Registered device AU0828 [Hauppauge HVR950Q]
...

I then ran w_scan in order to scan for stations (called "muxes" because each frequency has potentially multi multiplexed av streams).

w_scan -A1 -X -c US -fa -t3 > ~/.azap/channels.conf
tune to: 8VSB f=195000 kHz
service is running. Channel number: 10:1. Name: 'OPB'
service is running. Channel number: 10:2. Name: 'OPBPlus'
service is running. Channel number: 10:3. Name: 'OPB-FM'
... many services listed ...

This specific w_scan incantation is pretty important as the defaults are for European DVB setups and result in incorrect channel info. This takes a little while, but eventually produces a channels file that looks like:

OPB:195000000:8VSB:49:52:3
OPBPlus:195000000:8VSB:65:68:4
OPB-FM:195000000:8VSB:81:84:5
... more channels ...

Next, I tune one of these channels using azap:

azap -r OPB
using '/dev/dvb/adapter0/frontend0' and '/dev/dvb/adapter0/demux0'
tuning to 195000000 Hz
video pid 0x0031, audio pid 0x0034
status 00 | signal 0073 | snr 0087 | ber 000000ff | unc 000000ff |
status 1f | signal 00fa | snr 00fa | ber 00000000 | unc 00000000 | FE_HAS_LOCK
status 1f | signal 00fa | snr 00fa | ber 00000000 | unc 00000000 | FE_HAS_LOCK

Azap needs to continue to run while you view/capture, but it produces a bunch of output, so, after I verify it's working (HAS_LOCK), I usually run it like:

azap -r OPB >& /dev/null &

The -r option is important, it sets up the dvr device. Then we play using omxplayer (I can't recall how I got this installed but I'm sure there's other posts about this):

omxplayer -o hdmi /dev/dvb/adapter0/dvr0

-o hdmi was necessary to get hdmi audio (I guess the default is the headphone jack).

This results in flawless, stutter free video and audio on the hdmi connector! Yipee! The load is amazingly low (like 30%).


A couple points that were confusing to me:
The resulting data from the device is an MPEG "transport stream." This is pretty amazing because you can basically start reading it from any point and you'll quickly end up with valid video. You can cat (or dd, as cat sometimes reports errors) this to a file and then read it later. The sdcard I have seems plenty fast enough to support something like a pause live tv buffer although I haven't found software yet to do this.
There's different sets of related utilities for atsc versus dvb-t versus other stuff. It's important to use the right set, and the corresponding channels.conf file has a different format above. Be sure you use the right w_scan options for your part of the world.


These pages were super helpful:
http://forum.stmlabs.com/showthread.php?tid=2648
http://www.linuxtv.org/wiki/index.php/T ... DVB_device
http://linuxtv.org/wiki/index.php/W_scan
http://www.linuxtv.org/wiki/index.php/Zap


Things that didn't work:

• dvbstreamer (never sends anything)
  mplayer (segfaults)
  vlc on my mac (neither streaming nor reading the ts file directly)
  tvheadend

I struggled with tvheadend a bunch, and just can't get it to work. It shows some muxes and some services but none of the information seems right and attemps to play fail with "Transport error indicator." I've reported to the tvheadend folks, and I'll post again here if I get it working.

Nits:

• omxplayer is pretty finicky about hiccups in the stream and seems to get stuck in unkillable D (disk) wait easily. I end up rebooting a bunch. But once it's playing and has good signal it seems pretty solid.
  My (admittedly ancient) Panasonic Plasma isn't recognized very well and none of the modes are 16:9. (I'll make a separate post about this with the edid data.) This results in letterboxed video, which is not too terrible except that:
  I can't figure out a good way to hide the console and because the video is letterboxed it shows the blinking cursor and part of the pi logo. This is super annoying. I tried a number of things suggested on the forums (including setterm -blank force) but none of them worked I think perhaps because I'm ssh'd in.

Next steps:

• Investigate cheaper hardware. I've seem some listed for $40, although it's not clear it'll be compatible.
  Get tvheadend working or find something else.
  I'd really like to figure out transcoding so I can stream to normal web browsers without having special plugins installed.

[cbsmith402]

Awesome progress! I was hoping to do the same when I had some time, but I never got around to it.

I'd be especially interested in how you get transcoding working. I'm a developer, but looking at the hello_encode sample, dealing with OpenMAX seems pretty daunting. How were you planning to handle that?

Definitely keep the thread updated with your progress.

[batjac]

CMason, great post. I'll be watching, since I'm trying to get rid of the amazingly expensive cable bill. I'm a fellow Portlander (Hillsboro, actually), so if it works for you, it will probably work for me.

[cmason]

I've still had basically no luck with tvheadend. It finds services but never the names for them, it never creates any channels and when I try to stream, I get odd error messages. I'm still following up with them, but I've basically given up hope of it ever working.

I really have three use cases: One watch live TV on the PIs HDMI out, Two stream live TV from the PI to my mac, and Three schedule a recording to watch later. I've gotten One working as described above and hoped to use tvheadend to do Two and Three. For Two, I've had much better luck using VLC's command line streaming program clvc to send from the raspberry pi to VLC running on my mac.

sudo apt-get install vlc

You can either specify a particular channel directly by frequency:

cvlc atsc:// :dvb-frequency=195000000 :dvb-adapter=0 :dvb-srate=0 :dvb-modulation=8VSB --program=3 --sout "#standard{access=http,mux=ts,dst=:8081}"

Change 195000000 and 3 above as appropriate. Then choose Open Network on the Mac, and enter http://rasp.local:8081 (I have avahi installed) and it should stream over http.

Or you can use the text mode interface (-I ncurses) and specify a channels.conf as a playlist:

cvlc -I ncurses .azap/channels.conf --sout "#standard{access=http,mux=ts,dst=:8081}"

VLC also supposedly has an http remote interface, but I haven't gotten it to work.

I've also been experimenting with multicast and had some success:

cvlc atsc:// :dvb-frequency=183000000 :dvb-adapter=0 :dvb-srate=0 :dvb-modulation=8VSB --program=3 --sout '#rtp{dst=239.255.12.24,port=1234,sdp=sap://,name=KGWHD}'

This actually shows up automatically in VLC's "Network Streams (SAP)" tab. I haven't figured out how to get cvlc to advertise all channels in channels.conf via SAP but then change the tuner when one is requested.

Note that this is all highly dependent on network bandwidth. I've either been using ethernet directly, or a WNDR3700 and putting the rasp on 2.4 Ghz and my mac on 5ghz to ensure enough bandwidth. Turning on "turbo" and dramatically reducing the wireless ack timing range (to say 200m) helps as well.

Anyway, still haven't really de-futzified this to the point that my girlfriend can use it.

-c

[pedroserano]

Thanks for the great post, this has helped me get my eyetv hybrid going by leaps and bounds. One minor correction that almost drove me crazy: sudo apt-get install w_scan dvb-apps is incorrect, it should be:

sudo apt-get install w-scan dvb-apps

The hyphen tripped me up until I checked the package name on debian. Posting this so others can avoid this potential pitfall. Thanks again.

[edslatt]

What a great post and idea! I like your “de-futzing for girlfriend usage” point. I’m convinced that if I could provide an interactive menu like the one provided from the cable company I would be able to drop my cable subscription and just keep the internet connection. The stations we watch come in on the antenna but I find myself flipping inputs back to the cable box for the listings.

It seems like the rpi might support a very polished capability here. I’m a developer and would love to contribute if someone is interested in organizing a collaboration.

[bonelifer]

What a great post and idea! I like your “de-futzing for girlfriend usage” point. I’m convinced that if I could provide an interactive menu like the one provided from the cable company I would be able to drop my cable subscription and just keep the internet connection. The stations we watch come in on the antenna but I find myself flipping inputs back to the cable box for the listings.

It seems like the rpi might support a very polished capability here. I’m a developer and would love to contribute if someone is interested in organizing a collaboration.

I can suggest you look into XMLTV for getting the station feeds. Since you posted to this topic, I'll assume you're in the USA or Canada(ie North America), so I'll also suggest subscribing to SchedulesDirect and use XMLTV 's "tv_grab_na_dd" utility to grab your line-up and schedule.

For XMLTV I'll suggest these links:
http://wiki.xmltv.org/index.php/Main_Page
http://sourceforge.net/projects/xmltv/

For SchedulesDirect:
http://www.schedulesdirect.org/

As for what language you use, that's up to you. I'm not a programmer, but that definitely seems interesting. Hopefully some people pick this up and make it into a project.

[edslatt]

This is a great head start- thanks. I'm going to try to mess around with a menu. It'll be based on one of these feeds and use a conf file to limit the rows to the stations that are available on the antenna. I think the conf file should also allow custom “channel” numbers and descriptions to be specified. I want an experience as close to the cable box menu as I can get (at least to start) because that’s what’s I’m used to. I’d also like to work out a ir remote capability that actually uses the cable box remote.

I wonder if I can get signal strength for particular frequencies from the hardware. I also wonder if I can stream the video into a subsection of the screen so that the current channel is displayed while using the menu.

BTW - I live in NJ and I point my bowtie antenna at NYC. I get about 20 channels but only half are in English. I do get NBC, FOX and CBS but it doesn’t look like I get ABC.

[bonelifer]

Once you get the lineup for your area, you can edit which stations get pulled down when you update the XMLTV file on the SchedulesDirect site.

I wonder if I can get signal strength for particular frequencies from the hardware.

If you use a HDHomeRun tuner, you would be able to get that, at least I think you can.

[edslatt]

I just stumbled onto http://www.simple.tv, which looks very close to what I was thinking.

[BJG]

Thanks for the great post, this has helped me get my eyetv hybrid going by leaps and bounds. One minor correction that almost drove me crazy: sudo apt-get install w_scan dvb-apps is incorrect, it should be:

sudo apt-get install w-scan dvb-apps

The hyphen tripped me up until I checked the package name on debian. Posting this so others can avoid this potential pitfall. Thanks again.

Did you get the eyetv working?

[Inept-Get]

Concerning tvheadend and Hybrid tuners: I found after a lot of struggling that tvheadend will not initialize a hybrid tuner. However w_scan does it without a problem. So I set up a script to initialize it for me at boot. I called it Terratec-H5-Initialize.sh
#!/bin/bash
# Script to enable the Digital mode of a Hybrid DVB stick
# E.g. Terratec H5, Terratec Cinergy HTC
# WIN-TVR-900, WIN-TVR-950
#
# First stop tvheadend if it's already running
sudo service tvheadend stop
sleep 5
#
# Then run w_scan for a few seconds to switch the tuner
# into the correct mode for digital terrestrial (DVB-T)
timeout 5 w_scan -ft -cGB
sleep 5
#
# Then restart tvheadend
sudo service tvheadend start
#
# The end

You'll need to set it for Cable TV in the US, so change the w_scan parameters accordingly

Then to make it work at start time, I added the following 2 lines to /etc/rc.local
# The following script will initialize the DVB-T adapter
home/pi/scripts/Terratec-H5-Initialize.sh

You'll need to edit the above line to suit your script folder.