« March 2008 | Main | January 2009 »

December 2008 Archives

December 25, 2008

Tivo -> OS X Quicktime

(resuming blogging as if I had never stopped…)

I spent a good deal of time researching this recently, so I thought I’d put it down in an entry so that it can be found by others.

Two facts make it desirable for me to download shows from my Tivo to my Leopard OS X /MacBook Pro laptop and/or iPod:

  1. I travel not infrequently
  2. I’m usually behind on my Tivo shows

I bought the Tivo Roxio Toast application (which is Tivo’s native download/view application), but all it does is queue up downloads and have a native MPEG 2 player to play the Tivo videos. It does not make videos playable on your iPod, or allow you to “export” them to other players.

However, I do notice (from Toast) that Tivo now supports some flavor of “resuming” downloads. I have tried HTTP resume with curl (via manually HTTP downloading Tivo files) at one point and it didn’t work; perhaps one of the tivo software updates has enabled HTTP “resume”…? Resuming downloads is very handy if you’re halfway through downloading a 1+GB file and the connection gets cut for some reason.

So I abondoned Toast and started using TivoDecode Manager (TDM) for this task. TDM downloads files from your Tivo, decodes them, and then morphs the file into a format that is both viewable in Quicktime and on iPods. Unfortunately, I grew frustrated with its slow / balky / unpredictable / buggy graphical interface.

So I started doing the following manual process instead:

  1. Visit http://my_tivo_ip_address/
  2. Type in “tivo” for the username and my Media Access Key (MAK) for the password.
  3. See a list of all the shows that are currently on my Tivo.
  4. Right click to copy the link for each show I want to copy.
  5. Past the link into a perl script that downloads them all for me. Why so much trouble — why not download them directly from my web browser? There’s a few reasons:
    1. An hour long show is over a gigabyte — they’re huge files
    2. My Tivo is on my wireless network, and while I have a pair of rockin’ Cisco Aeronet 1132’s for 802.11g speeds, it still takes a long time to download each show (I never bothered to figure it out conclusively, but it seems like the download speed from Tivo is rate-limited, either by the cheap USB dongle-to-wireless NIC that I have on it, or by the Tivo software itself)
    3. Tivo only lets one show be downloaded at a time
    4. I have a Linux server at home which is powered on 24/7; it’s perfect for “in the background” downloading like this
    5. Plus, the Linux server is on the wired side of my LAN, so the Tivo is the only entity sucking up wireless bandwidth — I don’t have both the Tivo and the downloading computer competing for wireless bandwidth
    6. Summary: When recording at “medium” quality (which is what I have set as the default on my Tivo), it takes 45 minutes to download a 1 hour show to my Linux server (yow!). So it is much better to setup a batch script to download and let it run unattended.
  6. I then use the TivoDecode command line tool to decode the file into an MPEG 2.
  7. There are then 2 options for watching the file:
    1. The mplayer application is a fine player that natively plays MPEG 2 videos
    2. Or, if I want to watch it on my iPod (sometimes its just much more convenient to pull out an iPod on a plane rather than a big MacBook Pro — especially since I tend to fly cattle class…), I re-encode it to MPEG 4. The resulting MPEG 4 file can be dragged-n-dropped into iTunes and synced to my iPod in the normal way.

The recoding from MPEG 2 to MPEG 4 step is the tricky part, especially for those who are not experts in video / audio encoding technologies (like me). As I mentioned above, I used to use the TivoDecode Manager (TDM) for converting the files to MPEG 4. TDM simply invokes some other command line tools for the majority of its work, so I stole a bunch of its ideas and hacked up my own “convert” script and used the command line tools that were included in the TDM application bundle (specifically: “mencoder” from the mplayer package).

However, there were two problems with the “mencoder” that ships with TDM:

  1. It is not compiled with MMX or SSE support (even though mencoder does support both MMX and SSE). I’m running on Intel chips that have MMX support — using the mencoder support for MMX greatly speeds up the encoding process.
  2. It sometimes fails to convert Tivo MPEG 2 files to MPEG 4; it’ll convert the first minute or two of the file, and then abort, complaining about too many audio frames buffered.

The second one was the stickler for me — there were some shows that it just plain refused to convert. I don’t really know enough about audio / video encoding to know what the exact problem is, but it is very repeatable (with some shows). Shrug.

So I finally got some time and looked into this. Specifically, I downloaded and compiled my own mplayer suite (including mencoder) — that’s when I discovered that using MMX and/or SSE support makes a huge difference in encoding time. Ensure to have libflaac installed first so that mencoder can use the AAC audio encoder for the soundtrack. I’m sure there are other audio codecs that Quicktime supports, but I could only get videos encoded with AAC to play properly in Quicktime.

Note: I used Darwin ports to install the “faac” and “lame” packages to get mencoder (and later, ffmpeg — see below) to include both libflaac and libmp3lame encoding support.

But no matter what options I tried, I could not get mencoder to re-code my problematic MPEG 2 files to MPEG 4. Doh!

FWIW, here’s the command line that I was using for mencoder (originally taken from TDM):

mencoder input.mp2 -o output.mp4 -oac faac -ovc lavc -of lavf -lavcopts \
  keyint=250:aglobal=1:vglobal=1:coder=1:vcodec=mpeg4:vbitrate=1800:abitrate=128 \
  -vf pp=lb,scale=640:480,harddup -demuxer lavf -lavfdopts probesize=128

So then I tried a different encoder software package: ffmpeg. After a bunch of experimentation, I built it with the following (I used the gcc binaries from hpc.sf.net)

    # The --cc=gcc=4.0 is *necessary* otherwise the MMX stuff will
    # fail to compile (ensure to install gcc 4.x first!)
    ./configure --prefix=/usr/local \
             --extra-ldflags=-L/opt/local/lib \
             --extra-cflags=-I/opt/local/include \
             --disable-vhook \
             --enable-libfaac \
             --enable-libmp3lame \
             --enable-gpl \
             --enable-pthreads \
             --enable-shared \
             --enable-nonfree \
             --cc=gcc-4.0
    make -j4
    sudo make install

(assuming that libfaac and libmp3lame are installed in /opt/local, via Darwin ports)

I then encode MPEG 2 files to MPEG 4 with the following:

ffmpeg -y -i input.mp2 -acodec libfaac -vcodec mpeg4 -sameq -g 300 -bufsize \
    14745k -b 2000k -aspect 4:3 -maxrate 16000k -f mp4 output.mp4

Some disclaimers on this:

  • The resulting ffmpeg-generated MPEG 4 files are a bit larger than the MPEG 4 files generated by mencoder, but I don’t really understand all of the ramifications of all of the command-line options that I am using. So there’s probably a good reason for that, but I don’t know what it is.
  • The ffmpeg encoding process is a lot faster that when I’m used to with TDM’s mencoder. I think that there are (at least) three reasons for this:
    • I’m using ffmpeg’s MMX support (remember that the precompiled mencoder I had did not have MMX support, even though mencoder itself does support MMX)
    • I did play around with command line options to get “fast” settings (this may have sacrificed some audio/video quality, but I am nowhere near an expert, so my untrained eyes/ears can’t tell the difference in the output)
    • I got a recent SVN checkout of ffmpeg — a much, much newer version (SVN r16304) than the mencoder version I was using

It would be really great if someone would re-do TDM “right” — have a nice integrated GUI that does all the downloading and re-encoding and importing into iTunes for you. As it stands, both Toast and TDM fail “the spouse test”: my very-intelligent-but-not-a-computer-scientist wife would be unable to reliably download files from Tivo and convert them to iPod format. And the manual process that I use fails the spouse test because it requires shell commands (that’s an automatic disqualifier). Blah.

I wish that I had the time to re-do TDM, but sadly, I do not. Any takers?

About December 2008

This page contains all entries posted to JeffJournal in December 2008. They are listed from oldest to newest.

March 2008 is the previous archive.

January 2009 is the next archive.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.34