13. Juli, 2010

Encode and deliver MP4 Video for Flash, iPhone, iPad, Android, Quicktime with timeshift possibility

Von webfraggle

A big challenge was to find a solution to encode and deliver videos for as much platforms as possible. The only thing that was clear I had to use mpeg4 for max. compatibility.

Initial Position: We have many PAL videos with up to 20 different languages. All are around an hour. They are organized in chapters.

Client needs:

  • Multilanguage
  • Timeshift / Pseudo-streaming / Random access to jump directly to a chapter
  • Two different bandwidths
  • Maximum quality
  • Download security
  • Platforms: Desktop (Flash), iPhone, iPad, Android and maybe more in the future

My main thought was, ok, many of the features needed will work with Adobes Flash Media Server. But how can I deliver skippable video to iDevices with the same comfort. And another challenge was the client’s network. It’s a big company so they have a firewall and a proxy with a strong content filter.

Server possibilities:

  • Flash Media Server (rtmp and rtmpt via http tunneling, easy install, works out of the box)
  • red5 (rtmp, not so easy to configure, java knowledge needed)
  • Webserver only (http progressive download no skipping)
  • xmoovstream (http, php script, random access for iDevices but not for mp4 in combination with flash)
  • H264 streaming extension by trac for a variety of webservers (http, pseudo-streaming mp4 video for Flash and iDevices)

After some tests I made my decision to use Flash Media Server with the bundled Apache webserver extended by the h264 streaming extension. This works fine for all purposes and is in my opinion the best and easiest combination of rtmp and http with skipping directly into the video without loading it completely, I think it’s also called random access. And the best, you only need to store your files once on the server.

And now to the encoding part. It must be a fast encoding because we have many different audio languages, the video track is almost the same. The only way to do this, is the possibility to encode the video separately from the audio and muxing it together.

Encoding possibilities:

  • Adobe Media Encoder (Good encoding quality, No muxing (maybe it can be done with external tools), sometimes the videos will not play on iDevices, Flash needs to be installed on the encoding computer)
  • Sorenson Squeeze (No muxing(maybe it can be done with external tools), didn’t find any good quality encoding settings)
  • ffmpeg in combination with mp4box (Good encoding quality, muxing possible, nothing needs to be installed on the encoding computer, just put the binaries, a batch file and the source files on the network or a external hard-drive and go)

It’s clear, that I used the last one. If you want to use it too, just google for some binaries for your operating system. Important is that you find a ffmpeg version with libx264 included.

Encoding settings:

Both are two pass encoding settings for best quality

High Quality (Video & Audio) – Flash, iPad, Quicktime (not on iPhone because this uses a better h264 profile the iPhone can’t play) maybe iPhone 4:

Pass 1;

ffmpeg -y -i in.avi -pass 1 -vcodec libx264 -b 750k -g 300 -bf 3 -refs 6 -b_strategy 1 -coder 1 -qmin 10 -qmax 51 -sc_threshold 40 -flags +loop -cmp +chroma -me_range 16 -me_method umh -subq 7 -i_qfactor 0.71 -qcomp 0.6 -qdiff 4 -directpred 3 -flags2 +dct8x8+wpred+bpyramid+mixed_refs -trellis 1 -partitions +parti8x8+parti4x4+partp8x8+partp4x4+partb8x8 -acodec libfaac -ab 64k out-high.mp4

Pass 2:

ffmpeg -y -i in.avi -pass 2 -vcodec libx264 -b 750k -g 300 -bf 3 -refs 6 -b_strategy 1 -coder 1 -qmin 10 -qmax 51 -sc_threshold 40 -flags +loop -cmp +chroma -me_range 16 -me_method umh -subq 7 -i_qfactor 0.71 -qcomp 0.6 -qdiff 4 -directpred 3 -flags2 +dct8x8+wpred+bpyramid+mixed_refs -trellis 1 -partitions +parti8x8+parti4x4+partp8x8+partp4x4+partb8x8 -acodec libfaac -ab 64k out-high.mp4

Low Quality (Video & Audio) (works on all platforms):

Pass 1:

ffmpeg -y -i in.avi -pass 1 -s 384x288  -vcodec libx264 -b 230k -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8 -me_method hex -subq 7 -trellis 1 -refs 5 -bf 0 -flags2 +mixed_refs -coder 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4 -acodec libfaac -ab 32k -ac 1 out-low.mp4

Pass 2:

ffmpeg -y -i in.avi -pass 2 -s 384x288  -vcodec libx264 -b 230k -flags +loop+mv4 -cmp 256 -partitions +parti4x4+parti8x8+partp4x4+partp8x8+partb8x8 -me_method hex -subq 7 -trellis 1 -refs 5 -bf 0 -flags2 +mixed_refs -coder 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -qmin 10 -qmax 51 -qdiff 4 -acodec libfaac -ab 32k -ac 1 out-low.mp4

Adding the needed hinting stream and interleaving the stream in chunks (same for both):

mp4box -inter 500 -hint out-high.mp4
mp4box -inter 500 -hint out-low.mp4

Audio only encoding

ffmpeg -y -i "audio-lc.wav"  -acodec libfaac -ab 64k -ac 1 audio-lc-64.m4a
ffmpeg -y -i "audio-lc.wav"  -acodec libfaac -ab 32k -ac 1 audio-lc-32.m4a

Muxing Audio and Video together

mp4box -add audio-lc-32.m4a#audio out-low-lc.mp4
mp4box -add out-low.mp4#video out-low-lc.mp4
mp4box -inter 500 -hint out-low-lc.mp4<br/>
mp4box -add audio-lc-64.m4a#audio out-high-lc.mp4
mp4box -add out-high.mp4#video out-high-lc.mp4
mp4box -inter 500 -hint out-high-lc.mp4

A good player to test all this is the jw player. It is also available as html5 beta version so you can also try it on an iPad.

We are developing our own player because we need some special features like chaptermenus and subtitles.

Hope this can help someone. If you have some questions just ask in comments.