Grooming satellite video streams for recording on an Amino 5

[johnInHolyrood]

I want to share my experience with using VLC/x264 as a "video transrater" and look for some help tuning its performance. As I'm using the word, a transrater is a box that adjusts the bandwidth of a video stream.

APPLICATION : distributing satellite network HD video streams to a group of Amino 540 set top boxes for viewing and recording. The delivery network is capable of delivering a gigabit to each set top box, although the 540 only has a 100 mbps interface on it. The typical HD stream from the satellite receiver has average bandwidth around 10 mbps, but has sudden bursts to over 20 mbps. This can be seen in a wireshark packet capture i/o graph.

PROBLEM: a 540 could record one "raw" stream well, but when asked to record a second would introduce video glitches in the recording.

SOLUTION: Running the streams through VLC with video bandwidth set to 12 mbps allowed the 540 to record more than one stream without glitches.

IMPLEMENTATION: built machines using an i7-3770k processor running Debian Squeeze Linux, VLC 2.0.5 and x264 0.140 to process the streams and remove the bursts. On some streams, I found that the machine processed mpeg2 inputs more efficiently than mpeg4 inputs. Two streams can be processed per machine, if one stream used x264 preset=fast and the other used preset=faster. More streams could be processed per machine if a faster x264 preset was used, but there was a drop in quality that showed up in lost detail in faces, in stutters during slow scene pans, and in block patterns that "pulsed" on background set walls. Alternately, one machine could process a single stream at preset=medium, and the detailed improved, but when surveyed most people did not notice the difference at normal viewing distances.
EXPERIENCE: Since May 2013, four machines have been in production service 24 hours a day. They are slightly overclocked to allow peaks of 4.2 ghz. There is a script that watches stream output stalls and automatically restarts when it senses a stall. The machines do require rebooting twice a week to avoid a random lockup. I'm assuming there is a memory or resource leak somewhere causing this. I modified i7z to report CPU core temps to a Cacti, and also wrote a script to feed CPU uptime to Cacti, so I can watch loading and its relation to CPU core temps. The machine also runs a web server that allows reviewing the configurations and performance graphs, and on demand rebooting of the machine. Configuration is still done with VI and text files.

Here is the command line and vlm configuration being used:

/usr/local/bin/vlc -vvv -I telnet --vlm-conf ./data/$service.conf 2>> /home/admin/tc/data/$service.debug

# VLM CONFIGURATION
new ch123 broadcast enabled
setup ch123 input "udp://@237.1.2.3:1234"
setup ch123 option program=2
setup ch123 output #transcode{vcodec=mp4v,venc=x264{preset=fast,profile=high,vbv-maxrate=12000,vbv-bufsize=24000,min-keyint=60,keyint=120,threads=12},scale=1,acodec=mp4a,ab=128,channels=2,samplerate=48000}:std{access=udp{ttl=5},mux=ts,dst=238.1.2.3:1234}

REQUEST: I arrived at this configuration by a lot of reading not all of which I understood and a lot of trial and error. I'm sure there are people who have a much greater understanding of the VLC and x264 settings than I do. Can anyone suggest better settings to accomplish the task? In particular I would like to improve quality by setting both streams on a machine to use preset=fast or even better preset=medium. The i7-3770k has 8 cores, so a cpu load of 8 means all cpu resources are used. When I have tried for these more cpu intense presets after a few minutes of operation the cpu load heads for 10 or higher and the video starts to stutter. I have tried further overclocking of the i7-3770k to 4.4 ghz while watching the core temperatures, but it doesn't seem to change the uptime behavior.

Also, if anyone wants to send me a i7-4770k cpu/motherboard I'd be glad to try it. Likewise if anyone has suggestions for setting up a GPU to share some of the process, I've got the perfect lab.