Convert MPEGTS to MP4 and Maintain Sync
Posted: 18 Oct 2010 10:33
Overview:
Input:
We are doing a public project to record our city council meetings from two Axis IP MP4 cameras. Using VLC we are overlaying those videos on a larger canvas, side by side and adding the audio. The audio is coming in on one of the camera feeds as 8K ulaw that is fed to that camera via the line-in from the PA system.
The sections of the VLM instance we load up to grab the video and audio stream, this is fed into the record_session broadcast where it is combined and exported.
Stream.conf FileMuxing:
We convert the final video to H264 because it allows multi-threading, other formats have to use one thread and then can't keep up in the process with the live stream coming in. We save it to a TS format because if we try and save it to MP4 directly from VCL the resulting file is always out of sync, if we save it to TS we can play it with VLC with pretty good sync, I believe this is because TS includes timestamping through out the audio and video streams allowing it's playback to be synced up. Converting that TS to an MP4 with the same sync is the problem.
The VLM instance we load up, this is how we save our initial TS file, grabbing the from the "sfilter=mosaic"(video) and "bridge-in"(audio) as a sources, and also duplicate the chain stream it WOWZA for a live feed and the TS file.
Stream.conf FileProblem:
Obviously MPEG-TS is not a format we can put up on the web and give to our local cable company for broadcast on the public channel, so we try and use MP4Box / FFMPEG / VLC to convert the MPEG-TS file into an MP4 and FLV files. So while we can watch this file in VLC and it has sync when we feed it into any of these programs we get a 2 - 10 second drop in the sync.
When VLC makes the MPEG-TS file it seems to have problems with the video chain for the first 2 - 10 seconds as it builds a buffer for both of the two ip sources. This corrupts the streaming coming from "sfilter=mosaic" perhaps....
We have flagged the packets coming from the cameras and setup our hub to prioritize them so we can't really build the buffer faster then we already are, no matter what a buffer will need to build, unless VLC can fix this initial fire up of the video chain I need a way to deal with this outputted file.
FFMPEG see the corrupted frames and drops them but does not also throwing out the same amount of audio, causing our sync problem. We always have to go in and re-adjust the audio manually to get things into some sort of order.
The ffmpeg command we use to convert the ts to mp4, we use this output file to make the rest of our files.record_session.ts
build_template.mp4
Goals:
I'm posting this because I've spent literally way too many nights trying to plug this bug and it's obviously beyond me, too much new information in building this all up in the first place I can't come up with a decent combination of command or VLC setup that will produce a good insync file.
If we could get a VLC line that would save directly to a properly encoded h264 mp4 file that was as in sync as the TS file that would be great. We could run our script and make the FLV files out of that.
OR
So maybe that's just not possible, if we could get an FFMPEG or MP4Box command line, or some other tool that would convert the TS file we are getting into any type of file uncorrupted video file we could then use to make our FLV and MP4 files without introducing the desync we now consistently get.
Any help would be appreciated and I would certainly entertain compensation from people if someone wants to be paid to come up with a good fix for this.
All scrutiny welcome, I'm here to learn.
Thanks
Keith
Input:
We are doing a public project to record our city council meetings from two Axis IP MP4 cameras. Using VLC we are overlaying those videos on a larger canvas, side by side and adding the audio. The audio is coming in on one of the camera feeds as 8K ulaw that is fed to that camera via the line-in from the PA system.
The sections of the VLM instance we load up to grab the video and audio stream, this is fed into the record_session broadcast where it is combined and exported.
Stream.conf File
Code: Select all
#1
new camera1 broadcast enabled
setup camera1 input rtsp://xxxxx:xxxx@192.168.xxx.100/mpeg4/media.amp
setup camera1 output #duplicate{dst=mosaic-bridge{id=101},select=video,dst=std{access=file,mux=raw,dst="/usr/local/vlcdump/tmp/record_session-camera1.raw"}}
setup camera1 option no-sout-audio
#2
new camera2 broadcast enabled
setup camera2 input rtsp://xxxxx:xxxx@192.168.xxx.101/mpeg4/media.amp
setup camera2 output #duplicate{dst=mosaic-bridge{id=102},select=video,dst=std{access=file,mux=raw,dst="/usr/local/vlcdump/tmp/record_session-camera2.raw"}}
setup camera2 option no-sout-audio
#Audio coming in on #1
new audio_1 broadcast enabled
setup audio_1 input rtsp://root:greenlight@192.168.xxx.100/mpeg4/1/media.amp
setup audio_1 output #transcode{acodec=mp4a,ab=32,channels=1}:duplicate{dst=bridge-out{id=100},select=audio,dst=std{access=file,mux=mp4,dst="/usr/local/vlcdump/tmp/record_session-audio.m4a"},select=audio}
setup audio_1 option no-sout-video
We convert the final video to H264 because it allows multi-threading, other formats have to use one thread and then can't keep up in the process with the live stream coming in. We save it to a TS format because if we try and save it to MP4 directly from VCL the resulting file is always out of sync, if we save it to TS we can play it with VLC with pretty good sync, I believe this is because TS includes timestamping through out the audio and video streams allowing it's playback to be synced up. Converting that TS to an MP4 with the same sync is the problem.
The VLM instance we load up, this is how we save our initial TS file, grabbing the from the "sfilter=mosaic"(video) and "bridge-in"(audio) as a sources, and also duplicate the chain stream it WOWZA for a live feed and the TS file.
Stream.conf File
Code: Select all
new record_session broadcast enabled
setup record_session input "fake://" option "fake-file=/usr/local/vlcdump/lib/background-light-branded.png" option "fake-ar 16:9" option "fake-width=1440" option "fake-height=720"
setup record_session output #transcode{sfilter=mosaic,scale=1,venc=x264{keyint=60,profile=baseline,level=3.0,nocabac,qpmax=36,qpmin=10,me=hex,merange=24,subme=9,qcomp=0.6},vcodec=x264,vb=500,scale=.5,acodec=mp3,ab=128,channels=2,samplerate=44100}:bridge-in:duplicate{dst=std{access=file,mux=ts,dst="/usr/local/vlcdump/tmp/record_session.ts"},dst=rtp{dst=webserver,port-video=10000,port-audio=10002},dst=rtp{dst=127.0.0.1,port-video=10000,port-audio=10002,sdp=file:///usr/local/vlcdump/etc/record_stream.sdp}}
setup record_session option ts-shaping=500
setup record_session option ts-use-key-frames
# Mosaic options
setup record_session option mosaic-alpha=255
setup record_session option mosaic-height=720
setup record_session option mosaic-width=1440
setup record_session option mosaic-position=2
setup record_session option mosaic-rows=1
setup record_session option mosaic-cols=2
setup record_session option mosaic-order=101,102,Camera2Primary,Camera1Primary
setup record_session option mosaic-delay=0
setup record_session option mosaic-keep-picture
setup record_session option mosaic-keep-aspect-ratio
setup record_session option mosaic-offsets=0,8,736,8,107,2,107,2
Obviously MPEG-TS is not a format we can put up on the web and give to our local cable company for broadcast on the public channel, so we try and use MP4Box / FFMPEG / VLC to convert the MPEG-TS file into an MP4 and FLV files. So while we can watch this file in VLC and it has sync when we feed it into any of these programs we get a 2 - 10 second drop in the sync.
When VLC makes the MPEG-TS file it seems to have problems with the video chain for the first 2 - 10 seconds as it builds a buffer for both of the two ip sources. This corrupts the streaming coming from "sfilter=mosaic" perhaps....
We have flagged the packets coming from the cameras and setup our hub to prioritize them so we can't really build the buffer faster then we already are, no matter what a buffer will need to build, unless VLC can fix this initial fire up of the video chain I need a way to deal with this outputted file.
FFMPEG see the corrupted frames and drops them but does not also throwing out the same amount of audio, causing our sync problem. We always have to go in and re-adjust the audio manually to get things into some sort of order.
The ffmpeg command we use to convert the ts to mp4, we use this output file to make the rest of our files.
Code: Select all
ffmpeg -y -i ${SESSIONLOCAL}/original/[URL=http://www.green-light.ca/public/record_session.ts]record_session.ts[/URL] -vcodec copy \
-absf aac_adtstoasc -acodec libfaac -ab 16k -ac 1 -ar 11025 -copyts \
${SESSIONLOCAL}/debug/[URL=http://www.green-light.ca/public/build_template.mp4]build_template.mp4[/URL]
build_template.mp4
Goals:
I'm posting this because I've spent literally way too many nights trying to plug this bug and it's obviously beyond me, too much new information in building this all up in the first place I can't come up with a decent combination of command or VLC setup that will produce a good insync file.
If we could get a VLC line that would save directly to a properly encoded h264 mp4 file that was as in sync as the TS file that would be great. We could run our script and make the FLV files out of that.
OR
So maybe that's just not possible, if we could get an FFMPEG or MP4Box command line, or some other tool that would convert the TS file we are getting into any type of file uncorrupted video file we could then use to make our FLV and MP4 files without introducing the desync we now consistently get.
Any help would be appreciated and I would certainly entertain compensation from people if someone wants to be paid to come up with a good fix for this.
All scrutiny welcome, I'm here to learn.
Thanks
Keith