Exporting Files (Testing A/V Sync)

[quu]

I have been doing some testing about audio/video sync with h.264 video and MP3 and AAC audio, and I was curious about how easy it would be to test the sync with VLC. When I export a file with VLC, is the timings of the file just like it had been played? Also, is it possible to export in uncompressed video? When I try to export an an AVI, i don't see uncompressed as a codec option. Maybe, is it possible to have a timecode overlaid on the video display VLC provides?

What i found with AVISynth testing was that FFAudioSource (which is supossed to be based on ffmpeg) is about a frame off when compared to DirectShowSource on the same audio source. Nero AAC files are a further frame off, which worries me, since I was told that it is better quality than FAAC.

I understand that VLC uses ffmpeg, and want to see if it is an artifact of the AVISynth plugin, or if ffmpeg also causes the 1 frame skew. Plus I want to see if in a "real playback" the Nero audio is still off when compared to the FAAC audio.

[Jean-Baptiste Kempf]

First, do not use .avi export of VLC without remuxing, it is just broken. Use .ts or .mp4

VLC can export in raw video, just specify a suitable FourCC.

VLC should export the way it was played, yes.

[quu]

is the MKV export up to snuff?

I can't figure out how to export an uncompressed video... i don't see the option (unless it is command line only)
anyway, I open up VLC 1.1.2
I go Media->Convert/Save
I add the mpeg-4 test file I want to convert to uncompressed, and then i click the "Convert/Save" button
I pick a destination file "test.mkv", and click the "create new profile" button
I pick MKV as the encapsulation and then go to the video codec tab
I enable "video" and scroll through the drop down list of codecs... and I see nothing that is uncompressed video. I am not looking for lossless, but uncompressed. I see mpeg-1 through 4, divx 1 through 3, h-263/264, VP8, WMV1 and 2, MJPEG, Theora, and Dirac... but i don't see the abiity to enter my own fourCC and i don't see an option for uncompressed

[quu]

This is with VLC 1.1.2 under windows XP
Also, default install (the vlcrc and all the files in that directory were deleted and automatically rebuilt when VLC started)

I generated a clip using AVISynth (basically a wave form of the audio with a time code), encoded that clip with x264, FAAC, and mp4box
Here is the clip if you want to see it - http://pics.livejournal.com/quu/pic/0005ftxk/ (follow this link, then right click and save the further link, otherwise quicktime tries to play the file, freezing your browser until it loads)

when I inspect that clip in AVISynth using FFMS2 (the ffmpeg plug in), I can notice that the audio is running about 1 frame slow
which you can see here - http://pics.livejournal.com/quu/pic/0005ehkt
The two timecodes are identical, so I know the video is in sync
The two wave forms should almost be identical if the audio and video was in sync, as you can see, the audio is about 1 frame slow (this is using FFAudioSource for the audio, if I use direct show, it is perfect).

VLC uses ffmpeg (I think) so i wanted to see if VLC would show the same sync problem, or an artifact of my test script.

I ran the following command line to export an uncompressed copy of the video, along with the audio

Code: Select all

vlc --high-priority --no-ffmpeg-hurry-up --vout=yuv --yuv-file=test.yuv --yuv-yuv4mpeg2 --aout=audiofile --audiofile-file=test.wav test.mp4 vlc://quit

I then used an AVISynth script to load the raw YUV and the WAV and looked at its results. I expect to see, at worse, an identical picture to the original.
http://pics.livejournal.com/quu/pic/0005drey/g55
what I see is the audio running about 3-4 frames to fast, and the video running almost a full second fast. (the fact that the audio and/or video is fast does not matter so much that they are fast by different amounts). and the video starts in sync (the audio 1 frame fast), and it changes, getting more out of sync, as the yuv export goes on.

Am I exporting the video wrong? is there a better way to check for the video sync?

[quu]

btw, I tried using the GUI to export this MP4 as a new MP4 (ie i wanted to use VLC to re encode the test.mp4 from above to a new test-recode.mp4) and the resulting reencode had no audio. If i play test.mp4 in VLC, i hear the audio.

Again, from a "fresh" vlc 1.1.2 I did the following
Start VLC -> Media -> Convert/Save
Add the test.mp4 to the files box, and then click Convert/Save
I set the name as "test-reencode.mp4"
I pick the "Video - H.264 + AAC (MP4)" as the drop down
I click start, it encodes the file, but when i play the resulting MP4, there is no audio (media info confirms, no audio track)

[quu]

I just took a snapshot of VLC playing the test.mp4 at frame 4,000,
http://pics.livejournal.com/quu/pic/0005gtwh
as you can see, when played back, the video is in sync (sadly, i can't test the audio this way), but when i watch and listen, the audio apears to be in sync (ie could be withing a frame of correct)... and is NOT a second off, as the YUV export seams to show

so obviously, I am testing the A/V sync of VLC wrong... but i don't know how to test it correctly. The one frame off from FFAudioSource made me paranoid.

[quu]

OK, i can't figure out any other way to test the A/V sync of various files with VLC than using a camcorder
(all files can be referenced here)
http://otakuvideo.com/~quu/vlc%20timings/
given the "test - Original.avs" was rendered an an uncompressed AVI
then that AVS file was rendered with x-264 and the audio with FAAC and NeroAACEnc (QTAACEnc has the same timing as FAAC) and merged into mp4 files and MKV files

when i did my "in computer" testing with AVISynth, the AVI and MKv files were perfect, while the MP4 files had audio that was ~1 frame off... purely in software rendering and display

So, I wanted to see what the Sync would be like if these five files were played through VLC to an external viewer. This is a windows vista x64 machine, I7-920 and GTX 285

I made a playlist, set up my DV camcorder right in front of the monitor, hit record, then play and let all 5 videos play. Afterwords i chopped about 30 seconds from the middle of each (since the full DV recording was 6+ gig in size) and built AVS scripts to re render the audio waveform, to see if the audio and video was out of sync to the listener (you can download all of the .mp4 files of the sync tests from that link)

Now... my camcorder does not have Mic in, so there is some noise, you kind of have to watch the entire 30 seconds of each clip to find reference points, to find places where you can prove or disprove sync... but there are there... not easy, but possible.

I forgot to make virtualdub smaller before taking the screen shots, i wanted the full amount of frames to be visible. All of the reference images are available in the above link.

The uncompressed AVI the audio is running slow, as perceived by an external to the computer viewer.
this was perfect when inspected synthetically

Both MKV files, the nero encoded audio and the faac audio, are running a little audio slow also, the nero much more than the faac audio.
The MKv with FAAC audio was perfect in the synthetic testing, while the nero was slightly slow. So it kind of makes sense.

Now the MP4 transport files (please remember, same elementary streams as the MKV, just a different wrapper)
While it is impossible to be 100%, the FAAC mp4 timings look almost dead on perfect. The amount of noise from the recording does muddy the waters, but I can find multiple points in the 30 second clip where spikes and dips between the recorded video, and the newly rendered from audio match up. The nero MP4 seams to be in much better shape. While not as perfect as the FAAC mp4 container test, the nero audio in the mp4 wrapper seams to be less than 1 frame off at each solid marker i can find, well within human perception.

so... this hurt my brain with the amount of jury rigging I had to do. This also completely surprised me with the results. Given the results of my synthetic testing, I expected the uncompressed avi (this was played off an array, no skipped or dropped frames) and the MKV to be dead on perfect, but their audio ran behind. In the synthetic testing the audio ran early when using the ffmpeg avisynth plugin, and this kind of matched the expected results from the analog viewer, as if everything was shifted back audio wise, their faster audio now perfectly matches.

I think the next time I do my timing results, I make my life simpler and use a tone generator instead of a real song, sharp tones on and off every second would make it easier to test for sync issues.

[quu]

btw, now that I think about things some more... this is not as bad as it looks.

the MKV and the AVI are pretty much in sync with each other... which my synthetic testing has shown. The reason the AVI and the FAAC MKV were out of audio sync with thier respective video could very easily be my computer and/or speaker system. VLC has an audio sync adjustment global setting... so at a convention I could play a test file to sync up with any timing issues with the sound system and projector before hand, part of normal setup.

the biggest question I had was if the audio skew that I saw in my synthetic testing was an artifact of the test, or an artifact of the underlying ffmpeg library that both VLC and the AVISynth plugin used. With this camcorder test, i can be assured that the issue is with the library, and that the overall results of my synthetic test are accurate when using VLC as the playback engine. I just need to be sure to use VLC's audio timing adjustment to account for A/V hardware at conventions, but sync based on an MKV or AVI.

I do have a question about VLC... why is it that when I export a video in VLC using the "Video - H.264 + AAC (MP4)" setting, there is no audio skew, but when i render things externally and use mp4box to wrap the files, i get the audio skew... it bears some investigating... maybe I am using mp4box wrong, or is VLC doing some sort of mpeg-4 muxing that audio box does not?

[quu]

This is my plan for testing A/V sync using VLC
http://quu.livejournal.com/819616.html

I will be experimenting with the -sout params I guess

[quu]

Using this as my reference file
http://otakuvideo.com/~quu/BSGTest.mpg

vlc bsgtest.mpg --sout=#transcode{vcodec=HFYU,acodec=s16l}:file{dst="test-huff-wav.mkv"} vlc://quit
2,148,061,033 bytes, mediainfo shows both video and audio tracks, looks fine
playable file, media info seams to be fine

vlc bsgtest.mpg --sout=#transcode{vcodec=HFYU,acodec=flac}:file{dst="test-huff-flac.mkv"} vlc://quit
3,451,630,445 bytes
unplayable file, mediainfo shows nothing

vlc bsgtest.mpg --sout=#transcode{vcodec=HFYU,acodec=s16l}:standard{access=file,mux=ts,dst=test-huff-wav.mpg} vlc://quit
-- zero byte file

vlc bsgtest.mpg --sout=#transcode{vcodec=HFYU,acodec=s16l}:file{dst="test-huff-wav.mpg"} vlc://quit
-- zero byte file

vlc bsgtest.mpg --sout=#transcode{vcodec=HFYU,acodec=s16l}:file{dst="test-huff-wav.avi"} vlc://quit
3,517,808,420 bytes
playable file, mediainfo only shows video track, even though audio track plays

vlc bsgtest.mpg --sout=#transcode{vcodec=HFYU,acodec=flac}:file{dst="test-huff-flac.avi"} vlc://quit
3,455,688,668 bytes
no audio on playback, mediainfo looks identical to the wav avi one, shows video, no audio

vlc bsgtest.mpg --sout=#transcode{vcodec=FFV1,acodec=s16l}:file{dst="test-ffv1-wav.mkv"} vlc://quit
1,627,380,729 bytes
video panel resizes to 720x480, but black video... video locks up after 19 seconds in
MediaInfo shows both streams

vlc bsgtest.mpg --sout=#transcode{vcodec=FFV1,acodec=flac}:file{dst="test-ffv1-flac.mkv"} vlc://quit
1,606,794,666 bytes
video panel resizes to 720x480, but black video... video locks up after 19 seconds in
looks OK in media info

it seams my best option is huffYUV and PCM audio in MKV? except when I try to open it in avisynth (so i can open it in virtualdub to check the sync), i get a codec not found... now i need to check ffmpegsource2 to see if it supports huffyuv

am I using the wrong codec for mpeg ts streams? I thought the transport stream could hold nearly any codec
do I include audio-sync? or is there other commands I should include.
what do I include to make sure it is as close to the displayed output?
If I include the post processing filters and or normalizers, does that get applied before the re encode?

[quu]

ok, i used virtualdub to export an uncompressed AVI, and then inspected the fourCC of it... "RGB "

vlc bsgtest.mpg --sout=#transcode{vcodec="RGB ",acodec=s16l}:file{dst="test-rgb-wav.mkv"} vlc://quit

that does not work, no video stream is output