How to get accurate playback position?

[Darkcobra]

I'm trying to synchronize playback on two computers. Seems like a simple enough thing...

But I've been working the last four days, and have reached my frustration limit. Here's what I've tried so far:

1) Using VideoLAN with the netsync plugin. It performs a seek when the error reaches a threshold, which causes an obvious jump and is unacceptably ugly. I want to use VLC's ability to adjust the playback rate to smoothly correct instead.

2) Modifying the plugin. I'm a Windows/MCU developer and am not terribly familiar with the required C toolchains, Linux, repositories, etc. Spent two days without success, and gave up to try other options. (If a preconfigured VirtualBox image capable of successfully compiling VLC were available, it would have been a HUGE help, hint hint. I found better coders than me struggling with this as well.)

3) Writing my own simple media player using libvlc. Explored the available .NET wrappers to find them outdated, incomplete, or non-functional.

4) Writing my own wrapper. Was going well until I found that both libvlc_media_player_get_position and libvlc_media_player_get_time only update 3-4 times a second, crippling their accuracy. Worse, I found mention of a bug that causes them to stop updating altogether under certain conditions. Unacceptable.

5) Getting the playback position the way netsync does it - using the input_Control function in libvlccore with the INPUT_GET_PCR_SYSTEM parameter. This looked extremely promising until I discovered that if playback rate is altered, the times returned by this function don't likewise speed up/down, and no longer bear any relationship to actual media position.

I've run out of ideas, and am REALLY tired of dead end approaches at this point. So I'm asking for help. How do you get an instantaneous and reasonably accurate playback position, with none of the bugs/limitations/caveats mentioned above?

[Darkcobra]

Although I'd still be interested to hear any more elegant solutions, I have something that seems to work.

I poll libvlc_media_player_get_time 40x per second. When the value changes, I save both the value and the system time when it changed. When I want an instantaneous position, my wrapper returns savedvalue+(currentsystemtime-savedsystemtime). Of course, I'll also need to implement something for when video is paused, since libvlc_media_player_get_time stops updating, my synthetic position keeps increasing; I haven't gotten around to this yet. Also, using VLC callbacks would be slightly more accurate/efficient for getting position updates, though probably not so much that I'll bother.

A single server plays the video, and sends current position to any number of clients. If a seek is performed on the server, or the difference between the server position differs from the position on a client by more than five seconds, the client performs a seek. Otherwise, the client continuously adjusts the playback rate to minimize the difference.

This is working well enough that on two systems with identical delay between input and output, they can play in good enough sync - even in the same room - that no audio "echo" effect is evident. Nice.

So far as I can tell, the delay between input and output appears to be returned by the input_Control function in libvlccore with the INPUT_GET_PCR_SYSTEM parameter. I'll soon try factoring that into my sync, along with network latency (for passed synchronization messages), and a fixed manual adjustment as well - though none of these have yet proved necessary in my initial tests.

Also, I notice that on the client there is a faint click in the audio, which appears to occur when the playback rate crosses from one side of 1.0 to the other. I may be able to reduce or eliminate this by avoiding too many needless crossings, as there is currently no "dead zone" or hysteresis for rate adjustments. Perhaps some audio post-processing that detects and removes clicks may be beneficial as well.

Further tests and improvements are ongoing. In the next days I'm going to run the server as my main media player for everything, with three clients linked to it, and see if any additional issues crop up.

[Rémi Denis-Courmont]

Accurate timestamps, you can get within the pipeline. But outside of the pipeline, accurate playback time does not really have a clear meaning. Each component in the pipeline has a different notion of its current offset, with varying accuracy, and typically not expressed in terms of normal playback. I think you need to define your problem better.

[Darkcobra]

I understand the basic concept of the pipeline, even if I'm fuzzy on the exact details. And if viewed from the context of the pipeline, I agree that my problem was ill-defined.

But in the context of my stated application - synchronizing what the user sees and hears on two separate computers playing the same file - the problem was clear enough: what value can I get from VLC that is best suited for comparing the difference? And it seems that all of the available values have some sort of issue. At least the slow update rate of the value I settled on using could be worked around.

A few more updates:

1) I successfully solved the audio click/pop problem by having the server player play at a rate of 1.03, instead of 1.0. That way whatever clocks on the clients determine playback rate can be up to 3% faster than the server, without causing them to slow their rate to <=1.0.

2) I ran a test with four clients, two wired and two wireless. This worked quite well - most of the time. There were sporadic dropped frames on the wireless clients, resulting in ugly macroblock artifacts. Peak and average wireless throughput was more than sufficient, but it dropped periodically, causing the dropped frames. Perhaps the wireless could be set up better, but I want my player to handle issues such as these with relative grace. So I increased VLC's network cache setting, which helped, but it had to be turned up a surprising amount. And this caused new issues. It was then slow to start playing, as it has to fill the entire cache before playback starts. Worse, if two seeks (or more) are requested within quick succession, it seems VLC cannot abandon the previous cache fill operation - even though it's typically pointless to continue it - which makes skipping ahead/back in increments unusably slow. It would be nice if it had the option to start playing when a certain amount of the cache is filled, as I'd gladly accept dropped frames when starting a video or after seeks if it means faster response time, but I found no documentation for such an option. At least while searching for this I found other players that allow it, or otherwise handle the cache more intelligently, and did some quick tests. MPlayer produces far fewer dropped frames over my wireless than VLC, even when the cache sizes were equivalent and small. As it also appears to fulfill my other requirements for a back-end at least as well or better than VLC, I'm going to adapt my synchronized player to it and see how it works instead.

[Rémi Denis-Courmont]

There is nothing besides the netsync plugin and its "back-end". This predates the playback rate feature, so maybe this is broken as you say.

You may need to fix it or replace it.

[Jean-Baptiste Kempf]

Look at syncplay.pl too.