Lowest Possible Streaming Latency with libVLC?

[elb75]

Hi -

Our project requires the lowest possible latency when streaming rtsp (udp) into the LibVLCSharp.WPF.VideoView control. We've smashed all the latency we can on the encoder/server/LAN.

* What, definitively, are the *best* settings to use to minimize latency (we can tolerate stuttering, pauses, etc. It is latency that we must minimize).
* Where can I find or enumerate a list of available settings? I have logged the results of 'vlc.exe --help' but already I am using settings not listed in that output.

Right now our best performance is ~550ms of total latency, of which we attribute 60-90ms to the encoder/server/LAN, meaning that ~460ms is due to the VideoView (VLC) control. Our total allowable latency spec is 250ms.

So far, our initialization is like this:

Code: Select all

_libVLC = new LibVLC("--file-caching=2000", "-vvv"); _mediaPlayer = new MediaPlayer(_libVLC); videoView.MediaPlayer = _mediaPlayer; if (CommandLineArgs.Args.Length > 0) { Media m = new Media(_libVLC, CommandLineArgs.Args[0], FromType.FromLocation); m.AddOption(":network-caching=150"); m.AddOption(":clock-jitter=0"); m.AddOption(":clock-syncro=0"); _mediaPlayer.Play(m); }

But how do we know if these settings are sufficent and optimal?
Is there any statistics that are accessible through the library to evaluate performance?

Some websites claim that 60ms latency should be achievable. Others claim that the settings above have brought their latency from 2s down to 200ms. I realize some of this can be affected by the encoding parameters, so I'd like to know what a best practice for that (h.264 encoding) is as well?

Basically, there is a whole lot of 'anecdotal' information on the web about low/zero latency streaming, but very little that I can take confidence in as being the best for my scenario.

Any thoughts, help, guidance, and/or advice would be appreciated.

[InTheWings]

It's not possible to achieve low latency over RTSP for multiple reasons.
You can lower a bit latency with --low-delay in 4.0

Videolabs has 60ms latency working. That's not clean enough to be released yet.

[elb75]

Hi @InTheWings,

I think I made brief contact with you yesterday on the IRC channel.

We started down the RTSP approach (with live555) because it was suggested to us. We have considerable development experience on our team, but no expertise in video streaming.

If you were given a blank slate (ignoring RTSP), what approach would you suggest?

Our software access starts with a callback providing H.264SE frames on an embedded device. Our requirement is to display the video on a LAN-wired-connected (1gb ethernet) desktop machine with a maximum latency of 250ms. We are not committed to RTSP, it was simply a vehicle towards what we thought was a solution.

Again, there are many 'anecdotes' on the web about video streaming, but not much 'hard knowledge' that I can find.

I appreciate your response in advance. Thanks.

[InTheWings]

As I mentioned we have modifications, you can't lower latency up some point with public releases.

Using vlc-4.0, you can use --low-delay option to kill 2 frames buffering and decoder delay.
If you are using RTSP, you can lower latency if there's a single stream and change code to compute PCR without offset.
Since you kill all possible reordering, your video must only contain P frames then.