Delay in video streaming.

[Restoration]

I'm working on a project (based in windows 2000 O.S.) that requires streaming live video feeds (Mpeg4) over an internal network. The network isn't busy, so it moves pretty quickly. I am trying to decrease the delay in the video streams to as close to real-time as possible, but I am encountering difficulties getting the delays below ~5 seconds. I thought I might be able to reduce it a little further by setting the VLC buffer to zero, but this seems to have no effect. I'm wondering if anyone has any suggestions to increase the speed.

Network uses HTTP . I want to avoid switching to anything else if possible.

[Teta]

I'm remulticasting 6 satellite TV signals through a Windows 2003 Enterprise server on the network through udp multicast using untranscoded mpeg-ts signals that I get from 6 dreambox 7020's.

I have the same delay. It doesn't bother me, I just mention it so you know it has nothing to do with your network or chosen protocol.

Hope this helped,

Teta

[Restoration]

Thanks for your reply,

A little bit more information about it:
I'm multicasting using NI LabVIEW as a server and VLC as an ActiveX plugin for the video displays. I think (from reading through the forums) that the delay might be able to be resolved by reducing the buffer in the source code of VLC or even removing it entirely. Dropped frames are of not much concern here. Would this work?

Another option that I'm considering would be to work in a different packetizer. Does anyone know of any other players that would give real-time streaming?

[smccrudden]

we are also trying to eliminate the delay. any luck in reducing the buffer size?

[Restoration]

Nothing yet. I'll let you know what solution I come up with (looking more and more like a different media player at this point).

[DJ]

In VLC you can't eliminate the buffer as the player will stop functioning. Getting rid of the delay is a buffering issue for the length (size) of the buffer.

[Restoration]

What is the minimum buffer size that I could possibly use?
The video has to be as close to real time as possible (e.g. less than half a second at MOST) for safety reasons.

[DJ]

I'm not aware of any player that streams media files NOT having a delay as NONE are in real time. You can play with the buffer (Cache time in MS) size but this will depend on the format, container, resolution of the file, speed of the network and congestion on the network.

[Teta]

Are you sure the delay isn't coming from your multicast source, in your case LabView?

If you try another client (is there one with LabView?) do you have the same delay?

Teta

[Restoration]

I don't think its labview,

The delay appears even when labview is not running as the server/client.

[MikeQBF]

I'm not aware of any player that streams media files NOT having a delay as NONE are in real time. You can play with the buffer (Cache time in MS) size but this will depend on the format, container, resolution of the file, speed of the network and congestion on the network.

The problem we're referring to here is "latency". When you're dealing with live video streams and are controlling something (including PTZ) in real time by viewing that stream, buffering you can't control is a real teeth-gritter.

A lot of buffering is fine for stored-media video (not in real time), as the priority there is clean display. That's why WMV and QT have 5 seconds' worth of buffering. In real-time video (security applications, etc.), the priority is the real-time part of it - we don't care about frame drops or momentary breaks in the views, we want to see the camera pan when we press the pan button, and not 2-3 seconds after the fact.

DJ is right, latency is going to be dependent on nearly everything. However, I am finding that latency of some formats can be reduced to unnoticable levels, "unnoticable" being roughly 100-200ms. I am having good luck with MPEG-2 TS via UDP multicast with encoding rates ~3-4K.

However, I am not having much luck with MPEG-4 via RTSP. I can get latency down to 500-600ms by hammering on caching values, but after a day of poking at settings I am unable to do much better than that on either OS X or XP. There is a cache somewhere in the process that either I have missed, or can't be reduced through the GUI or command line.

One specific question: when you set a caching value to 0 ms or other low level, does VLC have a "nanny" function which overrides it with a minimum or optimum value you don't know about?

DJ, the problem with VLC in this area is that the values that affect latency are sort of spread all over the settings interface. If you are acquiring a camera view using "Default" for access and demux, without microsurgery (verbosity at "2"), it's difficult to figure out who the players are and what values need to be tweaked. If there was at least a way (...he queries...) to dump all current settings into human-readable form it would go a long way towards knowing what to mess with, and what not to (a/k/a "shotgunning").

[DJ]

I would tend to believe:

1. It should have been sufficiently discouraging to find that setting the cache to 0 MS the player stopped working correctly. VLC is a packet based player that can't work without a buffer.

2. VLC has been created as a collaborative work by programmers from all over the world. This probably includes some oddities revolving round one programmers method of doing things versus another as to how it relates to the whole. Perhaps the most noted example of this is seen in transcoding, where VLC is still source dependent.

[MikeQBF]

...I am encountering difficulties getting the delays below ~5 seconds. ...

Network uses HTTP . I want to avoid switching to anything else if possible.

Oooo. That's at least one of the problems. HTTP means TCP (versus UDP) and that means delays from handshaking and packet reconstruction.

[MikeQBF]

It should have been sufficiently discouraging to find that setting the cache to 0 MS the player stopped working correctly.

Ah, but nearly all caching values I've touched work fine at zero, but with the latency not solved. There seems to be a minimum point at <some value> where further adjustment makes no discernable difference. That's why I ask about "nanny" minimums.

2. VLC has been created as a collaborative work by programmers from all over the world. This probably includes some oddities revolving round one programmers method of doing things versus another as to how it relates to the whole. Perhaps the most noted example of this is seen in transcoding, where VLC is still source dependent.

Yes, that's a given, and of course we have to work around it, but, still, it's a major scavenger hunt for these values especially in the GUI.

I have the source code in-hand and will explore it tomorrow for insight.

[DJ]

...I am encountering difficulties getting the delays below ~5 seconds. ...

That's a very large buffer I would tend to believe there is something else wrong here. Though I suppose that the number of hops
and possible bounce off satellite could be adding to this, but collectively 5 sec. is kinda hard to swallow. I don't think watching the evening news from round the world live would be very intelligible or practical if we all needed to wait 5 sec for a response.

I remember putting a delayed feed into a sound stage to account for a satellite feed so that talent across the country could add to the collective work in real time. This concept even works for blue screen adds. One thing you will need to remember though, is that computers do not have PLLs and sync over time can be lost due to drift in clocks between machines. The other thing to remember is that this is done between 2 recording mediums and the compiled work will be done with an editor (picture) or fly it back into the original using Sample Frequency locked time code (multi-channel sound).

In the case of the News from round the world the original is delayed to match as closely as possible the satellite feed. Given the nature of the delay, the guess is never totally accurate, but does appear to be reasonably close in real time.

Unfortunately in the live news situation, form the performers point of view the delay is always present. Where as in the production studio situation the performers are provided a delayed fixed point in time but the product must be edited to appear correctly as a collective work.

Hope this is helpful somehow!

[Restoration]

DJ,

The signal is not going anywhere far. In fact, the signals come directly out of a frame grabber and into the server which then streams via LAN to a client station connected with less than 20 feet of cabling. There is 1 television hooked up to the frame grabber that shows real time footage, but when it comes through the server and gets run in VLC it is delayed by 5 seconds. Labview isn't causing the delay as far as I can tell (all parts of the code are executing in 1 second or less), so perhaps its in card itself. Hmmm.

On another note: I don't need sound at all, can I eliminate this to possibly increase the speed?

[DJ]

1. Seems doubtful this is a buffering issue as the buffer for 5 seconds of video would be massive. Setting up a bucket brigade for a FIFO would be very difficult to do to create a delay line for 5 seconds. Well I guess that today it's only a question of lots of memory, but I think I could sent it round the world in that time. Seems that some processing or transcoding would be necessary to create this delay.

2. VLC uses sync for sound and video. You can shut down the sound in preferences.

[Restoration]

Ok, So I have done a bit of searching and this person is having the same problem as I am:

viewtopic.php?p=103912&highlight=#103912

Looks like he has tried MPlayer, and the 1 second delay would be fine with me, but MPlayer doesn't have any activeX controls. This means I can't use it in Labview.

I am considering switching to WMP or RealPlayer and using UDP to see if that improves it any, but those media players don't have the proper codecs to view the streams. I'm still trying to figure out what codec is being used to encode the files into MPEG4 by the matrix cards. If anyone knows any good ones for WMP I would appreciate you sharing!

[DJ]

Quite a few people would like to have a 0 delay for whatever reason (mostly for playing games). This isn't going to happen!

[Restoration]

Thanks for your help DJ.

I'm now going through all the DLLs and codes that the server/video cards are using to see if theres an buffer written into the compression somewhere. I have been trying to work through the VLC code, so far nothing has worked, but with my limited knowledge on how to change things without screwing them up too much I think I might be in over my head.

[Restoration]

I fixed it, but I have no idea how. The video is displaying now with only ~0.5 seconds of lag time although it is freezing up a bit when speedy motion occurs.

I played around with the calling conventions of the in house code as well as setting each of the buffers in the advanced options to zero. I even tinkered with the stream portion of the VLC code (which failed horribly).
Some combination of what I did worked though. Its a start for others to get going on.

One downside: Something in the station now catastrophically fails every time the video is loaded giving me the BSOD.
Does VLC affect the ntdll.dll file in windows at all?

[schooloffish]

Hey, restoration I have the same problem with the video streaming delay that I am trying to resolve. Would you happen to still have the code, if so, I would greatly appreciate it if you could send it to me. If not, I would ask if you could help me out with my problem. Thanks