RTSP Frame Delay

[vocm88]

Hi forum!

I am using VLC for a project where I receive an RTSP stream from a network video encoder connected to a video camera. There is a delay from reality and what shows on the video feed, which I am aware can be configured using the RTSP caching value. (I have read the numerous posts on this forum regarding the cache factor and video performance). The strange thing is I have reduced the cache factor such that a 720 x 576 video feed playing at 25 fps appears live. However, if I configure the IP camera to encode at 3 FPS the video feed in VLC is clearly delayed by about half a second. The network IP manufacturer provides a browser interface which displays the video (which tunnels RTSP over HTTP) live at 3 FPS.

I am wondering if during decoding VLC holds an entire frame or two behind, such that for 25 FPS, the lag is not discernible, but for 3 FPS, a 2 frame lag would equate to about 0.666 seconds delay which is roughly what I observe. With this in mind, I don't think the delay has anything to do with the RTSP caching, but rather with another parameter or the decoding process.

If anyone can point me to suitable information/parameters I should investigate, or totally tear my argument up whilst shedding light to my misunderstandings, I would be incredibly grateful.

Kind regards,
Voc.

[Sébastien Escudier]

I am wondering if during decoding VLC holds an entire frame or two behind, such that for 25 FPS, the lag is not discernible, but for 3 FPS, a 2 frame lag would equate to about 0.666 seconds delay which is roughly what I observe.

Yes, this is the case.

[Rémi Denis-Courmont]

This is unavoidable. The video outputs or the video encoders can only deal with whole picture frames. So the VLC buffer is going to be at least one frame long. This is just a particular case of a more general rule: the buffer is always at least as long as the packetization time.

Then there can be further delays due to deinterlacing, if it's enabled.

[vocm88]

Hi guys,

Thanks a lot for the replies! Can I clarify a couple of things:

the buffer is always at least as long as the packetization time

Is it correct to say the above means that there is always an inherent lag from reality to encoder (packetization time during encoding)? Also, can there be any other buffers in VLC which hold back further frames? I ask this because the manufacturer provided video feed displays the 3 FPS video ahead of VLC which lags behind by about half a second. Is there some buffers/parameters which can be tweaked to lessen this.

Thanks again guys, I really appreciate it.
Kind regards,
Voc.

PS: I wasn't clear before, I'm NOT using VLC to encode as well as decode, the network device does the encoding. I am just using VLC to decode the H264 video.

[vocm88]

Hi guys,

I've decided to investigate compiling the latest release of ffmpeg with multi-threading along with vlc 1.2.0 in order to see if the multi-threaded decoding alleviates my problem. I will post results up here when I get them.

Cheers,
Voc.

[Sébastien Escudier]

Most of the delay is in vlc buffers, not in ffmpeg.

[vocm88]

Hi guys,

I have been away from my project for a while, but I have returned. I didn't have much success with the multi-threaded decoding (using latest ffmpeg and vlc). However, I want to touch up on what Remi said. The hardware encoder manufacturer provides an active X demo which shows decoded video. If I point my camera at a running stopwatch and the active x video display, I can see that for a 3FPS video, the delay is about 360 ms. This makes sense as it equates to encoding + packetization of 1 frame + other sources of delay. However, when I repeat the same experiment with VLC (with all configurable buffers removed) the delay is 630 ms. This delay would include encoding time, and similar decoding time to the active x demo, but about double the packetization time.

Does this means 2 frames are being buffered? If so why 2? Which file/method is this done within the VLC source code?

Any help would be appreciated. I want to be able to give back to this great project, so please give me some pointers/where I should start looking.
Kind regards,
Voc.

[Sébastien Escudier]

You may have to check the packetizer (modules/packetizer/*), and the demuxer (modules/demux/*).

[vocm88]

Hi Sebastien,

Thanks for the response! I am investigating h264.c and packetizer_Helper.h and have placed debug messages to show up in VLC logs when I connect to video. My findings show that the function packetizer_Packetize is called most of the times twice in a row. Is packetization done on a per frame basis and if so does the packetizer_Packetize function need only be called once per frame?

Again, I thank the forum for it's help.
Kind regards,
Voc.

[vocm88]

On further investigation, could the delay have anything to do with the video_output.c code, regarding when a frame is scheduled to be displayed?