streamed h264 decoding with small caches

[ipacker]

Hi,

I am streaming h264 and mpeg4-10 to VLC from various off the shelf hardware encoder units via RTSP. This is a latency-sensitive application so I am using --clock-jitter=0 and --rtsp-caching=100 to get lag down. When using h264 from several different units I often get the situation where I am spammed with the following line:

"number of reference frames exceeds max (probably corrupt input), discarding one"

The video also freezes (but doesn't 'stop').

If I increase rtsp-caching to 1000 or so the problem goes away but this is far too slow for my application. With rtsp-caching=100 I am getting aroudn 180ms total lag including encoding and network.

The fact that increasing the buffer fixes the problem suggests that this isn't a corrupt input, and isn't that the encoder is spamming vlc with reference frames?

One of the h264 encoder manufacturers has suggested that the error actually means that not enough previous frames are available to decode the current frame. I don't know the specifics of the h264 specs but I have been told that a frame can use up to 16 previous frames for calculating the current frame. At 25fps this would be about 640ms of previous frames.

Is this likely to be what is really happening or is this path a red-herring and the error actually points to something else?

Other things worth knowing are:

1) I don't get this problem with mpeg4
2) Certain hardware encoders never cause VLC to exibit this problem at all no matter what I do - others seem to trigger it a lot. The actual video the encoder is encoding seems to make a difference but it's not very predictable.
3) Other software players, such as ones specifically paired to the encoders don't have a problem with this (and can also produce total latencies of around 80-100ms smoothly!). They generally their own other little problems though.

P.S Since 1.1.0 I am now also testing with hardware acceleration using a 9300GE (VP3) and a GT240 (VP4) and although this particular problem still exists generally the hardware accerlation seems to be working well with these types of streams in terms of smoothness and cpu usage.

Thanks,
Ian