Question about VLC's handling of missing packets in a transport stream

[oviano]

Below is an HEVC encoded file that has had 1% of packets (as in 1316 UDP-sized packets of 7 x 188 TS packets) randomly removed from it.

When I play this in VLC the behaviour is for the stream to freeze repeatedly, as if it just gives up trying to display any corrupted frames. Sometimes it freezes for several seconds.

However, when I play this in MPV, it continues at the same framerate, but just produces corruption on the screen. I prefer this approach, as even though there is obviously considerable corruption, I find it more watchable as it never actually stops, and the corruption moves from place to place on the screen meaning it's actually possible to still watch.

Is it possible to configure VLC to behave like MPV in this respect?

https://www.dropbox.com/s/xh67mmie3x9h7zi/video.ts?dl=0

[oviano]

Actually, I figured it out - the solution is to force avformat demux instead of letting it use the VLC mpegts. This replicates the mpv behaviour.

[oviano]

So a general Q for the devs on this, out of curiosity why does VLC have it's own demuxes for certain types, eg mpegts, mp4, rather than use ffmpegs? Is this a historical thing where the support wasn't previously existing (or good enough) in ffmpeg? Or maybe VLC didn't always use ffmpeg?

[Jean-Baptiste Kempf]

Because we support more things and different things. Changing would be a nightmare for users.

[InTheWings]

Below is an HEVC encoded file that has had 1% of packets (as in 1316 UDP-sized packets of 7 x 188 TS packets) randomly removed from it.

When I play this in VLC the behaviour is for the stream to freeze repeatedly, as if it just gives up trying to display any corrupted frames. Sometimes it freezes for several seconds.

I'm working on this

[oviano]

Nice one.

I think there are two issues. The first is as mentioned the demux. The VLC demux does not handle this scenario as well as the avformat demux, dropping frames/freezing rather than just corrupting frames.

Secondly, as I am testing on macOS even if you force VLC to use the avformat demux, the VideoToolbox decoder quickly fails to decode what it is given by the demux and falls back to avcodec.

I don't think this is strictly necessary. I have written a simple ffmpeg/SDL2 based video player (not using libVLC) which uses Videotoolbox via the ffmpeg libs hardware acceleration. I have found that with this file even if a decode session succeeds, i.e. doesn't produce an error, it's possible for it not to produce an CVImageBufferRef in the VideoToolbox API callback. i.e. you get NULL.

With the ffmpeg implementation it uses this condition to abort the decode, as I suspect VLC does too. However, as long as you handle this scenario (i.e. just ignore and continue, and keep displaying the last frame) then things do carry on reasonably well. (I changed the ffmpeg code to stop this being a fatal condition). There seems no need to completely abandon the VT decode session; i.e. you can continue feeding it corrupt data and it recovers.

Maybe there is something similar going on with VLC's Videotoolbox implementation....