Video stutter/jitter with libVLC on TVOS

[gerlich]

I've been trying to use libVLC 3.0.0 with my TVOS application but encountering performance issues.
Every couple of seconds, the video stutters/jitters accompanied by "picture is too late to be displayed" line in the log.

I've been using the precompiled version from cocoapods (tried all builds) and even compiled from source, but all have these errors.

Here is the startup options I'm using (collected from various posts in this forum):
"--no-color",
"--no-osd",
"--no-video-title-show",
"--no-stats",
"--no-snapshot-preview",
"--avcodec-fast",
"--avcodec-hw",
"--text-renderer=freetype",
"--avi-index=3",
":dash-buffersize", "100",
"--network-cache=3000",
"--subsdec-encoding=ISO-8859-8"

Please help me. Do I need some other options? Did I compile the library wrong?

Here is a complete output of log: http://pastebin.com/APKRusdb

[ssbmaccom]

What kind of media you want to play back?
What is the source of the media?
Pure libvlc or via VLCKit?

At least "--avcodec-fast" and "--avcode-hw" are somehow contradicting. For tvOS (ARM64) you should drop "--avcodec-fast".
I surrounded this option with #ifndef __LP64__ ... #endif like this is done in the VLCKit default options.
Last not least - maybe try with VLCKit (if not already) and with nigthlies.
Dealing with the threading schemes is a bit painful and VLCKit makes this a bit easier - even though you still need to care, what shall not be called from main thread.

[gerlich]

I've tried with various external (i.e. from an external URL) streams: mp4, mkv, etc. All have this problem.
Sorry if my post wasn't clear, but I'm using TVVLCKIT. Even tried the latest build 24.
As per your suggestion I also dropped the avcodec-fast option, but jittering remains the same.

BTW, VLC app on appleTV shows does not have this problem with the same streams.

[ssbmaccom]

OK, TVVLCKit as well.
Build 24? - no clue... is it a pod version number?

I started to build VLCKit myself as I already added patches to it. The latest was not yet integrated, which is enforcing bitcode emission from clang. The nigthlies had issues there. I also inspected other things, like the deinterlace issue by some testing in libvlc myself and I was able to submit a bug report indicating, where something is going wrong - which makes it more likely, that it gets fixed.

Unfortunately I am not able to build VLCKit for a few weeks now. There has been no commit to the repository since Dec. 7th and no nightly since Dec. 3rd (including VLCKit for iOS and tvOS and VLC pre-3.0.0 for macOS).
So far I was not able to fix the build issues - even I tried, but this could be caused by 3rd party contributions as well - and well, the build is taking a while and it is hard to find the cause for the build problems. But I retry again and again - and keep the latest working version in a special trunk

No clue when this will be fixed - usually a failing nightly should trigger the alarms - we, in the office, have a traffic light installed that is green, when the nightly was built without issues and all automated tests were passed.

Back to the streaming issue. You surely use libc++ as recommended instead of libstdc++. The latter was causing performance issues and higher memory consumption, even leaks and crashes.
So maybe watch CPU and Memory usage in Xcode and also make use of Instruments to indicate potential bottlenecks.
Also check for deinterlacer settings. Due to a bug I reported a few months ago, libVLC is not reporting frame rate denominator correctly (usually 1 for interlaced streams and 2 for progressive streams). In my case PAL 576i streams are reported correctly, but PAL 720p are reported with 100 / 1 fps, but should be 50.
As side effect libVLC thinks, that a progressive stream with 50 fps is a interlaced stream with 100 fps - and activates the deinterlacer (blend by default) You can imagine, that this really causes performance penalty. So check, if you are using a progressive stream and if yes deactivate the deinterlacer.
On the official VLC for tvOS release this bug is not present, so it deactivates deinterlacing - but the preview of VLC 3.0 (on Mac) comes with this bug and you see high CPU and RAM usage while playback (no stutter/jitter because my Mac has enough performance to deal with this).

Let's hope, the issues get fixed soon, esp. the build issues.

[gerlich]

Thanks for your help.
I see in the VLC log that deinterlace is not used:
"deinterlace -1, mode blend, is_needed 0"
So that is not the problem.

I have actually been able to compile the latest nightlies. Except for the simulator that is. For simulator I'm encountering this problem: https://code.videolan.org/videolan/VLCKit/issues/56
If I remove the simulator option (i.e. only build for device), the build succeeds.

[ssbmaccom]

In my case I can't build libvpx contrib for Simulator. I manually disabled vpx in my local sandbox.
But this is unrelated to the fribidi issue, so we may have two issues with building.

[gerlich]

What about my original question? Any more insight on this?
I've literally tried everything...

[gerlich]

Update: I made some progress and added the "--no-drop-late-frames" parameter to the library options.
Now, there is no jittering (which I now know are dropped frames), but the video is often distorted.
The fact that the official VLC app does not have these problems suggests that the AppleTV is more than powerful enough to properly decode the stream at runtime.
So perhaps I'm missing something?
I'm not too knowledgable in regards to various decoders/encoders and there are a lot of options to pass to VLC. Perhaps someone can point me in the right direction.

[ssbmaccom]

As mentioned I rarely have such issues and aTV is usually powerful enough. It just has hickups with some deinterlacers esp. when used with progressive streams.
Maybe give it a try and enable Debug Logging in VLCKit - it is a property of VLCMediaPlayer.

I only see drop frames there when other GUI elements are displayed above the full screen video playback view - especially, when the come with transparency.

But check the debug log, maybe share it.

[gerlich]

I did share the debug log. See 1st post