Using libVLC ignores disabled hardware acceleration

[doubleaa93]

Are you passing any args when starting libvlc? I keep getting

Code: Select all

main generic debug: looking for opengl module matching "vgl": 2 candidates main generic debug: using opengl module "vgl" main generic debug: no vout display modules matched main video output error: video output creation failed

[sherington]

No, I don't pass any options when getting my LibVLC instance. I don't really do anything other than creating the OpenGL application and setting the video callbacks.

All I can suggest is that you make sure your VLC 4.x build is definitely the one you're using, maybe explicitly set the VLC_PLUGIN_PATH env variable. Check you have other working OpenGL apps I suppose.

Honestly I don't really do anything special. My test app is basically the bare minimum OpenGL demo app and the new video callbacks, that's it. The video callback code is deeply integrated into my Java library so I can't really post a useful isolated code fragment, sorry.

[mfkl]

Is this feature ready or still in development?

Still in development.

[Rémi Denis-Courmont]

Honestly I don't really do anything special. My test app is basically the bare minimum OpenGL demo app and the new video callbacks, that's it. The video callback code is deeply integrated into my Java library so I can't really post a useful isolated code fragment, sorry.

Maybe different platforms? It seems doubleaa93 uses Windows.

[sherington]

Honestly I don't really do anything special. My test app is basically the bare minimum OpenGL demo app and the new video callbacks, that's it. The video callback code is deeply integrated into my Java library so I can't really post a useful isolated code fragment, sorry.

Maybe different platforms? It seems doubleaa93 uses Windows.

Ah!

You have this uncanny knack of hitting the nail on the head!

I usually assume everyone sensible uses Linux.

[MikOfClassX]

Attaching callbacks will turn off any hw accell IMHO. That's a pity. Really.
I am asking myself if grabbing decoded frames from hw surfaces may be that difficult.

BTW Any success with LWJGL ?

[sherington]

BTW Any success with LWJGL ?

If you're asking me, then "yes"!

I use my own build of vlc-git, and I have those new video callbacks working very nicely in an LWJGL application (on Linux). I was actually gob-smacked that it worked so easily, I knew next to nothing about OpenGL when I built this application.

On the other hand I'm not sure LWJGL, or some other OpenGL library, is an optimal solution for my own projects - the issue for me is developing the rest of the application with an easy-to-use UI framework, decent widgets and what-not, stuff more suited to a traditional desktop application than a game. I don't have enough experience with OpenGL frameworks to know what's best.

[MikOfClassX]

Any luck with latest builds ?

[Jean-Baptiste Kempf]

please share your logs.

[MikOfClassX]

can you share a snippet sherington ?

[sherington]

@MikOfClassX What code snippet did you want?

[unidan]

MikOfClassX: Did you debug it with an openGL debugger ?

[MikOfClassX]

Phew, I am back..

@sherington, I am confused:
- seeting callbacks will disable hw accell decoding, do you agree ?
- using GL callbacks will also disable hw accell decoding, right ?
- are you using GL callbacks to minimize CPU-GPU memory copy ?

definitely a snippet of the callback will ease my mind

[sherington]

Honestly, I'm far from being knowledgeable on OpenGL...

Anyway, I said before my implementation is deeply integrated into library code so it's a bit difficult to extract a code snippet. The best I can provide here is some pseudo code showing how I initialise the callbacks and link them to the LWJGL code:

Code: Select all

// LibVLC call to initialise the callbacks, parameters 3-8 are function pointers to the callback implementations. libvlc_video_set_output_callbacks(mediaPlayer, libvlc_video_engine_opengl, setup, cleanup, updateOutput, swap, makeCurrent, getProcAddress, null); // I render video directly to a Window glWindow = glfwCreateWindow(800, 600, "OpenGL callback rendering", NULL, NULL); // These are the callback implementations that invoke LWJGL in my case (or some other OGL library): boolean setup(void* opaque) { return true; } void cleanup(void* opaque) { } long onGetProcAddress(void* opaque, String functionName) { return glfwGetProcAddress(functionName); } boolean onMakeCurrent(void* opaque, boolean enter) { glfwMakeContextCurrent(glWindow); } void onSwap(void* opaque) { glfwSwapBuffers(glWindow); } void onUpdateOutput(void* opaque, int width, int height) { glfwSetWindowSize(glWindow, width, height); glfwSetWindowAspectRatio(glWindow, width, height); }

Now, of course there's a bit more code needed for the OpenGL application itself, but the code above is essentially all that I needed to provide to make the bridge between LibVLC and my LWJGL implementation.

I have no idea what this means for your hardware decoding question but I can tell you that CPU usage with VLC itself is about 11% compared with about 20% when using this implementation to play the same video.

[MikOfClassX]

Ok, got it. I miss the texture update code, i.e. the way you get/draw the pixels.

[unidan]

Hi,

- seeting callbacks will disable hw accell decoding, do you agree ?

Setting vmem callbacks don't break hardware decoding, but you might have copy from GPU memory to RAM.

- using GL callbacks will also disable hw accell decoding, right ?

- using GL callbacks change nothing to hw accel decoding as long as the interop is available, which is the case for almost everything (except computer like raspberry maybe).

[MikOfClassX]

Hi @unidan,

you say:

but you might have copy from GPU memory to RAM.

Color conversion and copy is something that happens inside libVLC after you set the video format callback (hw decoding gone away).
So, am I (probably) missing something ?

[unidan]

Hi, you're right.

In this case, you can consider color conversion and this kind of copy as the same thing. It's just a conversion from a GPU chroma to a CPU chroma. But it still happen in libVLC whereas you can avoid that with either CPU decoding or GPU callbacks. When I'm talking about copies, I think about the whole decoding+rendering pipeline and either in CPU or back and forth CPU and GPU memories.