sdl_opengl_player on Windows runtime error and not uses decoding acceleration

[leoncinip]

I've managed to compile sdl_opengl_player.cpp for Windows using Visual Studio 19 and vlc 4.0. For using the runtime infrastructure of vlc I moved and run the resulting executable into the vlc-4.0.0-dev nightly snapshot (until the one of May 10).

With the original code's libvlc_video_engine_opengl set in libvlc_video_set_output_callbacks, I get a runtime error:

Assertion failed: render_cfg.opengl_format == GL_RGBA, file /builds/videolan/vlc/extras/package/win32/../../../modules/video_output/vgl.c, line 96

Instead, by changing libvlc_video_engine_opengl into libvlc_video_engine_disable (maybe it tells the renderer to choose the video output), an OpenGL rendering window opens up, and the video gets rendered (and audio is audible too). Yet, the playing suffers of video frames dropping, thus by looking at the system performances/GPU decode graph, I understand that no hw decoder is being used. Lastly, by reading the debug output, I noticed this: gl gl error: Could not create interop (both on Intel UHD 620 and Nvidia 1070 graphics hw).

Thus, my questions are:
- is there a way to work around the stated runtime error, so to directly use the asked OpenGL video output device?
- how to enable hw (GPU) based video decoding?
- by creating OpenGL interop hopefully even increases the video frame to texture transfer performances, how to do that?

The purpose of my interest in a OpenGL-based libvlc example is to get 360° frames off 360° videos, and reproject to a CAVE walls. I wrote an OpenGL reprojection code, which presently works in a ffmpeg-based prototype, yet it's based on pipe communication, thus it suffers performance issues with video > 4K, so I was searching for a decoding-rendering integrated solution.

[leoncinip]

Update:
when setting libvlc_video_engine_disable - the only video output engine the program goes on with - the OpenGL window opened after libvlc_media_player_play seems not be the main one (in the example called "test"), and, maybe, it gets opened by the player thread aside the main one, which is only set to black by the glClear. Thus, the latter is apparently not affected by the OpenGL drawing commands of the main thread, nor by the texture update.

[mfkl]

Instead, by changing libvlc_video_engine_opengl into libvlc_video_engine_disable (maybe it tells the renderer to choose the video output)

By using disable, none of the callbacks set with libvlc_video_set_output_callbacks will be called. It effectively disables using that feature. And VLC creates a Windows when it doesn't have one and you start playback, so that's what you're seeing.

- is there a way to work around the stated runtime error, so to directly use the asked OpenGL video output device?

Probably the bug is in LibVLC, so that's what needs fixing.

- how to enable hw (GPU) based video decoding?

If you are using libvlc_video_set_output_callbacks, it is enabled by default.

- by creating OpenGL interop hopefully even increases the video frame to texture transfer performances, how to do that?

I didn't understand the question.

[leoncinip]

Instead, by changing libvlc_video_engine_opengl into libvlc_video_engine_disable (maybe it tells the renderer to choose the video output)

By using disable, none of the callbacks set with libvlc_video_set_output_callbacks will be called. It effectively disables using that feature. And VLC creates a Windows when it doesn't have one and you start playback, so that's what you're seeing.

Thank you mfkl, that fully explain what I observed (and reported in the Update reply).

- is there a way to work around the stated runtime error, so to directly use the asked OpenGL video output device?

Probably the bug is in LibVLC, so that's what needs fixing.

I haven't setup the requested toolchain to compile video_output/vgl.c and related video output drivers, so I can't help in the fixing. But if I could use Visual Studio to this purpose (in the end they are plugin dlls...) I'd be glad to contribute - assumed nobody has ever been able to run the sdl_opengl_player sample program successfully in libvlc 4.0.
I'm new to the libvlc community, how do you suggest to proceed? is there a reference person for the OpenGL video output plugins?

- by creating OpenGL interop hopefully even increases the video frame to texture transfer performances, how to do that?

I didn't understand the question.

Oh, this is a step further and maybe is it premature to be treated until the libvlc_video_set_output_callbacks pathway gets fully operative.

Thanks again for the support.

[mfkl]

But if I could use Visual Studio to this purpose

VS isn't officially supported. VS Code is your closest best bet.

I'm new to the libvlc community, how do you suggest to proceed? is there a reference person for the OpenGL video output plugins?

No official reference person no. You can join the libvlc discord server for more pointers https://discord.gg/3h3K3JF