Help a beginner with libvlc4 and openGL

[davidemania]

hi friends,
I am trying to write an addon for the OpenFrameworks ecosystem, starting with unix, actually Ubuntu 18.04, with a video player based on libvlc (the default one on Linux uses gstreamer). In order to obtain good performance with higher resolutions a direct GPU path is mandatory, and so I'd like to try the libvlc v4 with the openGL callbacks.

Can someone help me starting, a link to a suitable branch with library code would be much appreciated! The repository has lots of different branches and it is not evident which one I should choose. Some examples or code snippets would be very useful too.

Of course I understand that this is still unstable code, but the idea is to start now with some experiments and then switch to the stable v4 library when it will be released.

Thank you
Davide

[unidan]

Hi, on the master branch, you have a sample with SDL in doc/libvlc/sdl_opengl_player.cpp, will it be enough for you as a starter code ?

[davidemania]

good, here is where I got so far, for future reference:

1) cloned the repo from https://github.com/videolan/vlc.git (quite large, ~350MB)
2) sudo apt install flex
sudo apt install bison
4) ./bootstrap
5) sudo apt-get install lua5.2 liblua5.2-dev
sudo apt install liba52-0.7.4-dev
sudo apt install libxcb-xkb-dev
6) ./configure --disable-qt
7) ./make
8) sudo make install
9) added "/usr/local/lib" to /etc/ld.so.conf (at the end)
10) sudo ldconfig

vlc binary is working, --version says "revision 4.0.0-dev-7661-gddf1e6c7d4". It complains about some libva error but this may be because it is running in a VM. Video and audio are correctly decoded.

Now I'll study the example and see what I can do. Just to be sure, at this point to develop with libvlc v4 is it enough to include the headers from the "include/vlc" directory and link against the libvlc.so.12 that has been built from the sources?

[EDIT] Yes it is :)

Thanks for helping
Davide

PS VLC code is impressive, much respect to its developers!

[davidemania]

Well I believe I understood the mechanics behind it, but at the moment there are a few black spots I cannot solve.

One of them is the call to "SDL_GL_GetProcAddress" in the "get_proc_address" callback, I tried hard but could not find how to replace it with a vanilla openGL call (without SDL). It should be glXGetProcAddress, but I can't manage to find out where it is defined in Ubuntu Linux

[unidan]

Hi,

vlc binary is working, --version says "revision 4.0.0-dev-7661-gddf1e6c7d4". It complains about some libva error but this may be because it is running in a VM. Video and audio are correctly decoded.

You're only missing the libva libraries. They are checked at runtime for hardware decoding but aren't mandatory.

Now I'll study the example and see what I can do. Just to be sure, at this point to develop with libvlc v4 is it enough to include the headers from the "include/vlc" directory and link against the libvlc.so.12 that has been built from the sources?

Yes.

One of them is the call to "SDL_GL_GetProcAddress" in the "get_proc_address" callback, I tried hard but could not find how to replace it with a vanilla openGL call (without SDL). It should be glXGetProcAddress, but I can't manage to find out where it is defined in Ubuntu Linux

It depends whether you're relying on EGL or GLX. What did you do to initialize your opengl context ?

[davidemania]

Well thank you for your help, the glXGetProcAddress definition turned out to be in GL/glx.h, so that part should be ok. Now I have to figure out how to share the GL content of my application with libvlc, and how to replace SDL_GL_MakeCurrent calls.

The openFramworks library wraps all openGL in a nice API, but this hides all the inner workings and unfortunately I am not familiar with the matter enough to get this information from source code alone. GL context is initialized internally when creating the window for the main application. I asked on their forum too, let's hope someone knowledgeable will be willing to help.

[unidan]

For MakeCurrent : https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glXMakeCurrent.xml

The callbacks are here to make libvlc share its content and synchronization features with your application, through glFramebuffer (FBO).

Check the following calls in the samples for the libvlc part. it is making use of triple buffering to avoid tearing between your application and libvlc.

Code: Select all

// generate 3 framebuffers and 3 output textures glGenTextures(3, that->m_tex); glGenFramebuffers(3, that->m_fbo); // configure the framebuffers for (int i = 0; i < 3; i++) { glBindTexture(GL_TEXTURE_2D, that->m_tex[i]); glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glBindFramebuffer(GL_FRAMEBUFFER, that->m_fbo[i]); glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, that->m_tex[i], 0); } glBindTexture(GL_TEXTURE_2D, 0); // Check that everything was correct GLenum status = glCheckFramebufferStatus(GL_FRAMEBUFFER); if (status != GL_FRAMEBUFFER_COMPLETE) { return; } that->m_width = width; that->m_height = height; // Prepare for rendering glBindFramebuffer(GL_FRAMEBUFFER, that->m_fbo[that->m_idx_render]); // and in swap static void swap(void* data) { VLCVideo* that = static_cast<VLCVideo*>(data); std::lock_guard<std::mutex> lock(that->m_text_lock); that->m_updated = true; std::swap(that->m_idx_swap, that->m_idx_render); // Bind an new framebuffer for libvlc glBindFramebuffer(GL_FRAMEBUFFER, that->m_fbo[that->m_idx_render]); } // in your application, because of triple buffer, you always have one of the three buffer available for rendering, and you have a texture handle on the video frame

[davidemania]

I feel I am almost there but still can't figure out how to share the GL context between the application and libvlc code. The example uses SDL_GL_SHARE_WITH_CURRENT_CONTEXT that looks handy, but the openFrameworks library internally uses GLFW and even after a lot of research I couldn't find anything equivalent. There are methods to share the GL context among different GLFW windows but nothing (that I am aware of) with some external code.

Some help on this would be very much appreciated... thanks

[unidan]

Unfortunately I've never used GLFW, but GLFW is high level so you should be able to use GLX/EGL/WLG/EAGL shareContext features at least