Opengl video output

This forum is about all development around libVLC.
kolor
New Cone
New Cone
Posts: 8
Joined: 15 May 2014 14:26

Opengl video output

Postby kolor » 03 May 2018 17:08

Hi there,

When setting libvlc to render in an opengl window, is there a way to interact with opengl window created by libvlc ? I mean, set a custom opengl context for the window, or just get the opengl context created by libvlc, get back framebuffer associated to window, ...

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Opengl video output

Postby Jean-Baptiste Kempf » 05 May 2018 00:28

What are you trying to achieve?
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

kolor
New Cone
New Cone
Posts: 8
Joined: 15 May 2014 14:26

Re: Opengl video output

Postby kolor » 14 May 2018 08:56

At the end, I would like to get the opengl texture containing the decoded frame.
Since VLC can use an opengl output I think it is already possible but maybe not exposed ?

FL53
Blank Cone
Blank Cone
Posts: 39
Joined: 09 Mar 2018 17:58

Re: Opengl video output

Postby FL53 » 08 Jun 2018 14:22

I think it would be difficult to exploit the OpenGL texture out of its context. If you want the OpenGL texture you need the OpenGL context too (context created by VLC to draw when using OpenGL). This is not something exposed I think.

I suppose that you need an OpenGL texture to display video on a 3D object (or fullscreen quad ?) in your OpenGL application. If so, working from the SDL example (https://wiki.videolan.org/LibVLC_SampleCode_SDL/) seems to be a good idea : you will have to fill your texture each time a new frame has been fully received (after lock/unlock callbacks). You can look at PBO (http://www.songho.ca/opengl/gl_pbo.html) to create your own "double buffer" rendering process to get video data asap in your texture.

kolor
New Cone
New Cone
Posts: 8
Joined: 15 May 2014 14:26

Re: Opengl video output

Postby kolor » 08 Jun 2018 15:59

Thank you for the links FL53. Indeed I already know the callbacks lock/unlock but it is not the most efficient, I mean there are copies between gpu and cpu. I would like to have a zero copy pipeline, in other words stay on gpu in the whole pipeline.

FL53
Blank Cone
Blank Cone
Posts: 39
Joined: 09 Mar 2018 17:58

Re: Opengl video output

Postby FL53 » 17 Jun 2018 19:20

I had the same concern about reading video in OpenGL texture but without modifying the libVLC sources I think this is not something easy to do. I used as basis the SDL tutorial and resulting performances are pretty good (but I only load a single video and apply it as texture with a simple shader on a mesh).

Nonetheless, I agree with you : avoiding copy (from VRAM to RAM then RAM to VRAM) would be great. Does someone involved in libVLC development have a trick to do that kind of thing ? (I'm quite interested too by the answer :) )

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Opengl video output

Postby Jean-Baptiste Kempf » 17 Jun 2018 22:24

Thank you for the links FL53. Indeed I already know the callbacks lock/unlock but it is not the most efficient, I mean there are copies between gpu and cpu. I would like to have a zero copy pipeline, in other words stay on gpu in the whole pipeline.

You need a patch to libVLC to do that.
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

neosettler
Cone that earned his stripes
Cone that earned his stripes
Posts: 107
Joined: 18 Dec 2012 17:44

Re: Opengl video output

Postby neosettler » 06 Jul 2019 16:08

Hello Jean-Baptiste,

Code: Select all

You need a patch to libVLC to do that.
Is this something Libvlc user could expect in the future? With the amount of data required for custom 360 stereoscopic videos, copying the texture is killing performance to a point where vlclib becomes useless. I hope you've been giving this feature a serious though.

sherington
Cone that earned his stripes
Cone that earned his stripes
Posts: 491
Joined: 10 Sep 2008 11:57
VLC version: master
Operating System: Linux

Re: Opengl video output

Postby sherington » 07 Jul 2019 09:24

VLC/LibVLC 4.x, currently in development, has new callbacks for OpenGL.

Code: Select all

LIBVLC_API bool libvlc_video_set_output_callbacks( libvlc_media_player_t *mp, libvlc_video_engine_t engine, libvlc_video_setup_cb setup_cb, libvlc_video_cleanup_cb cleanup_cb, libvlc_video_update_output_cb update_output_cb, libvlc_video_swap_cb swap_cb, libvlc_video_makeCurrent_cb makeCurrent_cb, libvlc_video_getProcAddress_cb getProcAddress_cb, void* opaque );
I have used this with LWJGL+Java and it seems to work really well.

neosettler
Cone that earned his stripes
Cone that earned his stripes
Posts: 107
Joined: 18 Dec 2012 17:44

Re: Opengl video output

Postby neosettler » 07 Jul 2019 18:52

Thank you for your suggestion sherington. I got 4.0 and it seems to work well so far. I'll give this function I tried. Is there any example I could chew on?

unidan
Developer
Developer
Posts: 1493
Joined: 25 Mar 2018 01:00

Re: Opengl video output

Postby unidan » 08 Jul 2019 11:08

Hi, check in doc/libvlc/sdl_opengl_player.cpp.

neosettler
Cone that earned his stripes
Cone that earned his stripes
Posts: 107
Joined: 18 Dec 2012 17:44

Re: Opengl video output

Postby neosettler » 10 Jul 2019 02:08

Thank you for your input unidan.

I've got my hands on the latest 4.0 nightly build and the libvlc_video_set_output_callbacks signature is different from the example you've mentioned. I'm on Windows 10 and the behavior seems to be awkward as the cleanup, procAdress and setup callbacks are called every single frame. I'm not quite sure where to go from here.

Also, it might work quite nicely in a one cpp example but chances are that other people like me have their GL context on an other thread, which make this process difficult to implement. In order to move the GL context to the libvlc_video_set_output_callbacks thread, is it possible to access the libvlc_video_set_output_callbacks thread some how?

unidan
Developer
Developer
Posts: 1493
Joined: 25 Mar 2018 01:00

Re: Opengl video output

Postby unidan » 10 Jul 2019 10:49

Hi,
Thank you for your input unidan.

I've got my hands on the latest 4.0 nightly build and the libvlc_video_set_output_callbacks signature is different from the example you've mentioned. I'm on Windows 10 and the behavior seems to be awkward as the cleanup, procAdress and setup callbacks are called every single frame. I'm not quite sure where to go from here.
4.0 and libvlc_video_set_output_callbacks are still experimental and haven't been released; so it might even change again. It was mostly done as a prototype before integrating the same mechanism for d3d11 and vulkan, then it will probably become an unified API for doing this.
Also, it might work quite nicely in a one cpp example but chances are that other people like me have their GL context on an other thread, which make this process difficult to implement. In order to move the GL context to the libvlc_video_set_output_callbacks thread, is it possible to access the libvlc_video_set_output_callbacks thread some how?
You need to provide a context to libvlc. As libvlc output runs into its own thread, you need to dedicate a full context that you don't use in another thread. So you'll likely have two contexts that share resources.
Keep in mind that you're running video, that probably runs with a different framerate than your own application, so you have to clock gate with for example triple buffering, or sync with the video rendering.

onelistme
New Cone
New Cone
Posts: 8
Joined: 26 Dec 2020 14:07

Re: Opengl video output

Postby onelistme » 27 Dec 2020 05:50

Hi, check in doc/libvlc/sdl_opengl_player.cpp.
When run this example, it's founded that the CPU using is so high than normal relaesed player. Is there any more information on Why the CPU using growth so high and the video play is not smooth. There is more info at https://forum.videolan.org/viewtopic.php?f=14&t=155928
would you like comment on it.

onelistme
New Cone
New Cone
Posts: 8
Joined: 26 Dec 2020 14:07

Re: Opengl video output

Postby onelistme » 27 Dec 2020 06:00

Thank you for the links FL53. Indeed I already know the callbacks lock/unlock but it is not the most efficient, I mean there are copies between gpu and cpu. I would like to have a zero copy pipeline, in other words stay on gpu in the whole pipeline.

You need a patch to libVLC to do that.
Hi, is there copies between CPU and GPU in this example doc/libvlc/sdl_opengl_player.cpp? I found CPU consumption is so higher(example 30%) than the normal released player(3%). More info at https://forum.videolan.org/viewtopic.php?f=14&t=155928

unidan
Developer
Developer
Posts: 1493
Joined: 25 Mar 2018 01:00

Re: Opengl video output

Postby unidan » 03 Feb 2021 09:07

Hi,

As mentioned in the other post for the sake of search engine completeness:

This is tracked upstream as https://trac.videolan.org/vlc/ticket/25234. The issue is that hardware decoding interops are not used because platform extensions are not available through the callbacks. We're investigating a way to have them anyway. I'll keep you posted.


Return to “Development around libVLC”

Who is online

Users browsing this forum: No registered users and 12 guests