format callbacks not working with video callbacks

[mirswith]

I am writing a little app to render videos to an OpenGL texture, which is working but is quite slow. Currently I am using libvlc_video_set_callbacks and giving my lock callback a block of system memory that I then copy to an OpenGL texture in my render loop anytime the video has updated the system memory. I am currently setting the format to RV24 as this is a direct copy into my OpenGL texture.

I am looking for ways to improve the performance, it would be nice if I could tell libVLC to just render to an OpenGL texture and let me access it for rendering; this would be my first choice but I do not think/know if that is possible. Short of that I would like to have libVLC give me the decoded data in I422 and do the YUV conversion in a shader. The problem I am having with this is that if I set format callbacks using libvlc_video_set_format_callbacks then my callbacks set with libvlc_video_set_callbacks never get called. Looking at the docs http://www.videolan.org/developers/vlc/ ... 021d714bf0 it says to

use libvlc_video_set_format or libvlc_video_set_format_callbacks to configure the decoded format.

Is this a bug or do I have do something else to make these work together?

Apart from that are there any ideas how else I might speed things up?

Thanks!

[mirswith]

Ok, a stupid oversight on my part, I was returning zero from my format setup callback. I'm getting the lock/unlock callbacks now. Next question is proper setup for the setup, the incoming chroma is I420 which is what I want but pitch and lines are both zero. I'm trying to set them like so:

Code: Select all

pitches[0] = *width; pitches[1] = *width / 2; pitches[2] = *width / 2; lines[0] = *height; lines[1] = *height / 2; lines[2] = *height / 2;

Is this correct?

Thanks.

[mirswith]

To answer my own question, yes that was correct. I now have I420 being converted through a shader, performance already seems much improved. Are there any other ways to improve performance of getting video into an OpenGL texture?

Thanks!

[Rémi Denis-Courmont]

Don't change the resolution in the callback. Don't change the chroma (but that obviously creates many different cases, I420 being only the most common).

[mirswith]

I'm not familiar with the plugin system but would this be one route to get the video into an OpenGL texture without doing the additional copy into system memory and then into OpenGL? Right now the trail looks something like this:

(libVLC) -> (lock -> copies to my system memory) -> (unlock -> copies to my gl texture)

It would be nice for it to look like this:

(libVLC) -> (gl texture)

Maybe libVLC would have to do the additional system memory copy anyway though; i'm not sure.

Thanks for the tips, I am not changing resolution; but I am changing chroma (for now; however so far I've only been seeing I420 in my test cases).

[Jean-Baptiste Kempf]

Sorry to ask, but would you have a libVLC sample code drawing to OpenGL that you could openly share?

[Rémi Denis-Courmont]

It boils down to the lock and unlock callback being set up such that VLC writes straight into a memory mapping of the OpenGL texture. In other words, It is your application that needs fixing, not LibVLC.

[mirswith]

Sorry to ask, but would you have a libVLC sample code drawing to OpenGL that you could openly share?

I'll see what I can do, libVLC is giving me so much it would be nice to contribute if I can.

[mirswith]

It boils down to the lock and unlock callback being set up such that VLC writes straight into a memory mapping of the OpenGL texture. In other words, It is your application that needs fixing, not LibVLC.

I've been using OpenGL ES2 which does not have the needed functionality to do this, however that is a good point I am only using libVLC for the desktop versions; i'll have to explore the wider functionality within OpenGL and see if there is a better approach.

[Jean-Baptiste Kempf]

Sorry to ask, but would you have a libVLC sample code drawing to OpenGL that you could openly share?

I'll see what I can do, libVLC is giving me so much it would be nice to contribute if I can.

Would be great.