LibVLC and OpenCV

[oscargot]

Hi all,

Im trying to use libVLC to retrieve an online video stream and display frames as OpenCV IplImages.
This is what I have so far, and for some reason it seems to work sometimes but most of the time it just displays nothing but the audio is still playing.

Code: Select all

struct ctx { IplImage* image; HANDLE mutex; uchar* pixels; }; void *lock(void *data, void**p_pixels) { struct ctx *ctx = (struct ctx*)data; WaitForSingleObject(ctx->mutex, INFINITE); *p_pixels = ctx->pixels; return NULL; } void display(void *data, void *id){ (void) data; assert(id == NULL); } void unlock(void *data, void *id, void *const *p_pixels){ struct ctx *ctx = (struct ctx*)data; /* VLC just rendered the video, but we can also render stuff */ uchar *pixels = (uchar*)*p_pixels; cvShowImage("image", ctx->image); ReleaseMutex(ctx->mutex); assert(id == NULL); /* picture identifier, not needed here */ } int main() { cvNamedWindow("image", CV_WINDOW_AUTOSIZE); libvlc_media_t* media = NULL; libvlc_media_player_t* mediaPlayer = NULL; char const* vlc_argv[] = {"--plugin-path", "C:\\Users\\Oscar\\Documents\\libvlc\\vlc-1.1.4"}; libvlc_instance_t* instance = libvlc_new(2,vlc_argv); mediaPlayer = libvlc_media_player_new(instance); media = libvlc_media_new_path(instance, "mms://81.89.49.210/musicbox"); struct ctx* context = ( struct ctx* )malloc( sizeof( *context ) ); context->mutex = CreateMutex(NULL, FALSE,NULL); context->image = cvCreateImage(cvSize(VIDEO_WIDTH, VIDEO_HEIGHT), IPL_DEPTH_8U, 4); context->pixels = (unsigned char *)context->image->imageData; libvlc_media_player_set_media( mediaPlayer, media); libvlc_video_set_callbacks(mediaPlayer, lock, unlock, display, context); libvlc_video_set_format(mediaPlayer, "RV32", VIDEO_WIDTH, VIDEO_HEIGHT, VIDEO_WIDTH*4); libvlc_media_player_play(mediaPlayer); while(1) { } return 0; }

I'm not sure what's wrong, I've spent days learning about libvlc, linking it into Visual Studio, trying to figure out how to use the libvlc_video_set_callback functions.

Sorry if the code is messy...I'm just trying to get it work before I clean it up a bit..

Thanks!

[oscargot]

After some fiddling around, I realized that if I put some irrelevant code (some random image processing code) into the while loop, I was able to display the frames of the vmem output. Would anyone know the cause for this?? Thanks again.

[brezn]

Hi oscargot,

first of all I want to thank you for your good example. It works for me right away.

Now to your problem:
Since you did the most common error in OpenCV rendering ( to forget about the cvWaitKey( )-statement a solution is easy:
if you change your loop to:

Code: Select all

while( 1 ) { cvWaitKey( 10 ); };

it will work.

See also here:
http://www.cs.indiana.edu/cgi-pub/oleyk ... _cvWaitKey
Best, brezn

[alexanderpopov]

It's an old post but thanks to published code i was able to manage working with OpenCv2 & LibVlc in x64 :
I post updated code for other folks.

Code: Select all

using namespace cv; struct ctx { Mat* image; HANDLE mutex; uchar* pixels; }; // define output video resolution #define VIDEO_WIDTH 1920 #define VIDEO_HEIGHT 1080 void *lock(void *data, void**p_pixels) { struct ctx *ctx = (struct ctx*)data; WaitForSingleObject(ctx->mutex, INFINITE); // pixel will be stored on image pixel space *p_pixels = ctx->pixels; return NULL; } void display(void *data, void *id){ (void) data; assert(id == NULL); } void unlock(void *data, void *id, void *const *p_pixels){ // get back data structure struct ctx *ctx = (struct ctx*)data; /* VLC just rendered the video, but we can also render stuff */ // show rendered image imshow("test", *ctx->image); ReleaseMutex(ctx->mutex); } int main() { // VLC pointers libvlc_instance_t *vlcInstance; libvlc_media_player_t *mp; libvlc_media_t *media; const char * const vlc_args[] = { "-I", "dummy", // Don't use any interface "--ignore-config", // Don't use VLC's config "--extraintf=logger", // Log anything "--verbose=2", // Be much more verbose then normal for debugging purpose }; vlcInstance = libvlc_new(sizeof(vlc_args) / sizeof(vlc_args[0]), vlc_args); // Read a distant video stream //media = libvlc_media_new_location(vlcInstance, "rtsp://192.168.0.218:554/Streaming/Channels/1"); // Read a local video file media = libvlc_media_new_path(vlcInstance, "d:\\a.mp4"); mp = libvlc_media_player_new_from_media(media); libvlc_media_release(media); struct ctx* context = ( struct ctx* )malloc( sizeof( *context ) ); context->mutex = CreateMutex(NULL, FALSE, NULL); context->image = new Mat(VIDEO_HEIGHT, VIDEO_WIDTH, CV_8UC3); context->pixels = (unsigned char *)context->image->data; // show blank image imshow("test", *context->image); libvlc_video_set_callbacks(mp, lock, unlock, display, context); libvlc_video_set_format(mp, "RV24", VIDEO_WIDTH, VIDEO_HEIGHT, VIDEO_WIDTH * 24 / 8); // pitch = width * BitsPerPixel / 8 int ii = 0; int key = 0; while(key != 27) { ii++; if (ii > 5) { libvlc_media_player_play(mp); } float fps = libvlc_media_player_get_fps(mp); printf("fps:%f\r\n",fps); key = waitKey(100); // wait 100ms for Esc key } libvlc_media_player_stop(mp); return 0; }

Thanks to libvlc team.

[kkirtacc]

Hello alexander, thanks for the latest post.
Please consider me as a beginner.
I already have a stable OpenCV setup on Windows 7. I am trying to obtain a video stream, process frame by frame and output each resulting frame.
How can I use the following code? Do I need to compile vlc source code from scratch? Or, is it enough to copy and paste libvlc.dll to the OpenCV bin folder?
Thanks.

[joecmama]

Hi kkirtac,

Not sure if you still are searching for an answer, but hopefully this may help others in case not you. I had your same question and it took me a while to get libVLC installed and working since I was new to it as well. The code posted here by others helped a lot, and I've just documented how to get to the point to use them, at least for me. I use MS Visual Studio on Windows 7 and 8.1.

Details are given here: http://answers.opencv.org/question/65932