[SOLVED] Custom input (imem?)

[Arkaid]

Hello!

I'm in need of providing a custom stream of data to VLC for it to decode and play. If you must ask, I'm writing my own network protocol, to stream a VP8/webm file using UDP, instead of HTTP / TCP. The idea is that my client opens the UDP stream and pumps it into libVLC for decoding. (the objective is to reduce latency)

Apparently imem is the way to go, but there is little to no information about how to make it work. Some tips would help. Also, if you can think of some other way, I'm all open to suggestions!

Thanks in advance.

[ivoire]

It depends on how you want to distribute this work.
You can either use libvlc and distribute your application that work on top of libvlc or you can create an access module that will do this trick and distribute as a module of vlc.

[Arkaid]

Thanks for the answer.

I'm thinking to do this as a separate application running with libVLC, rather than an access module. The reason is because I need to send input data (mouse movements, keypresses) back to the server.

[ivoire]

So if I understand correctly, your application will give the packet that it received using UDP directly without any demuxing or anything ?

In this case you must use imem as an access plugin that will act like a wrapper around open/read/close calls.
For this you should create an item with imem:// as a protocol.
Moreover when creating libvlc you should give the righ arguments (imem-get and imeme-release).
At the moment, their is no API inside libvlc for imem so you should use the command line arguments. I hope that we can add a small API for imem soon.

[Arkaid]

Yes! Thank you! That's exactly what I did. (Sorry, read the reply a bit too late).

The API would be wonderful, but I'd just be happy with better documentation on how to use the stuff that's already there. I mostly found out how to work imem (what each argument of the release and get functions mean) by going to the source code and looking at where the callbacks are called and what the arguments do.

It's a bit cryptic. So here it goes for further reference if someone bumps into the same problem:

If you want to use the imem access module, you first have to create a VLC instance, by using the libvlc_new function. For arguments, you can use whatever you want, but you must add the imem aruments, namely:

1) --imem-get and --imem-release. These callback functions get called by vlc ONCE PER FRAME to allocate and release information about each frame of the video. In other words, before rendering a new video frame, these functions will get called so you can give VLC the frame data. Each argument must be given the memory address of your callback function as a long integer: You'd do something like this:

Code: Select all

sprintf(arg[0], "--imem-get=%ld", myImemGetCallback); sprintf(arg[1], "--imem-release=%ld", myImemReleaseCallback);

where arg is an array of char* you need to pass to libvlc_new, and myImemGetCallback and myImemReleaseCallback are the two callback functions. The prototype of the two functions are as follows

Code: Select all

int myImemGetCallback (void *data, const char *cookie, int64_t *dts, int64_t *pts, unsigned *flags, size_t * bufferSize, void ** buffer);

The get function gets called when VLC needs frame (or audio) information. Each argument is as follows (in: indicates an inbound argument, out: a value you should set as a return):

[in]void *data is a pointer to user-defined data. It can be anything you want. You set it by adding the "--imem-data=<memory_address>" argument when initializing VLC. You set it up in the same manner as the callbacks.

[in]const char *cookie is a user-defined string. Works in the same way as data, but for strings. You set it by adding the "--imem-cookie=<your_string>" argument. your_string can be anything you want. It can be used to identify what instance is calling your callback, should you have multiple VLC instances.

[out]int64_t *dts and int64_t *pts are the Decode Timestamp and Presentation Timestamp values in microseconds, respectively. Usually the same value, but depending on the codec they might be different. In simple words, this tells VLC when to show this frame.
This can be a bit confusing, so to to make it a bit more intuitive, in a 30 fps video, each frame is played every 33 milliseconds or so. so the dts and pts for each consecutive call would be 0, 33333, 66666, 99999, 133332 and so forth. In my case I just took'em out of the rtp stream. (rtp streams come with a similar timestamp on a clock of 90kHz)

[out]unsigned *flags: unused, as far as I'm aware. You can ignore this.

[out]size_t * bufferSize: here you should return the size of buffer, in bytes

[out]void ** buffer: a pointer where the video frame data, encoded with your preferred codec is in memory.

After this, your release function will be called

Code: Select all

int myImemReleaseCallback (void *data, const char *cookie, size_t bufferSize, void * buffer);

void *data and const char *cookie are exactly the same as above

[in]size_t bufferSize is the size of buffer, in bytes

[in]void * buffer is the buffer you must deallocate. Basically you release the memory allocated for the video frame in your "get" function. bufferSize and buffer should be the same you passed to the get function.

2) Setting the codec. For this you use "--imem-codec=<four_letter>", where four_letter is the four letter code for your codec. In my case, I used VP80. You also have to tell vlc that is this a video stream, so you have to add the "--imem-cat=2", where 2 stands for video (there is another one for audio, but it skips my mind right now).

Once you have all your arguments set, create the instance, instantiate a media player with the location (libvlc_media_new_location) set to "imem://" and set it to "play". Your callback function should be called almost immediately and then you can pass your video data from memory to VLC.

[johnthesoftware]

I managed to get imem working using this information except that I had to add --imem-caching=3000 to get it to work.

Unfortunately, at the time of writing I was unable to get transcoding to work properly; it wouldn't start until the player was released, not much use for my programme which was intended to stream real-time video from 40Hz RV24 frames in memory.

Thanks to Arkaid for the help with imem though.

John

[aradeonas]

Hi,
can any body help me with a simple example?I read answer and other articles about imem but I cant make it work.
Any language is good for me.

[SmileyMerx]

Hi everyone,

I am trying to get the stuff with imem done, but i always get the error "access_imem access error: Invalid get/release function pointers"

Can anyone help? A small example would be great too!

Best regards,
Merx