Getting audio from LibVLC

[SilentSin]

I'm using ffmpeg to stream video and audio from a WebCam over a network then using a modified version of Hyper VLC (a C# wrapper around LibVLC) to get it into Unity. After a bit of work I've managed to get the video working well, but HyperVLC doesn't handle audio so I'm trying to add it myself and I can't figure out how to get the audio data out of LibVLC.

I initialise the audio similar to the video using libvlc_audio_set_format and libvlc_audio_set_callbacks, giving it OnPlayAudio(void* data, void* samples, uint count, long pts) as the play callback. But I can't figure out how to use the parameters that method gives me. "data" is always null, and I've tried marshalling "samples" into a float array, but it only gives exponentially small values and NaNs.

I need to get the audio as a float array (with all values between 0 and 1 I believe) to give to Unity. Any help would be greatly appreciated.

[Rémi Denis-Courmont]

I believe this is documented in the LibVLC Doxygen. IIRC, <data> is your opaque data pointer, which you set in the format callback. I'm not sure floating point is implemented yet there (though VLC supports it internally).

[SilentSin]

Thanks for your response.

Is this what you mean by the LibVLC Doxygen? Or is there a different section I should be looking at?

Can you please explain or point me to somewhere that explains what an opaque data pointer actually is?

If it doesn't come in as floats, should I marshal it as bytes or ints then scale them down into the 0-1 range floats Unity needs?

[Rémi Denis-Courmont]

Gee, the opaque data is what you make it. LibVLC does not know or care what it is.

[SilentSin]

That doesn't make sense. Either the parameter is completely useless or it is actually used in some way. If I pass in a pointer to a float[], it can't possibly be used in the same way as if I were to pass in a pointer to a single int or a string or something else entirely.

[Jean-Baptiste Kempf]

opaque is used for your usage, so you get it in the callback.