Audio and Video Custom Callbacks

[cannono]

Hello, I am trying to write a program that streams audio and video from one computer to another. I am using ffmpeg to open and segment the files into video and audio frames. The frames are then sent over a socket to another computer which plays the media file using libvlc. So far I have the video frames displaying properly by using libvlc_media_new_callbacks, but I am unable to find a way to make the audio play as well. Can this be accomplished through the media custom callbacks as well? Or do I have to create custom audio callbacks with libvlc_audio_set_callbacks? Or something entirely different?

[chubinou]

Hi,
libvlc_audio_set_callbacks is for retrieving decoded audio frames (output)
For having both video and audio input, 2 things you might try:
* provide audio and video in a muxed format to libvlc_media_new_callbacks.
* provide audio as a slave input of your video media using libvlc_media_slaves_add

Is there a particular reason you don't use a common streaming protocol which may be directly be opened by VLC?

[Rémi Denis-Courmont]

Those callbacks are meant to feed a full "multiplex" bit stream, not any individual elementary stream. You can definitely send audio through that, but you need to pick a file format so VLC knows what the audio format is. It does not seem to make much sense though.

[cannono]

Thanks for the replies,

So it sounds like I need to mux the video and audio frames and then a specify a demuxer so vlc knows how the data is formatted?

Currently, I am doing: libvlc_media_add_option(m, ":demux=rawvid") to play the raw video data. Can I also specify that the audio will be in a raw format? Or do I have to pick a container such as avi to format the audio and video data in and then tell vlc the mux container is avi?

Also, could you expand on why you think this approach doesn't make much sense. Thanks.

[Rémi Denis-Courmont]

Playing raw audio through LibVLC does not make much sense. You could send your raw audio to the OS audio HAL directly instead of piling layers of reverse abstraction.