libVLC audio samples buffer format

[peetonn]

I use libvlc_audio_play_cb callback for delivering audio sample to my code. My libvlc_audio_setup_cb tells me that the format is "S16N" and there are 2 channels.
There is a "count" argument in libvlc_audio_play_cb which says how many samples were delivered.
However, I can't find anywhere in VLC docs information on how samples buffer is formatted - are channels interleaved, i.e. buffer[0] is a left channel sample, buffer[1] is a right channel sample?
I also can't find information on S16N - is it 16bit PCMU native endianness?

Any links for the info would be also appreciated.

[Rémi Denis-Courmont]

It's linear signed 16-bits in native endianess.

[peetonn]

Are samples for different channels interleaved in the buffer I'm getting in the callback? Left channel goes first?

[peetonn]

Alright, I had to figure out everything myself by trial and error which took considerable time. Sadly, available information on this topic is not enough to figure out the problem, neither VLC community is helpful. So, I'll explain my approach that works for me:

When you get call in your libvlc_audio_play_cb callback, you receive few arguments:

- count - number of samples
- samples - pointer to the array of samples
- pts - expected playback time (in microseconds)

This info is available from the VLC doxygen. However, what is not obvious from the doxygen is that:

- samples for left and right channels are interleaved, i.e. samples[0] belongs to left channel, samples[1] belongs to right channel;
- sample size depends on format you get in your libvlc_audio_setup_cb. if it's S16N - your sample size is 2 bytes (this is obvious, but nevertheless);
- if you have more than one channel, "count" does not represent amount of samples for all channels!!!, i.e. in my case I have 2 channels, thus, I have to consider that actual size in bytes of "samples" array is not "count * sizeof(short)" but "2*count * sizeof(short)"! I find it ridiculous that it's not documented!

Here's stripped pseudocode one could implement for allocating audio samples buffer and copying delivered samples into it:

Code: Select all

int handleAudioFormat(void **opaque, char *format, unsigned *rate, unsigned *channels) { rate_ = *rate; channels_ = *channels; memcpy(format_, format, 4); } void audioPlay(void *opaque, const void *samples, unsigned count, int64_t pts) { // account for channels number; if you have 2 channels, number of samples should double count *= channels_; unsigned bufSize = count * sizeof(short); delayUsec_ = libvlc_delay(pts); if (!audioBuffer_ || bufSize > audioBufferSize_) { nAudioSamples_ = count; audioBufferSize_ = bufSize; audioBuffer_ = (short*)realloc(audioBuffer_, audioBufferSize_); } memcpy(audioBuffer_, samples, audioBufferSize_); }

I hope this will be helpful for someone and will save her time and headaches.

[Rémi Denis-Courmont]

1) Interleaved audio is defacto standard, as is the channel order. Planar is quite rare and right-left is unheard of.
2) This is utterly obvious.
3) count represents the number of audio samples, as in the number of sampling periods described by the buffer. That's pretty much by definition. Besides, some formats don't even separate channels, so how would you count samples??

I must disagree with you. And a number of people have used that documentation either without your problems. Typically, people misunderstand when the callbacks are invoked or how to interoperate with C code in their language. Not what the parameters mean.