I'm using VLC 2.2.0 compiled from git. to stream H264 and signed 24 bit big-endian PCM using RTP Audio/Video profile.
cvlc stream.mkv :sout='#rtp{dst=10.161.156.201,port=6748,ttl=1,sdp="file:///tmp/test.sdp"}' :sout-all :sout-keep -L
vlc plays stream.mkv just fine but if I stream it with the above command, then the receiving instance of VLC interprets the audio incorrectly. The audio can be heard and on closer inspection it seems that the audio is interspersed with binary values and noise. In this image the received audio is at the top and the original at the bottom (y-axis is logarithmic - the values are zero and something very small, most likely 1):
The input audio is here: https://db.tt/OBYxiBl7
The received audio is here: https://db.tt/u9IWez9X
If I transcode the audio to s16l or s16b, either in the VLC command line or in advance with ffmpeg, then everything is fine.
I see in the release notes for 1.1.6:
Demuxers:
* Support for 24-bits PCM over RTP (audio/L24)
Does anyone know what the problem might be? Perhaps some memory being copied from somewhere it shouldn't be?