Help with rtp streaming on the web. Synchronisation and other headaches.

[VisitorG]

Hello,

So I am building a video conferencing tool using webRTC through the Janus Gateway.
Janus allow you to broadcast an RTP stream to a bunch of clients, and vlc allows you to stream RTP, a perfect combo.

I have two things bothering me, though. But let me first explain the full process:
1/ the user uploads a video.
2/ the uploads are converted to .mkv using ffmpeg with libvpx for video and libopus for audio:

Code: Select all

ffmpeg -i /path/to/uploads/ltOD93RT -c:v libvpx -map 0:0 -c:a libopus -map 0:1 -shortest /path/to/videos/HSv04CMjeHDc.mkv

3/ the output is played with cvlc:

Code: Select all

cvlc -I rc /path/to/videos/HSv04CMjeHDc.mkv --sout=#rtp{dst=127.0.0.1,name=87935a2c-7464-4743-9463-ad9d705cbeaa,sdp=file:///path/to/sdps/87935a2c-7464-4743-9463-ad9d705cbeaa.sdp,rtcp-mux,port-audio=5000,port-video=5001} --sout-keep

4/ I scan the sdp output file to get the configuration of the stream and setup the Janus broadcasting webRTC negotiations.
5/ And finally I use a customized RC interface to communicate with the cvlc process and get a feedback on the process and status.

Video and audio work every time, and although I run my linux server on a virtual machine, the process usage is very low.
Almost perfect, so thanks to all the developpers for this wonderfull and versatil app. Love it as a user and as a developer.

Now that I have kissed your butts, let me show you my but

But I have two issues.
For now I will only ask about the first one as it is definitely the big issue, a deal breaker even.

So, first issue: The audio and video is either out of sync, or the video keeps freezing.
So, when I put the rtcp-mux, I get audio and video synchronisation. But the video keeps freezing and jumping ( the audio is smooth ).
When I don't put it, both video and audio are smooth, but they get out of sync.
That's true for both Chrome and Firefox.
So, it's pretty obvious that it's a problem of synchronisation of the sources, one being faster than the other and the video seems to align with the audio.
When I try playing the file locally, of course everything is fine.
I would try to read the rtp stream with VLC, but when I try I get an error:

Code: Select all

cvlc a23829f1-492f-465e-92c6-8355eeee7100.sdp VLC media player 2.2.2 Weatherwax (revision 2.2.2-0-g6259d80) [09bacf08] dummy interface: using the dummy interface module... [09be6fb8] core decoder error: could not identify codec [09be6fb8] core decoder error: Unidentified codec [09be6fb8] core decoder error: VLC could not identify the audio or video codec [09bb1580] live555 demux error: no data received in 10s, aborting

Any idea how I can fix that? An option in VLC? A different option in encoding?

Thank you, I'll be waiting for your answers.

[Jean-Baptiste Kempf]

Can you share all the logs, with -vvv ?

[VisitorG]

Hi Jean-Baptiste, and thank you for answering.
Here are some logs:
http://pastebin.com/tjfuVgsS
Nothing seems to be output once the streaming starts.

[VisitorG]

[Jean-Baptiste Kempf]

On those logs, everything is fine.