Help with rtp streaming on the web. Synchronisation and other headaches.
Posted: 16 Feb 2017 16:49
Hello,
So I am building a video conferencing tool using webRTC through the Janus Gateway.
Janus allow you to broadcast an RTP stream to a bunch of clients, and vlc allows you to stream RTP, a perfect combo.
I have two things bothering me, though. But let me first explain the full process:
1/ the user uploads a video.
2/ the uploads are converted to .mkv using ffmpeg with libvpx for video and libopus for audio:3/ the output is played with cvlc:
4/ I scan the sdp output file to get the configuration of the stream and setup the Janus broadcasting webRTC negotiations.
5/ And finally I use a customized RC interface to communicate with the cvlc process and get a feedback on the process and status.
Video and audio work every time, and although I run my linux server on a virtual machine, the process usage is very low.
Almost perfect, so thanks to all the developpers for this wonderfull and versatil app. Love it as a user and as a developer.
Now that I have kissed your butts, let me show you my but
But I have two issues.
For now I will only ask about the first one as it is definitely the big issue, a deal breaker even.
So, first issue: The audio and video is either out of sync, or the video keeps freezing.
So, when I put the rtcp-mux, I get audio and video synchronisation. But the video keeps freezing and jumping ( the audio is smooth ).
When I don't put it, both video and audio are smooth, but they get out of sync.
That's true for both Chrome and Firefox.
So, it's pretty obvious that it's a problem of synchronisation of the sources, one being faster than the other and the video seems to align with the audio.
When I try playing the file locally, of course everything is fine.
I would try to read the rtp stream with VLC, but when I try I get an error:
Any idea how I can fix that? An option in VLC? A different option in encoding?
Thank you, I'll be waiting for your answers.
So I am building a video conferencing tool using webRTC through the Janus Gateway.
Janus allow you to broadcast an RTP stream to a bunch of clients, and vlc allows you to stream RTP, a perfect combo.
I have two things bothering me, though. But let me first explain the full process:
1/ the user uploads a video.
2/ the uploads are converted to .mkv using ffmpeg with libvpx for video and libopus for audio:
Code: Select all
ffmpeg -i /path/to/uploads/ltOD93RT -c:v libvpx -map 0:0 -c:a libopus -map 0:1 -shortest /path/to/videos/HSv04CMjeHDc.mkvCode: Select all
cvlc -I rc /path/to/videos/HSv04CMjeHDc.mkv --sout=#rtp{dst=127.0.0.1,name=87935a2c-7464-4743-9463-ad9d705cbeaa,sdp=file:///path/to/sdps/87935a2c-7464-4743-9463-ad9d705cbeaa.sdp,rtcp-mux,port-audio=5000,port-video=5001} --sout-keep5/ And finally I use a customized RC interface to communicate with the cvlc process and get a feedback on the process and status.
Video and audio work every time, and although I run my linux server on a virtual machine, the process usage is very low.
Almost perfect, so thanks to all the developpers for this wonderfull and versatil app. Love it as a user and as a developer.
Now that I have kissed your butts, let me show you my but
But I have two issues.
For now I will only ask about the first one as it is definitely the big issue, a deal breaker even.
So, first issue: The audio and video is either out of sync, or the video keeps freezing.
So, when I put the rtcp-mux, I get audio and video synchronisation. But the video keeps freezing and jumping ( the audio is smooth ).
When I don't put it, both video and audio are smooth, but they get out of sync.
That's true for both Chrome and Firefox.
So, it's pretty obvious that it's a problem of synchronisation of the sources, one being faster than the other and the video seems to align with the audio.
When I try playing the file locally, of course everything is fine.
I would try to read the rtp stream with VLC, but when I try I get an error:
Code: Select all
cvlc a23829f1-492f-465e-92c6-8355eeee7100.sdp
VLC media player 2.2.2 Weatherwax (revision 2.2.2-0-g6259d80)
[09bacf08] dummy interface: using the dummy interface module...
[09be6fb8] core decoder error: could not identify codec
[09be6fb8] core decoder error: Unidentified codec
[09be6fb8] core decoder error: VLC could not identify the audio or video codec
[09bb1580] live555 demux error: no data received in 10s, abortingThank you, I'll be waiting for your answers.
