Having Problems with streaming/H264 RTP/UDP VLC-1.2.0

[mrgoblin]

Hi, (MOD: I hope this is the correct forum, please move if it belongs in the Beta section)

I'm using a Decklink capture card for input, encoding with h264. I'm outputting an RTP multicast stream.
I'm running 64bit Linux and built 1.2.0-git Twoflower (revision a2e0062) with Decklink 8.6.0 (Have to use the vlc 1.2.x for Decklink support)

The machine is an Intel i7 930 @ 2.80GHz (8 cores) 3Gb Ram

I'm capturing 1080i/50 from the Decklink SDI (Both audio and video), outputting 3000 bitrate at 960x540/25fps via this commadline.

Code: Select all

cvlc decklink:// --decklink-card-index="0" --decklink-mode="Hi50" --decklink-video-connection="sdi" --decklink-aspect-ratio="16:9" --decklink-audio-channels=2 --clock-synchro=0 --sout '#transcode{vcodec=h264,vb=3000,fps=25,scale=1,width=960, height=540, acodec=a52,ab=128,channels=2,samplerate=44100}:rtp{mux=ts,dst=224.1.1.1,port=5004}' :no-sout-rtp-sap :no-sout-standard-sap :sout-all :ttl=1 :sout-keep --rt-priority -vvv

All seems to work well for some time even though I get a constant stream of this warning message

Code: Select all

[0x6f1c68] main mux warning: late buffer for mux input (1958129)

After several minutes (10-20) vlc continues to run but stops encoding/streaming for a while (again a random number of minutes) and then resumes without any intervention.
During encoding, load average is up around 1.00 but cpu usage is only around 30% overall.
It appears that encoding stops during the pauses (although all threads still appear to be running) and cpu usage drops to next to nothing.

I have a couple of pastebin logs here to show what is happening.
The first is starting vlc and the stream beginning normally here:
http://pastebin.com/MyhYpkxh

The second is during the period with no output and shows it recovering here:
http://pastebin.com/ETsusYzT

Please let me know if any further info would be helpful.

Thank you

[Rémi Denis-Courmont]

You should probably increase caching.

[mrgoblin]

Thanks Rémi,

I did try to increase the caching (added caching=3000) but every attempt I made was met with

[0x6f6398] main stream out warning: option caching is unknown

Could you please suggest the syntax I should use?

Edit: Ok I added :sout-mux-caching=3000 which it didn't complain about.
But it still eventually starts doing this

Code: Select all

[0x6ee628] main mux warning: late buffer for mux input (2237121) [0x6ee628] main mux warning: late buffer for mux input (11334) [0x6ee628] main mux warning: late buffer for mux input (2237143) [0x6ee628] main mux warning: late buffer for mux input (16510) [0x6ee628] main mux warning: late buffer for mux input (2237205) [0x6ee628] main mux warning: late buffer for mux input (21751) [0x6ee628] main mux warning: late buffer for mux input (2237143) [0x6ee628] main mux warning: late buffer for mux input (2236772) [0x6ee628] main mux warning: late buffer for mux input (2236188) [0x6ee628] main mux warning: late buffer for mux input (5522) [0x6ee628] main mux warning: late buffer for mux input (2611) [0x6ee628] main mux warning: late buffer for mux input (2237244) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (233) [0x6ee628] main mux warning: late buffer for mux input (10080) [0x6ee628] main mux warning: late buffer for mux input (12830) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (739) [0x6ee628] main mux warning: late buffer for mux input (18007) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (834) [0x6ee628] main mux warning: late buffer for mux input (23189) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (828) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (548) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (492) [0x6ee628] main mux warning: late buffer for mux input (3688) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (717) [0x6ee628] main mux warning: late buffer for mux input (8818) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (677) [0x6ee628] main mux warning: late buffer for mux input (13973) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (674) [0x6ee628] main mux warning: late buffer for mux input (19622) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (1106) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (223) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (758) [0x6ee628] main mux warning: late buffer for mux input (133) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (813) [0x6ee628] main mux warning: late buffer for mux input (5288) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (906) [0x6ee628] main mux warning: late buffer for mux input (10421) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (877) [0x6ee628] main mux warning: late buffer for mux input (15572) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (836) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (64) [0x6ee628] main mux warning: late buffer for mux input (23143) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (619) [0x6f3fa8] stream_out_transcode stream out debug: drift is too high, resetting master sync [0x6ee628] main mux warning: late buffer for mux input (2958284) [0x6ee628] main mux warning: late buffer for mux input (6736) [0x6ee628] main mux warning: late buffer for mux input (1426) [0x6f3fa8] stream_out_transcode stream out debug: late picture skipped (854)

[mrgoblin]

Code: Select all

cvlc decklink:// --decklink-card-index="0" --decklink-mode="Hi50" --decklink-video-connection="sdi" --decklink-aspect-ratio="16:9" --decklink-audio-channels=2 --clock-synchro=0 --sout '#transcode{vcodec=h264,vb=3000,fps=25,scale=1,width=960, height=540, acodec=a52,ab=128,channels=2,samplerate=44100}:rtp{mux=ts,dst=224.1.1.1,port=5004}' :sout-mux-caching=10000 :no-sout-rtp-sap :no-sout-standard-sap :sout-all :ttl=1 :sout-keep --network-caching=10000 --sout-transcode-hurry-up --no-drop-late-frames --no-skip-frames --rt-priority -vvv

Ok so even with the cache settings above the problem persists.

One observation I've made is that this consistently happens when the number in the brackets on the mux warning goes above 2.3 Million.
Does this number refer to memory usage and if so would this indicate a lack of ram?

Code: Select all

[0x6f76b8] main mux warning: late buffer for mux input (2237123) [0x6f76b8] main mux warning: late buffer for mux input (13748) [0x6f76b8] main mux warning: late buffer for mux input (2237228) [0x6f76b8] main mux warning: late buffer for mux input (19045) [0x6f76b8] main mux warning: late buffer for mux input (2237383) [0x6f9248] stream_out_transcode stream out debug: late picture skipped (556)

[Rémi Denis-Courmont]

sout-mux-caching maybe. But it's only a work around. Someone should debug the problem and improve the clock/caching code.

[mrgoblin]

I'd be more than willing to work with someone to debug this.
Should I open a ticket or is there already work happening in this area?

[Rémi Denis-Courmont]

Nobody is working on this.

[mrgoblin]

Ok well I've been experimenting with different values for --sout-mux-caching and it seems the setting this to 15000 seems to solve the problem.
It's been running for the past few hours with no issues so far. I still get the "late buffer for mux input [1938487]" warning but the number seems to sit around that value and is pretty stable.

Thanks for your help with this Rémi.

I appreciate everything you guys do.

Regards

mrgoblin

[mrgoblin]

Seems I spoke too soon....
The --sout-mux-caching doesn't seem to have any effect at all.

Seems that some random alignment of the planets was responsible for the timing being aligned in such a way that the mux buffer stayed in sync.
After I rebooted the machine I was back to the original behaviour no matter what sout-mux-caching setting I used....
Back to the drawing board I guess.. sigh........

I did spot this in the output though I'm not sure how much help it is.

Code: Select all

[0x6fa508] main mux warning: late buffer for mux input (2322071) [0x6fa508] main mux warning: late buffer for mux input (2328489) [0x6fa508] main mux warning: late buffer for mux input (1967863) [0x6fa508] mux_ts mux debug: adjusting rate at 40000/480000 (345/120) [0x6fa508] mux_ts mux debug: adjusting rate at 40000/440000 (86/34) [0x6fa508] mux_ts mux debug: adjusting rate at 0/400000 (25/9) [0x6fa508] mux_ts mux debug: adjusting rate at 0/400000 (9/0) [0x6fa508] main mux warning: late buffer for mux input (1970695) [0x6fa508] main mux warning: late buffer for mux input (1971257) [0x6fa508] main mux warning: late buffer for mux input (1963535) [0x6fa508] main mux warning: late buffer for mux input (1963467) [0x6fa508] main mux warning: late buffer for mux input (1963853) [0x6fa508] main mux warning: late buffer for mux input (1963938) [0x6fa508] main mux warning: late buffer for mux input (1964035) [0x6fa508] main mux warning: late buffer for mux input (1964121) [0x6fa508] main mux warning: late buffer for mux input (1966669)

[jahanzeb]

I was having the same problem while capturing from Firewire/DV camera. Adding the --dv-caching=5000 option fixed the warning for me.

[Luckz]

Encoding from http/h264/ac3/TS to http/h264/mp3/TS, I have a similar problem (using 2.0.3 on Windows 7, also with the newest nightly, for example vlc-2.1.0-git-20121003-1150-win64). It will just stop doing anything at some point. It still shows as playing (pause button), in codec information it still shows the codec on the source end, but in statistics everything shows "0" and it is not connected to the source anymore.

Log bit when it stops working:

Code: Select all

main warning: late buffer for mux input (54142) main warning: late buffer for mux input (31699) main warning: late buffer for mux input (7943) ts warning: lost synchro ts debug: skipping 156 bytes of garbage ts warning: discontinuity received 0xe instead of 0x8 (pid=767) main warning: late buffer for mux input (25675) main warning: late buffer for mux input (17873) main warning: late buffer for mux input (18517) main warning: late buffer for mux input (11084)

Full log:
http://pastebin.com/YzEedVtF


This could also be related: https://trac.videolan.org/vlc/ticket/4774

[VeNoMouS]

After playing around with tvheadend streams, I was experiencing this myself, i found its due to x264 encode, add the following to your transcode

Code: Select all

venc=x264{preset=ultrafast}

I found the cpu on my i3 550 dropped from ~200% to ~30%, and those warns/errors didn't appear again

[vyadalam]

Hi, could anyone fix the issue and successfully stream a video ?
i have increased the network and mux caching, but the video still gets freezed and after a long gap goes to the next image and gets freezed again.
I urgently need to fix the issue and get the video streaming. Any kind of assistance is appreciated.

[Rémi Denis-Courmont]

If you need a fix urgently, you can either fix it yourself or contract someone.

Asking on the forum is not going to get the issue solved quickly in any case.