High CPU Usage ... Dual Quad Xeon

[jasonberresford]

Hello everyone,

i'm looking to do as much as I can to help optimize a Mosaic...

It consists of 12 12-16meg/sec HD streams, which are combined into a Mosaic ...

The system itself
(Dual , Quad Xeon
16 G RAM

Puts up with 10 -11 Streams okay ... Throw in the 12th, and the breakup is noticable .. so I need to get a little more out of the system ...

First a question ...

Does VLC use only CPU to do the transcoding? I've noticed it doesn't use much ram .. maybe 2% ...

Could it be the drives slowing things down ? If so, I would imagin loading VLC from RAM instead would sort that out ..

Or perhaps is there a better way to transcode then I currently am ...

****

new background broadcast enabled
setup background input fake:
setup background output #transcode{sfilter=mosaic,vcodec=mp1v,vb=10000,scale=1}:bridge-in{delay=400,id-offset=100}:standard{access=udp,mux=ts,url=10.10.143.184,sap,name="mosaic"}
setup background option mosaic-height=720
setup background option mosaic-width=1280

*******

Bandwidth shouldn't be an issue as it is on a GigE network ..

Oh .. and CPU is at around 550% .. Should be able to reach 800% ..

Which makes me belieave the bottleneck isn't CPU ...

Any insight you guys can give .. would be great! ..

[dionoea]

If you could paste your whole Mosaic configuration and command line it would be easier to help you out.

[jasonberresford]

Of course...

new channel1 broadcast enabled
setup channel1 input udp://@225.100.53.1:9053
setup channel1 output #duplicate{dst=mosaic-bridge{id=1,height=432,width=540},select=video,dst=bridge-out{id=1}}
setup channel1 option programs=3

new channel2 broadcast enabled
setup channel2 input udp://@225.100.25.5:3003
setup channel2 output #duplicate{dst=mosaic-bridge{id=2,height=432,width=540},select=video,dst=bridge-out{id=2}}
setup channel2 option programs=1

new channel3 broadcast enabled
setup channel3 input udp://@225.100.25.2:3002
setup channel3 output #duplicate{dst=mosaic-bridge{id=3,height=432,width=540},select=video,dst=bridge-out{id=3}}
setup channel3 option programs=1

new channel4 broadcast enabled
setup channel4 input udp://@225.100.54.1:9054
setup channel4 output #duplicate{dst=mosaic-bridge{id=4,height=432,width=540},select=video,dst=bridge-out{id=4}}
setup channel4 option programs=5

new channel5 broadcast enabled
setup channel5 input udp://@225.100.25.5:3003
setup channel5 output #duplicate{dst=mosaic-bridge{id=5,height=432,width=540},select=video,dst=bridge-out{id=5}}
setup channel5 option programs=2

new channel6 broadcast enabled
setup channel6 input udp://@225.100.25.2:3002
setup channel6 output #duplicate{dst=mosaic-bridge{id=6,height=432,width=540},select=video,dst=bridge-out{id=6}}
setup channel6 option programs=2

new channel7 broadcast enabled
setup channel7 input udp://@225.100.51.1:905
setup channel7 output #duplicate{dst=mosaic-bridge{id=7,height=432,width=540},select=video,dst=bridge-out{id=7}}
setup channel7 option programs=3

new channel8 broadcast enabled
setup channel8 input udp://@225.100.25.4:3004
setup channel8 output #duplicate{dst=mosaic-bridge{id=8,height=432,width=540},select=video,dst=bridge-out{id=8}}
setup channel8 option programs=1

new channel9 broadcast enabled
setup channel9 input udp://@225.100.25.3:3001
setup channel9 output #duplicate{dst=mosaic-bridge{id=9,height=432,width=540},select=video,dst=bridge-out{id=9}}
setup channel9 option programs=1

new channel10 broadcast enabled
setup channel10 input udp://@225.100.55.1:9055
setup channel10 output #duplicate{dst=mosaic-bridge{id=10,height=432,width=540},select=video,dst=bridge-out{id=10}}
setup channel10 option programs=1

new channel11 broadcast enabled
setup channel11 input udp://@225.100.25.4:3004
setup channel11 output #duplicate{dst=mosaic-bridge{id=11,height=432,width=540},select=video,dst=bridge-out{id=11}}
setup channel11 option programs=2

new channel12 broadcast enabled
setup channel12 input udp://@225.100.55.1:9055
setup channel12 output #duplicate{dst=mosaic-bridge{id=12,height=432,width=540},select=video,dst=bridge-out{id=12}}
setup channel12 option programs=2

new background broadcast enabled
setup background input fake:
setup background output #transcode{sfilter=mosaic,vcodec=mp1v,vb=10000,scale=1}:bridge-in{delay=400,id-offset=100}:standard{access=udp,mux=ts,url=10.10.143.184,sap,name="mosaic"}
setup background option mosaic-height=720
setup background option mosaic-width=1280
control background play
control channel1 play
control channel2 play
control channel3 play
control channel4 play
control channel5 play
control channel6 play
control channel7 play
control channel8 play
control channel9 play
control channel10 play
control channel11 play
control channel12 play

[dionoea]

You forgot the command line you use to launch it

[jasonberresford]

nothing fancy ...

/usr/bin/vlc -vvv --fake-file background.jpg -I http

[dionoea]

Ok, so a few pointers to help increase performance (or at least use all your cores):
1/ Use the size of the picture in the mosaic in the mosaic-bridge element. This will resize in the mosaic-bridge thread instead of in the filter thread.
2/ Add --mosaic-keep-picture to your command line to make it keep pictures sent from mosaic-bridge as is (1/ is required for this to work properly).
3/ If using 0.9.x or later, you can improve blending performance of your sub pictures on the mosaic if you add chroma=I420 to the list of mosaic-bridge options. This will improve performance only if you don't use transparency at all in the mosaic (so alpha=255).

Once you've given that a try I might be able to find a few other ideas to further improve performance, but this should already help a lot.

Also, make sure that your read http://wiki.videolan.org/Mosaic and the pages it links to at the bottom.

Cheers,

[jasonberresford]

Thank you so much,

I'll give that a shot and let you know how it goes.

[jasonberresford]

So i'm starting to apply this all ..

Could you give me an example of :


1/ Use the size of the picture in the mosaic in the mosaic-bridge element. This will resize in the mosaic-bridge thread instead of in the filter thread.

[dionoea]

Well just change the height and width you're using. You're currently providing the current input dimension if I understand correctly. What you should provide instead is the destination dimension.

For the example on http://wiki.videolan.org/Mosaic, the source streams are standard definition streams but I resize them to 180x144 in the mosaic-bridge modules.

[jasonberresford]

First off .. I'd like to thank you very much ..

I'm still tweaking around with stuff to get the sizes right .. and I have to make it look pretty....

Our plans are to throw this back into a cherry picker and output on a digital station .. (Which has been tested and works) ..

If you happen to have any more tricks to reduce load that would be great.. as sustained running time results in a seg Fault... and I think that would be reduced if I can manage a bit more fine tuning.

The HD streams are dynamic, so worst case, all streams turn HD, and things still start to go funny.




Below is the code I am using now ...


new channel1 broadcast enabled
setup channel1 input udp://@225.100.53.1:9053
setup channel1 output #duplicate{dst=mosaic-bridge{id=1,height=90,width=160},select=video,dst=bridge-out{id=1},select=audio}
setup channel1 option programs=3

new channel2 broadcast enabled
setup channel2 input udp://@225.100.25.5:3003
setup channel2 output #duplicate{dst=mosaic-bridge{id=2,height=90,width=160},select=video,dst=bridge-out{id=2},select=audio}
setup channel2 option programs=1

new channel3 broadcast enabled
setup channel3 input udp://@225.100.25.2:3002
setup channel3 output #duplicate{dst=mosaic-bridge{id=3,height=90,width=160},select=video,dst=bridge-out{id=3},select=audio}
setup channel3 option programs=1

new channel4 broadcast enabled
setup channel4 input udp://@225.100.54.1:9054
setup channel4 output #duplicate{dst=mosaic-bridge{id=4,height=90,width=160},select=video,dst=bridge-out{id=4},select=audio}
setup channel4 option programs=5

new channel5 broadcast enabled
setup channel5 input udp://@225.100.25.5:3003
setup channel5 output #duplicate{dst=mosaic-bridge{id=5,height=90,width=160},select=video,dst=bridge-out{id=5},select=audio}
setup channel5 option programs=2


new channel6 broadcast enabled
setup channel6 input udp://@225.100.25.2:3002
setup channel6 output #duplicate{dst=mosaic-bridge{id=6,height=90,width=160},select=video,dst=bridge-out{id=6},select=audio}
setup channel6 option programs=2


new channel7 broadcast enabled
setup channel7 input udp://@225.100.51.1:9051
setup channel7 output #duplicate{dst=mosaic-bridge{id=7,height=90,width=160},select=video,dst=bridge-out{id=7},select=audio}
setup channel7 option programs=3

new channel8 broadcast enabled
setup channel8 input udp://@225.100.25.4:3004
setup channel8 output #duplicate{dst=mosaic-bridge{id=8,height=90,width=160},select=video,dst=bridge-out{id=8},select=audio}
setup channel8 option programs=1

new channel9 broadcast enabled
setup channel9 input udp://@225.100.25.3:3001
setup channel9 output #duplicate{dst=mosaic-bridge{id=9,height=90,width=160},select=video,dst=bridge-out{id=9},select=audio}
setup channel9 option programs=1

new channel10 broadcast enabled
setup channel10 input udp://@225.100.55.1:9055
setup channel10 output #duplicate{dst=mosaic-bridge{id=10,height=90,width=160},select=video,dst=bridge-out{id=10},select=audio}
setup channel10 option programs=1

new channel11 broadcast enabled
setup channel11 input udp://@225.100.25.4:3004
setup channel11 output #duplicate{dst=mosaic-bridge{id=11,height=90,width=160},select=video,dst=bridge-out{id=11},select=audio}

setup channel11 option programs=2

new channel12 broadcast enabled
setup channel12 input udp://@225.100.55.1:9055
setup channel12 output #duplicate{dst=mosaic-bridge{id=12,height=90,width=160},select=video,dst=bridge-out{id=12},select=audio}
setup channel12 option programs=2

new background broadcast enabled
setup background input fake:
setup background output #transcode{sfilter=mosaic,vcodec=mp2v,vb=4096,scale=1}:bridge-in{delay=1000,id-offset=10,chroma=I420}:standard{access=udp,mux=ts,url=10.10.143.184,sap,name="mosaic"}
setup background option mosaic-alpha=255
setup background option mosaic-chroma=I420
setup background option mosaic-keep=picture
setup background option mosaic-height=480
setup background option mosaic-width=720



control background play
control channel1 play
control channel2 play
control channel3 play
control channel4 play
control channel5 play
control channel6 play
control channel7 play
control channel8 play
control channel9 play
control channel10 play
control channel11 play
control channel12 play



With command line:

/usr/bin/vlc --fake-file 720.jpg --mosaic-keep-picture --mosaic-width 720 --mosaic-height 480 -I http

[dionoea]

Your config looks fine ... did you try the chroma=I420 trick yet? (to further reduce load)

About segfaults, we know that some occur sometimes but haven't been able to fix them yet. A rework of picture handling in the development version (1.0.0-git) will hopefully fix most of those. If you feel like giving that a try... (but I'd stay on 0.9.x if I were you).

Another trick to reduce the load would be to change the scaling algorithm used (default is bi-cubic). check "vlc -p swscale --advanced" for more info. (Check your debug output first to make sure that it's using the swscale video filter for scaling)

[jasonberresford]

So I do feel like a bit of an idiot .. Seems i've been running 8.6 (Debian Stable)

And unfortunate, 9.2 Crashes with Seg Faults without fail, under high load .. And 1.0.0 Git ... well ... Above my level of expertise to get it running .. managed to get it compiled, but get lib errors when I launch VLC

Is worth the time to invest and get 9.0 , or 9.1 running? (Does it have a lot of pluses when it comes to CPU usage ?)
(I think I have to downgrade ffmpeg to get it running, the good ol'd video.c compile errors with Chroma) ..

Thanks again by the, your help has been invaluable.

[dionoea]

Well it could be worth it especially if it enables fixing bugs. (Of course if you already have 0.8.6 working I can understand if you don't see the point of upgrading to a newer version which maybe isn't as stable yet)

[jasonberresford]

you've answered every one of my questions and solved my issues ..

Thanks again ..

Have a good day