Load Tests

[yohann.martineau]

hello,

I'm performing some load tests on vlm with the http interface for control and raw udp packets for media.

VLC version is vlc 0.8.6e.

My operating system is debian etch and my hardware is a quad core xeon 2.33GHz, cache size 4MB with 8GB of memory. I'm on a gigabit ethernet network with only one switch, one server and client computer.

I use only one source file which has the following properties:

• container format: mp4
• bitrate: 1496kb/s
• video codec: mpeg4
• audio codec: aac

and here is how I create my vods (or sessions):

I have a bash script on the client computer which sends http requests to the server using wget to create the vod, setup input, output, start playing, stop playing and destroy the vod:

here are the telnet-like commands:

Code: Select all

new test_172.20.2.184_1234 broadcast enabled setup test_172.20.2.184_1234 input /home/d2cs/media/out/point_break9.mp4 setup test_172.20.2.184_1234 output #standard{mux=ts,access=udp,dst=172.20.2.184:1234} control test_172.20.2.184_1234 play ... control test_172.20.2.184_1234 stop del test_172.20.2.184_1234

During the first tests that I've performed I've seen that memory consumption increased linearly with the number of vods. When I ran those tests on my server there was 2GB of memory. I was able to stream 35 streams concurrently. I thought that more memory would imply more vods. So I've added 6GB of memory, and now I reach 8GB of memory.
But the problem is that I cannot stream more than 40 vods.

When I reach the 40th vod, vlc crashes, here are the error messages displayed by vlc:

d2cs@tahiti:~/programs/vlc-0.8.6e$ ./vlc -I http --rtsp-host 0.0.0.0:5554
VLC media player 0.8.6e Janus
[00000302] main interface: creating httpd
[00000302] main interface: creating VLM
[00000798] [Media: test_172.20.2.177_1275] main input error: input thread could not be created at input/input.c:265 (Ne peut allouer de la mémoire)
[00000798] [Media: test_172.20.2.177_1275] main input error: cannot create input thread
Erreur de segmentation

I find this behaviour strange, because my cpu is under 20%, RAM is at 2GB (I have 8GB and I've installed -bigmem kernel package to benefit of all memory). And there is no collision on the network.

As the problem was about thread creation, I've looked at src/misc/threads.c. I've made some tests on my system and the maximum number of threads created (using pthread) was easily reached: about 330. But it was possible to bypass this issue using detached threads. That's the reason why I've tried to modify vlc 0.8.6e source code with the following patch on threads.c:

Code: Select all

573c573,583 < i_ret = pthread_create( &p_this->thread_id, NULL, func, p_data ); --- > pthread_attr_t pthread_attr; > //TODO destroy attributes at thread destroy > if (pthread_attr_init(&pthread_attr)) { > msg_Err( p_this, "%s pthread_attr_init failed", psz_name ); > vlc_mutex_unlock( &p_this->object_lock ); > } > if (pthread_attr_setdetachstate(&pthread_attr, PTHREAD_CREATE_DETACHED)) { > msg_Err( p_this, "%s pthread_attr_setdetachstate(..., PTHREAD_CREATE" > "_DETACHED failed", psz_name ); > vlc_mutex_unlock( &p_this->object_lock ); > } > i_ret = pthread_create( &p_this->thread_id, &pthread_attr, func, p_data );

Now, I can create more vods, but ram seems to be reaching a threashold of 2,4 GB max and network throughput does not exceed 7.6 MB/s. If I create my vods every two seconds, I can even create 100 vods, but each stream seems to be incomplete. If I open a stream with ffmpeg it complains about (many) missing or broken frames. And I think that the latest vods do not send any packet.

Is there an arbitrary limit on the number of parallel vods? any hidden magic constant anywhere in the code that I would have missed?

If anyone is inspired, inspite the length of this post...
As usual, any clue is welcome.