VLC Memory leak problem

[horsedorf]

Can you remove the --enable-optimize-memory and try again?
Can you provide a full valgrind log and post it here or send it to me by mail?

Yes, I can remove teh enable optimize memory flag and rebuild and try again, however, I think it will just verify that the problem is still there as I have observed this behaviour prior to rebuilding with enable optimize memory.
Also, I just took a look at it this morning and two interesting things have happened, one, the top is showing the process to now be using 117meg of memory. On the receive side, I'm using vlc to watch the stream and it had to close with the message "Too many RTP streams".
Also, I have two vlc servers running, one that is doing a transcode, one that is not, the one doing the transcode is the one tha thas gotten huge, the one that is not doign the transcode start at 4532k resident memory and now is showing 4736k of resident memory after running all night. The transcoded stream had a vlc client watching it using "udp" and just looking at socket 8080.

Now, as for running valgrind, I have little experience using it, could you tell me what flags I should use with it to get information you might need on it?
Thanks!
Rick

[Rémi Denis-Courmont]

VLC only follows one RTP stream by default. You can increase the limit, but you will get two videos at the same time... Normally, you'd rather want to not send two streams to the same destination...

[horsedorf]

I started up valgrind using the following...

valgrind -v --leak-check=full --log-file=tc-vlc-valgrind.log vlc -vvv NineShaneAcker.flv.mpg --repeat --sout='#transcode{vcodec=h264}':'rtp{dst=192.168.2.54,port=1238,sdp=rtsp://192.168.2.34:8088/test.sdp}'

and I did so on a version of vlc that I built using the following config.

./configure --disable-dvdnav --disable-cdda --disable-libcddb --disable-x11 --disable-xvideo --disable-glx --disable-opengl --disable-sdl --disable-skins2 --enable-x264

The problem I'm having with this now is that I get a constant stream of "late picture skipped" errors now, apparently my laptop does not have the testicle fortitude to run valgrind on vlc without consuming too much of the laptop's resources (cpu throughput, namely). Is this a 'normal' result of running valgrind on a process?

running it without valgrind shows me what I was expecting, a slow gradual increas of resident memory being used over time, only starting with a larger memory set due to it having been built without memory optimization.

Has anyone else been able to reproduce this with their own systems? Hopefully someone can since what I'm doing does not require any specialized hardware. Just an flv file, that's been converted to mpg with ffmpeg and two computers. And then serving the stream from a linux box to the other machine that's viewing the stream. Hopefully you can see what I am seeing. More eyes on this would be a good thing. In the mean time, I'll do what I can to try and further isolate where/when and how this is happening.

Rick

[horsedorf]

VLC only follows one RTP stream by default. You can increase the limit, but you will get two videos at the same time... Normally, you'd rather want to not send two streams to the same destination...

Yes, I had gathered that, and when running this "for real" I'll be doing that, however, I wanted to see memory useage for the two different processes with one running the transcoding to mpeg4 part10 and the other not doing any transcoding at all.

It ran fine for quite a while, I let it run over night and when I got back to it in the morning, the receive side was complaining to too many rtp streams. I was wondering if the invocation I used caused it to start generating a new stream each time which caused problems somehow? At any rate, i think it might be a different issue other than the memory useage issue I'm seeing. But of course, until it's dug into further I can't really be certain of that. As I move forward though, I'll not be running two instances of vlc and instead just run one instance of it.

Rick

[Rémi Denis-Courmont]

valgrind is very CPU intensive, yes. I never managed to decode video under valgrind.

[horsedorf]

valgrind is very CPU intensive, yes. I never managed to decode video under valgrind.

Darn, that's inconvenient! It's going to be hard to find a memory leak, then, if the tool that lets us find it takes up most of the cpu's time. Perhaps what is needed here is someone who has a latest greatest up to datest cpu running linux who can possibly try running these experiments to see if the problem is reproducible on their machine, and then, if so, run valgrind against vlc and hopefully have enough processing power to be able to do that as well as encode the mpeg4part10 video stream with vlc so that it can find the memory leak?

Sadly, I do not have that kind of computing power myself. My laptop is running an old celeron 1.4ghz processor and it simply does not have the raw throughput to run valgrind and vlc and be able to repproduce this.. though I'll try again with just sending out the mpg file since I do notice that it tends to slowly creep in memory size as well. I'll set it up friday evening and let it run over the weekend to see if it can produce some results to look at.

Any and all input is appreciated.

Rick

[sanitariu]

Rémi Denis-Courmont if you like i can give you root access to one of these DVB-S streaming arround 10 channels.
I tried latest git and it still leaks.
Server can not work more than 15-20 hours. After start swapping vlc streams are dead and need restart.
Contact me if you like access.

[Rémi Denis-Courmont]

DVB is quite outside my area of expertise. I doubt I can help.

[krakorar]

Hello,

I believe that I have found this memory leak in vlc-0.9.10. I am using a Creative Optia AF USB web cam and a PCM microphone. If the PCM microphone is not plugged then I see vlc consume memory until the Linux OOM killer makes an appearance some 20 or so odd minutes later. The problem is that video is buffered by the MPEG4 video encoder and stuffed into a FIFO to be consumed by the MPEG2 transport packetizer as does the MP3 audio encoder. However, the MPEG2 transport packetizer (the Mux function in ts.c) performs a short-circuit return if either of the aforementioned video or audio FIFOs contain less than one packet. In my case the audio FIFO did not contain any packets causing the Mux function in ts.c to perform a short circuit return before the video FIFO could be emptied on the next iteration of the loop. The video FIFO would then fill until I killed vlc, until the Linux OOM killer intervened or until the PCM microphone was plugged into the PCM audio input. Here is my command line:

/usr/bin/vlc -I dummy -V dummy -vvvvv --dscp=0x01 v4l2:// :v4l2-dev=/dev/video0 :v4l2-adev="hw.0,0" :v4l2-width=720 :v4l2-height=480 :v4l2-standard=0 --sout '#transcode{vcodec=mp4v,vb=6000,fps=30,scale=1,acodec=mpga,ab=128,channels=2}:duplicate{dst=rtp{mux=ts,sdp=rtsp://192.168.1.160:16200/camera.sdp}}' vlc://quit

Here is the modified Mux function in ts.c...I am sure someone could find a more elegant solution...

Code: Select all

/***************************************************************************** * Mux: Call each time there is new data for at least one stream ***************************************************************************** * *****************************************************************************/ static int Mux( sout_mux_t *p_mux ) { sout_mux_sys_t *p_sys = p_mux->p_sys; ts_stream_t *p_pcr_stream; if( p_sys->i_pcr_pid == 0x1fff ) { int i; for( i = 0; i < p_mux->i_nb_inputs; i++ ) { block_FifoEmpty( p_mux->pp_inputs[i]->p_fifo ); } msg_Dbg( p_mux, "waiting for PCR streams" ); msleep( 1000 ); return VLC_SUCCESS; } p_pcr_stream = (ts_stream_t*)p_sys->p_pcr_input->p_sys; for( ;; ) { sout_buffer_chain_t chain_ts; int i_packet_count; int i_packet_pos; mtime_t i_pcr_dts; mtime_t i_pcr_length; mtime_t i_shaping_delay; int i, j; if( p_pcr_stream->b_key_frame ) { i_shaping_delay = p_pcr_stream->i_pes_length; } else { i_shaping_delay = p_sys->i_shaping_delay; } /* 1: get enough PES packet for all input */ for( ;; ) { bool b_ok = true; block_t *p_data; /* Accumulate enough data in the pcr stream (>i_shaping_delay) */ /* Accumulate enough data in all other stream ( >= length of pcr)*/ for( i = -1; i < p_mux->i_nb_inputs; i++ ) { sout_input_t *p_input; ts_stream_t *p_stream; int64_t i_spu_delay = 0; if( i == -1 ) p_input = p_sys->p_pcr_input; else if( p_mux->pp_inputs[i]->p_sys == p_pcr_stream ) continue; else p_input = p_mux->pp_inputs[i]; p_stream = (ts_stream_t*)p_input->p_sys; if( ( ( p_stream == p_pcr_stream ) && ( p_stream->i_pes_length < i_shaping_delay ) ) || ( p_stream->i_pes_dts + p_stream->i_pes_length < p_pcr_stream->i_pes_dts + p_pcr_stream->i_pes_length ) ) { for( j = 0; j < p_mux->i_nb_inputs; j++ ) { if ( ( ( p_mux->pp_inputs[j]->p_fmt->i_cat == AUDIO_ES ) || ( p_mux->pp_inputs[j]->p_fmt->i_cat == VIDEO_ES ) ) && block_FifoCount( p_mux->pp_inputs[j]->p_fifo ) > 1) { break; } } if ( j == p_mux->i_nb_inputs ) { /* We need more data */ return VLC_SUCCESS; } /* Need more data */ if( block_FifoCount( p_input->p_fifo ) <= 1 ) { if( !(( p_input->p_fmt->i_cat == AUDIO_ES ) || ( p_input->p_fmt->i_cat == VIDEO_ES )) && (block_FifoCount( p_input->p_fifo ) <= 0)) { /* spu, only one packet is needed */ continue; } else if( p_input->p_fmt->i_cat == SPU_ES ) { /* Don't mux the SPU yet if it is too early */ block_t *p_spu = block_FifoShow( p_input->p_fifo ); i_spu_delay = p_spu->i_dts - p_pcr_stream->i_pes_dts; if( ( i_spu_delay > i_shaping_delay ) && ( i_spu_delay < INT64_C(100000000) ) ) continue; if ( ( i_spu_delay >= INT64_C(100000000) ) || ( i_spu_delay < INT64_C(10000) ) ) { BufferChainClean( &p_stream->chain_pes ); p_stream->i_pes_dts = 0; p_stream->i_pes_used = 0; p_stream->i_pes_length = 0; continue; } } } b_ok = false; if( p_stream == p_pcr_stream || p_sys->b_data_alignment || p_input->p_fmt->i_codec != VLC_FOURCC('m', 'p', 'g', 'a') ) { p_data = NULL; if ( ( block_FifoCount( p_input->p_fifo ) > 0 ) || ( p_input->p_fmt->i_cat == SPU_ES) ) { p_data = block_FifoGet( p_input->p_fifo ); } if( p_input->p_fmt->i_codec == VLC_FOURCC('m', 'p', '4', 'a' ) ) p_data = Add_ADTS( p_data, p_input->p_fmt ); } else p_data = FixPES( p_mux, p_input->p_fifo ); if (p_data) { if( block_FifoCount( p_input->p_fifo ) > 0 && p_input->p_fmt->i_cat != SPU_ES ) { block_t *p_next = block_FifoShow( p_input->p_fifo ); p_data->i_length = p_next->i_dts - p_data->i_dts; } else if( p_input->p_fmt->i_codec != VLC_FOURCC('s', 'u', 'b', 't' ) ) p_data->i_length = 1000; if( ( p_pcr_stream->i_pes_dts > 0 && p_data->i_dts - 10000000 > p_pcr_stream->i_pes_dts + p_pcr_stream->i_pes_length ) || p_data->i_dts < p_stream->i_pes_dts || ( p_stream->i_pes_dts > 0 && p_input->p_fmt->i_cat != SPU_ES && p_data->i_dts - 10000000 > p_stream->i_pes_dts + p_stream->i_pes_length ) ) { msg_Warn( p_mux, "packet with too strange dts " "(dts=%"PRId64",old=%"PRId64",pcr=%"PRId64")", p_data->i_dts, p_stream->i_pes_dts, p_pcr_stream->i_pes_dts ); block_Release( p_data ); BufferChainClean( &p_stream->chain_pes ); p_stream->i_pes_dts = 0; p_stream->i_pes_used = 0; p_stream->i_pes_length = 0; if( p_input->p_fmt->i_cat != SPU_ES ) { BufferChainClean( &p_pcr_stream->chain_pes ); p_pcr_stream->i_pes_dts = 0; p_pcr_stream->i_pes_used = 0; p_pcr_stream->i_pes_length = 0; } } else { int i_header_size = 0; int b_data_alignment = 0; if( p_input->p_fmt->i_cat == SPU_ES ) { if( p_input->p_fmt->i_codec == VLC_FOURCC('s','u','b','t') ) { /* Prepend header */ p_data = block_Realloc( p_data, 2, p_data->i_buffer ); p_data->p_buffer[0] = ( (p_data->i_buffer - 2) >> 8) & 0xff; p_data->p_buffer[1] = ( (p_data->i_buffer - 2) ) & 0xff; /* remove trailling \0 if any */ if( p_data->i_buffer > 2 && p_data->p_buffer[p_data->i_buffer -1] == '\0' ) p_data->i_buffer--; /* Append a empty sub (sub text only) */ if( p_data->i_length > 0 && !( p_data->i_buffer == 1 && *p_data->p_buffer == ' ' ) ) { block_t *p_spu = block_New( p_mux, 3 ); p_spu->i_dts = p_spu->i_pts = p_data->i_dts + p_data->i_length; p_spu->i_length = 1000; p_spu->p_buffer[0] = 0; p_spu->p_buffer[1] = 1; p_spu->p_buffer[2] = ' '; EStoPES( p_mux->p_sout, &p_spu, p_spu, p_input->p_fmt, p_stream->i_stream_id, 1, 0, 0, 0 ); p_data->p_next = p_spu; } } else if( p_input->p_fmt->i_codec == VLC_FOURCC('t','e','l','x') ) { /* EN 300 472 */ i_header_size = 0x24; b_data_alignment = 1; } else if( p_input->p_fmt->i_codec == VLC_FOURCC('d','v','b','s') ) { /* EN 300 743 */ b_data_alignment = 1; } } else if( p_data->i_length < 0 || p_data->i_length > 2000000 ) { /* FIXME choose a better value, but anyway we * should never have to do that */ p_data->i_length = 1000; } p_stream->i_pes_length += p_data->i_length; if( p_stream->i_pes_dts == 0 ) { p_stream->i_pes_dts = p_data->i_dts; } /* Convert to pes */ if( p_stream->i_stream_id == 0xa0 && p_data->i_pts <= 0 ) { /* XXX yes I know, it's awful, but it's needed, * so don't remove it ... */ p_data->i_pts = p_data->i_dts; } EStoPES ( p_mux->p_sout, &p_data, p_data, p_input->p_fmt, p_stream->i_stream_id, 1, b_data_alignment, i_header_size, 0 ); BufferChainAppend( &p_stream->chain_pes, p_data ); if( p_sys->b_use_key_frames && p_stream == p_pcr_stream && (p_data->i_flags & BLOCK_FLAG_TYPE_I) && !(p_data->i_flags & BLOCK_FLAG_NO_KEYFRAME) && (p_stream->i_pes_length > 400000) ) { i_shaping_delay = p_stream->i_pes_length; p_stream->b_key_frame = 1; } } } } } for( j = 0; j < p_mux->i_nb_inputs; j++ ) { if ( ( ( p_mux->pp_inputs[j]->p_fmt->i_cat == AUDIO_ES ) || ( p_mux->pp_inputs[j]->p_fmt->i_cat == VIDEO_ES ) ) && block_FifoCount( p_mux->pp_inputs[j]->p_fifo ) <= 1) { /* We need more data */ return VLC_SUCCESS; } } if( b_ok ) { break; } } /* save */ i_pcr_dts = p_pcr_stream->i_pes_dts; i_pcr_length = p_pcr_stream->i_pes_length; p_pcr_stream->b_key_frame = 0; /* msg_Dbg( p_mux, "starting muxing %lldms", i_pcr_length / 1000 ); */ /* 2: calculate non accurate total size of muxed ts */ i_packet_count = 0; for( i = 0; i < p_mux->i_nb_inputs; i++ ) { ts_stream_t *p_stream = (ts_stream_t*)p_mux->pp_inputs[i]->p_sys; block_t *p_pes; /* False for pcr stream but it will be enough to do PCR algo */ for( p_pes = p_stream->chain_pes.p_first; p_pes != NULL; p_pes = p_pes->p_next ) { int i_size = p_pes->i_buffer; if( p_pes->i_dts + p_pes->i_length > p_pcr_stream->i_pes_dts + p_pcr_stream->i_pes_length ) { mtime_t i_frag = p_pcr_stream->i_pes_dts + p_pcr_stream->i_pes_length - p_pes->i_dts; if( i_frag < 0 ) { /* Next stream */ break; } i_size = p_pes->i_buffer * i_frag / p_pes->i_length; } i_packet_count += ( i_size + 183 ) / 184; } } /* add overhead for PCR (not really exact) */ i_packet_count += (8 * i_pcr_length / p_sys->i_pcr_delay + 175) / 176; /* 3: mux PES into TS */ BufferChainInit( &chain_ts ); /* append PAT/PMT -> FIXME with big pcr delay it won't have enough pat/pmt */ GetPAT( p_mux, &chain_ts ); GetPMT( p_mux, &chain_ts ); i_packet_pos = 0; i_packet_count += chain_ts.i_depth; /* msg_Dbg( p_mux, "estimated pck=%d", i_packet_count ); */ for( ;; ) { int i_stream; mtime_t i_dts; ts_stream_t *p_stream; sout_input_t *p_input; block_t *p_ts; bool b_pcr; /* Select stream (lowest dts) */ for( i = 0, i_stream = -1, i_dts = 0; i < p_mux->i_nb_inputs; i++ ) { p_stream = (ts_stream_t*)p_mux->pp_inputs[i]->p_sys; if( p_stream->i_pes_dts == 0 ) { continue; } if( i_stream == -1 || p_stream->i_pes_dts < i_dts ) { i_stream = i; i_dts = p_stream->i_pes_dts; } } if( i_stream == -1 || i_dts > i_pcr_dts + i_pcr_length ) { break; } p_stream = (ts_stream_t*)p_mux->pp_inputs[i_stream]->p_sys; p_input = p_mux->pp_inputs[i_stream]; /* do we need to issue pcr */ b_pcr = false; if( p_stream == p_pcr_stream && i_pcr_dts + i_packet_pos * i_pcr_length / i_packet_count >= p_sys->i_pcr + p_sys->i_pcr_delay ) { b_pcr = true; p_sys->i_pcr = i_pcr_dts + i_packet_pos * i_pcr_length / i_packet_count; } /* Build the TS packet */ p_ts = TSNew( p_mux, p_stream, b_pcr ); if( p_sys->csa != NULL && (p_input->p_fmt->i_cat != AUDIO_ES || p_sys->b_crypt_audio) && (p_input->p_fmt->i_cat != VIDEO_ES || p_sys->b_crypt_video) ) { p_ts->i_flags |= BLOCK_FLAG_SCRAMBLED; } i_packet_pos++; /* */ BufferChainAppend( &chain_ts, p_ts ); } /* 4: date and send */ TSSchedule( p_mux, &chain_ts, i_pcr_length, i_pcr_dts ); } }

Best Regards,

Rob Krakora

[ILEoo]

please send unified diff against git master about that change to vlc-devel if you think it fixes your problem

[Makarov]

Any news about patch of krakorar?

[Cwelle]

Has this been implemented? I still experience memory leaks while transcoding.

[balta2ar]

I have the same memory leak problem with vlc 1.1.4 on ubuntu 9.10 x86_64.

vlc runs on a machine with 4GB RAM. For the first couple of hours it consumes about 10% of RAM. But then usage increases to nearly 90%. Few hours later the machine stops responding. The only thing which helps is reboot.

[Rémi Denis-Courmont]

No patch means no fix.