Segmentation Fault Transcoding to x264

[rogerlittin]

Hi,

Using 0.9.6 on CentOS 5 I am getting a segmentation fault when trying to transcode a rtsp stream to x264. The fault occurs as soon as the sdp details are displayed in the output screen.

Is this related to the Win32 crashing with x264 transcoding?

Is there a way to get a stack trace with gdb or similar from the release package without rebuilding it.

Roger.

[rogerlittin]

Here is the backtrace for the fault.

I realy need to get this to work. I have it working on another machine with Fedora 9. I notice that the Fedora version has a later version of libx264 & libc.so.6.

Both versions are installed with 'yum install vlc' which should install the latest version.

Code: Select all

Program received signal SIGSEGV, Segmentation fault. [Switching to Thread -1327539312 (LWP 879)] 0x05df8c4e in x264_pixel_satd_8x8_ssse3 () from /usr/lib/libx264.so.55 (gdb) bt #0 0x05df8c4e in x264_pixel_satd_8x8_ssse3 () from /usr/lib/libx264.so.55 #1 0x05daad16 in x264_slicetype_mb_cost (h=0xb72ca930, a=0xb0df2528, frames=0xb0df4da4, p0=0, p1=0, b=0, dist_scale_factor=128) at encoder/slicetype.c:207 #2 0x05daaf6d in x264_slicetype_frame_cost (h=0xb72ca930, a=0xb0df2528, frames=0xb0df4da4, p0=0, p1=0, b=0, b_intra_penalty=0) at encoder/slicetype.c:291 #3 0x05dacb0e in x264_rc_analyse_slice (h=0xb72ca930) at encoder/slicetype.c:531 #4 0x05dcc55e in rate_estimate_qscale (h=0xb72ca930) at encoder/ratecontrol.c:1293 #5 0x05dccfe9 in x264_ratecontrol_start (h=0xb72ca930, i_force_qp=0) at encoder/ratecontrol.c:640 #6 0x05de6576 in x264_encoder_encode (h=0xb72ca930, pp_nal=0xb0df50f8, pi_nal=0xb0df50f4, pic_in=0xb0df50b8, pic_out=0xb0df50b8) at encoder/encoder.c:1411 #7 0x0056afa7 in Encode (p_enc=0x97a3e9c, p_pict=0xb721c4b8) at x264.c:1339 #8 0x00e9d902 in Send (p_stream=0x9791504, id=0x9791cd0, p_buffer=0xb7221258) at transcode.c:2026 #9 0x00617a4d in sout_InputSendBuffer (p_input=0x97a3300, p_buffer=0xb7221258) at stream_output/stream_output.c:284 #10 0x005ad709 in DecoderDecode (p_dec=0xb7207ffc, p_block=0xb728af28) at input/decoder.c:950 #11 0x005af1b5 in DecoderThread (p_this=0xb7207ffc) at input/decoder.c:594 #12 0x00601a98 in thread_entry (data=0xb72086a0) at misc/threads.c:465 #13 0x00c7246b in start_thread () from /lib/libpthread.so.0 #14 0x0026edbe in clone () from /lib/libc.so.6

[xtophe]

That seems quite deep inside libx264. so it's unlikely it's a vlc problem.
And as you said it is fixed in later version of x264. So just use a recent version of x264

[rogerlittin]

I did try that. It seems that the vlc on the centos box is using libx264.so.55 directly and not through the libx264.so symlink. I tried to re-compile x264 and got libx264.so.65 and put it in /usr/lib & adjusted the symlink but vlc doens't want to know about it. If I delete the 55 file it complains that x264 cannot be found.

Code: Select all

[00000465] x264 encoder debug: version x264 0.55.X x264 [info]: using SAR=12/11 x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 SSSE3 [00000465] main encoder debug: using encoder module "x264"

When I remove libx264.so.55 then it is libavcodec_plugin that is complaining. Is it this one that needs to be recompiled?

Code: Select all

[00000464] main decoder warning: cannot load module `/usr/lib/vlc/codec/libavcodec_plugin.so' (libx264.so.55: cannot open shared object file: No such file or directory)

I know I need to recompile either the libavcodec plugin or the x264 plugin modules but I am lost on how to do it. I did try to recompile the whole player but it kept on failing.

Can anyone help on how to recompile the module only?

Please.....

Thanks,

Roger.