[SOLVED] 2.0.0-rc1: segfault in demux.c

[Ancurio]

Hi all!

First off, here's the problem: Whenever I try to open a h264 mkv video, vlc segfaults.
gdb says this happens at:

Code: Select all

OpenDemux (p_this=<optimized out>) at demux.c:261 261 AVCodecContext *cc = s->codec;

When I type "backtrace" the output is:

Code: Select all

#0 OpenDemux (p_this=<optimized out>) at demux.c:261 #1 0x00007ffff7b81cde in vlc_module_load (p_this=0x7fffd0c05078, psz_capability=0x7ffff7bba499 "demux", psz_name=<optimized out>, b_strict= true, probe=0x7ffff7b81680 <generic_start>) at modules/modules.c:347 #2 0x00007ffff7b40b8e in demux_New (p_obj=<optimized out>, p_parent_input=<optimized out>, psz_access=0x7fffd0c0525c "mkv", psz_demux= 0x7ffff7bc2d12 "", psz_location=<optimized out>, s=<optimized out>, out= 0x7fffd0004270, b_quick=true) at input/demux.c:189 #3 0x00007ffff7b4da32 in InputSourceInit (p_input=<optimized out>, in=<optimized out>, psz_mrl=<optimized out>, psz_forced_demux=<optimized out>, b_in_can_fail=false) at input/input.c:2569 #4 0x00007ffff7b4e7ef in Init (p_input=0x7fffd0000a28) at input/input.c:1258 #5 0x00007ffff7b50010 in input_Preparse (p_parent=<optimized out>, p_item=<optimized out>) at input/input.c:202 #6 0x00007ffff7b3051d in Preparse (p_item=0x7fffe03498d0, p_playlist=<optimized out>) at playlist/preparser.c:139 #7 Thread (data=0x6ce250) at playlist/preparser.c:219 #8 0x0000003b6ca07b31 in start_thread (arg=0x7fffde212700) at pthread_create.c:305 #9 0x0000003b6c6dfd2d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

(If there's anything else I should ask gdb tell me please..)

And here's my skippable explanation for why in the hell I'd use a rc in the first place and expect it to work:
I'm desperately trying to play back 10bit h264, and the current version 1.1.13 from rpm-fusion doesn't support it.
So I thought "well, just grab the very latest source and pray". And that's what I did.
Could someone please tell me what would be considered the first "stable" version with support for 10bit h264?

Thank you very much for your time!

[TypX]

First you apparently modified your preferences because vlc shouldn't use avformat by default to demux mkv.

If you are using a packaged 2.0-rc1 you can either reset your preferences or go into tools->preferences show settings=all then Input/Codecs->demux and reset the demux module to "predefined"
If you compiled it yourself you may want to check that you have libmatroska-dev and libebml-dev installed before the ./configure step.

[Ancurio]

First you apparently modified your preferences because vlc shouldn't use avformat by default to demux mkv.

If you are using a packaged 2.0-rc1 you can either reset your preferences or go into tools->preferences show settings=all then Input/Codecs->demux and reset the demux module to "predefined"
If you compiled it yourself you may want to check that you have libmatroska-dev and libebml-dev installed before the ./configure step.

I am sorry to have taken your time with this siliness.. actually, after vlc refused even XviD video, I got suspicious.
Turns out ffmpeg doesn't enable any video codecs by default.. wow.. would have never guessed that.

The next surprise I encountered was that, when building it with x264 support,
obviously libx264 was needed, but a way higher version that the fedora repos currently bear.
So I got the latest source of that as well, compiled.. and it turns out: x264 actually needs FFMPEG first! That was baffling..
In the end I built ffmpeg without x264, then built x264 with this ffmpeg, and finally built ffmpeg again, this time with x264
(I'm never gonna wrap my head around how that works.. what ever).
After that, apparently vlc didn't accept my libass, and it turned out (I believe) that I had to explicitly state --enable-libass without any other (wrong) parameters,
even though libass was supposed to be enabled by default..
So now, it all works.. except that ass subtitles render badly most of the time, and for 10bit video I get thousands of
[0x7f7e6633c2d8] main blend error: blending RGBA to I0AL failed
errors. Oh well.

[This thread can be closed]

[TypX]

So now, it all works.. except that ass subtitles render badly most of the time, and for 10bit video I get thousands of
[0x7f7e6633c2d8] main blend error: blending RGBA to I0AL failed
errors. Oh well.
[This thread can be closed]

Blend has been rewritten two days ago, if you recompile a vlc from git you should have a correct blending on 10bits or you can just use the opengl video output.

[Ancurio]

Blend has been rewritten two days ago, if you recompile a vlc from git you should have a correct blending on 10bits or you can just use the opengl video output.

Thank you, I actually did just that after reading what was posted here.
Did this blending fix you're referring to close the bug above?

[TypX]

yes it does.