vaapi segmentation fault

[10robinho]

Hi,

I'm trying to make vlc work with vaapi. I've done that many times before, but this time it is not working. I've tried almost everything, but nothing helped.

So, this is vainfo output

Code: Select all

libva: VA-API version 0.32.0 libva: User requested driver 'xvba' libva: Trying to open /usr/lib/dri//xvba_drv_video.so libva: va_openDriver() returns 0 vainfo: VA-API version: 0.32 (libva 1.0.15) vainfo: Driver version: Splitted-Desktop Systems XvBA backend for VA-API - 0.7.8 vainfo: Supported profile and entrypoints VAProfileH264High : VAEntrypointVLD VAProfileVC1Advanced : VAEntrypointVLD

this is fglrxinfo

Code: Select all

display: :0.0 screen: 0 OpenGL vendor string: Advanced Micro Devices, Inc. OpenGL renderer string: ATI Radeon HD 5700 Series OpenGL version string: 4.2.11318 Compatibility Profile Context

I've compiled ffmpeg with vaapi and mplayer plays same video with vaapi that vlc fails with segmentation fault.

this is vlc output

Code: Select all

VLC media player 1.3.0-git Rincewind (revision 1.3.0-git-498-g1772a14) [0x9adfc08] main interface error: no suitable interface module [0x9a408f0] main libvlc error: interface "globalhotkeys,none" initialization failed [0x9adfc08] dummy interface: using the dummy interface module... libdvbpsi error (PSI decoder): TS discontinuity (received 3, expected 0) for PID 0 libdvbpsi error (PSI decoder): TS discontinuity (received 5, expected 0) for PID 18 libdvbpsi error (PSI decoder): TS discontinuity (received 11, expected 0) for PID 6000 libva: VA-API version 0.32.0 libva: User requested driver 'xvba' libva: Trying to open /usr/lib/dri//xvba_drv_video.so libva: va_openDriver() returns 0 xvba_video: XVBA_GetSurface(): status 2 [0x9ad4d10] avcodec decoder: Using VA API version 0.32 for hardware decoding. Segmentation fault

vlc -vvv is similar

Code: Select all

....... [0x97789d8] avcodec decoder debug: Trying VA API [0xb6f005c0] main input debug: Buffering 56% libva: VA-API version 0.32.0 libva: User requested driver 'xvba' libva: Trying to open /usr/lib/dri//xvba_drv_video.so [0xb6f005c0] main input debug: Buffering 57% [0xb6f005c0] main input debug: Buffering 59% libva: va_openDriver() returns 0 [0xb6f005c0] main input debug: Buffering 60% [0x976d0f0] ts demux debug: PSINewTableCallBack: table 0x53(83) ext=0x2b5c(11100) [0xb6f005c0] main input debug: Buffering 62% [0xb6f005c0] main input debug: Buffering 64% [0x976d0f0] ts demux debug: PSINewTableCallBack: table 0x4e(78) ext=0x2b5c(11100) [0xb6f005c0] main input debug: Buffering 65% [0xb6f005c0] main input debug: Buffering 66% xvba_video: XVBA_GetSurface(): status 2 [0x97789d8] avcodec decoder: Using VA API version 0.32 for hardware decoding. Segmentation fault

I'm realy stuck here, I must be missing somethig stupid because I made this work before with same methods...

[Jean-Baptiste Kempf]

VLC version?

[10robinho]

I wrote in first post

Code: Select all

VLC media player 1.3.0-git Rincewind (revision 1.3.0-git-498-g1772a14)

I've cloned git yesterday

[giaur]

The same in my case. Some movies causes vlc to crash immediatelly when using vaapi hardware decoding. And other movies works fine. The same codec (h.264/avc) and the same containter (mkv).

I've also tested the same movies with smplayer (vdpau) and xbmc (vaapi and vdpau) and hardware decodings works fine without any errors.

[Jean-Baptiste Kempf]

xvba_video: XVBA_GetSurface(): status 2

What does this mean?

I'm afraid that we are not working in the same way as mplayer amd xbmc since they have a special video output, I think.

[giaur]

I'm really sorry but vlc's gpu accelerated decoding does not work as it should. What's wrong or what should be done:
- vdpau filters support
- sometines causes segmentation fault (as described above)
- in my case (have no idea if anybody can confirm this), but: when vaapi is enabled, there is video freeze and pixelate (in a a few minutes period) for almost every 720p h.264/avc and some 1080p. The same problem exist when I use xbmc, but everything seems to be ok with mplayer - have no idea why.
- gpu acceleration on Windows. It seems it works partially. What can I say - something works there, cpu usage is low, but still movie playback is not smooth for some 1080p movies (but better than without hardware acceleration). The same movies works fine if I use SplashLite.

My intention is not to find fault in vlc in the worst way, really. But you can look at this if you want (or if you can) and I assume you should know when something does not work as should.

[Rémi Denis-Courmont]

1/ VDPAU is supported only by NVidia and VIA (ex-S3), while VA-API supports Intel (directly), AMD (ex-ATI) via the XvBA back-end and the earlier via VDPAU back-end. So long as VLC does not support the extra features of VDPAU over VA-API, there is little point in supporting it directly.
2/ That is probably a bug in your VA-API driver or run-time, though it could also be a bug in FFmpeg/libav.
3/ That is certainly a limitation of your GPU or VA driver - poor support for in-memory decoding. XBMC and mplayer are pure player, VLC is a media pipeline. This is not going to change as it would remove far too many useful VLC features that XBMC and mplayer do not have.
4/ Again, that is mostly a matter of FFmpeg/libav integration. If you are not happy with the status, you are more than welcome to contribute time or money to the FFmpeg or libav projects. VLC developers have enough VLC work for their free time already.

[giaur]

No. VDPAU is NVidia-specific, proprietary, and has a VAAPI front-end so there is little to no point in supporting it.

Yes I know difference between vaapi and vdpau. But I thnik vdpau direct support should be interesting.
By the way: I should retract some things I wrote. Mplayer has the same decoding problems. It seems nvidia drivers are crap and you are right - there is nothing to do with this in vlc. There are no problems on Windows, so definitelly the problem is in nvidia linux drivers.

Again, that is mostly a matter of FFmpeg/libav integration. If you are not happy with the status, you are more than welcome to contribute time or money to the FFmpeg or libav projects. VLC developers have enough VLC work for their their free time already.

There is better solution - not use ffmpeg/libav on windows. This is probably not easy to do, but I think you should think about this - this is not critical thing but hardware decoding in (for example) splash lite way seems to be better.

[Rémi Denis-Courmont]

I do not know any alternative to ffmeg/libav at the moment. The VLC developers definitely do not have the time to work on their own decoder library.

[Jean-Baptiste Kempf]

No. VDPAU is NVidia-specific, proprietary, and has a VAAPI front-end so there is little to no point in supporting it.

Yes I know difference between vaapi and vdpau. But I thnik vdpau direct support should be interesting.

Why would that be the case?

[giaur]

No. VDPAU is NVidia-specific, proprietary, and has a VAAPI front-end so there is little to no point in supporting it.

Yes I know difference between vaapi and vdpau. But I thnik vdpau direct support should be interesting.

Why would that be the case?

vdpau supports video filters: sharpen, denoise and others. I'm not sure but I think they are not available with vaapi. I know that probably there are more important things to do and to implement but I think you should add this feature to your "todo" list, as low priority item. Or any other way to use hardware sharpen/denoise etc provided by gpu.

[Jean-Baptiste Kempf]

Those are really low priority filters... And would not be exposed by VLC anyway...

[Jean-Baptiste Kempf]

This is the Linux forum...

[jackyspy]

Same problem here.

vlc --version

Code: Select all

VLC media player 2.0.5 Twoflower (revision 2.0.5-0-g1661b7d) VLC version 2.0.5 Twoflower (2.0.5-0-g1661b7d) Compiled by pbuilder on http://www.marillat.net (Dec 13 2012 11:27:59) Compiler: gcc version 4.7.2 (Debian 4.7.2-4)

vlc -vv a.avi

Code: Select all

libva: VA-API version 0.32.0 libva: va_getDriverName() returns 0 libva: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so libva: va_openDriver() returns 0 [0x163ac58] avcodec decoder: Using VA API version 0.32 for hardware decoding. Segmentation fault

Both mplayer and xbmc with vaapi accelerate work fine.

[Rémi Denis-Courmont]

Without stack trace, and since your problem is hardware-dependant, nobody can help you.

VLC uses different sets of VAAPI calls than mplayer/XBMC as already explained above.

[Rémi Denis-Courmont]

Digging an old thread from the grave...

vdpau supports video filters: sharpen, denoise and others. I'm not sure but I think they are not available with vaapi.

I cannot comment on what specific graphic cards and drivers support. But to be honest, the VAAPI interface has provisions for deinterlacing, colour adjust, rotation and various scaling algorithms. Compared to VDPAU, it lacks sharpening and noise reduction, which are not so important in my personal opinion.

I'm really sorry but vlc's gpu accelerated decoding does not work as it should. What's wrong or what should be done:
- vdpau filters support

As of now, VLC 2.2 supports high-quality scaling, deinterlacing, colour adjust, sharpening and noise reduction with VDPAU, and of course subtitle/OSD alpha blending. Rotation is not yet supported.

- sometines causes segmentation fault (as described above)

Frankly, most VAAPI related crashes have been due to broken VA drivers, or breakages in the VA binary interface. There was a common crash case due to a memory alignment error in VLC, but it was fixed in october 2012 (VLC 2.0.4).

- in my case (have no idea if anybody can confirm this), but: when vaapi is enabled, there is video freeze and pixelate (in a a few minutes period) for almost every 720p h.264/avc and some 1080p. The same problem exist when I use xbmc, but everything seems to be ok with mplayer - have no idea why.

This may very well be due to VLC versions 1.1 through 2.1 copying pictures back from GPU to CPU. As far as VDPAU is concerned, this limitation was recently fixed in VLC 2.2.

- gpu acceleration on Windows. It seems it works partially. What can I say - something works there, cpu usage is low, but still movie playback is not smooth for some 1080p movies (but better than without hardware acceleration). The same movies works fine if I use SplashLite.

Same thing. So far, VLC copies pictures to CPU when using DirectX Video Acceleration.