I ended up hard coding the string "vaapi_x11" into the va.c file of modules/codec/avcodec. Anything else I tried just didn't work.
I think the problem is that the "any" default setting doesn't pick up the hardware, so in the vlc player I have to set it to VAAPI X11. But libvlc seemed to ignore this setting in ~/.config/vlcrc. It also did4 do anything with this setting in a .vlcrc file I placed next to the executable. I also tried passing --no-ignore-config to libvlc_new, but this didn't work. I then tried passing --ignore-config as well as --avcodec-hw, vaapi_x11 / --avcodec-hw=vaapi_x11 to libvlc_new, but in the debug messages it would always search for "any" and not match anything.
[00007f12a00009e8] core generic debug: looking for hw decoder module matching "any": 2 candidates
[00007f12a00009e8] core generic debug: no hw decoder modules matched
So I found where the name was being passed and hard coded it to what I wanted, in this case vaapi_x11 (taken from the settings file).
(line 57 of va.c)
Code: Select all
-- va->module = vlc_module_load(va, "hw decoder", "$avcodec-hw", true, vlc_va_Start, va, avctx, fmt);
++ va->module = vlc_module_load(va, "hw decoder", "vaapi_x11", true, vlc_va_Start, va, avctx, fmt);
[00007fc4200009e8] core generic debug: looking for hw decoder module matching "vaapi_x11": 2 candidates
libva info: VA-API version 0.35.1
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_35
libva info: va_openDriver() returns 0
[00007fc4200009e8] core generic debug: using hw decoder module "vaapi_x11"
[00007fc4340b3828] avcodec decoder: Using Intel i965 driver for Intel(R) Bay Trail - 1.3.2 for hardware decoding.
Although this works and I can live with the solution, it doesn't seem very elegant. Is this the only way to pursade libvlc to take the vaapi setting?