how to handle chroma == "dx11" ?

[roc1]

With libVLC 2.2.8, I was handling I420, but with version 3.0.x a new "dx11" format is provided first. I have been unable to find a way to handle it. In my app I ultimately need to copy the decoded video to a standard D3D11 texture.

Things I tried:

- return 1 from video_format_cb() when chroma is "dx11", but this is immediately followed by a callback to the same with a different format.
- passing a shared dx11 surface as **planes in video_lock_cb(), which is seemingly never touched by the library.
- return 0 for all formats except I420 (fall back to system memory copies), but even though this appears to work on the surface, the buffers I provide at video_lock_cb() are not modified. I verified the logic by swapping the 2.2.8 libVLC dll, and playback worked.

Please suggest the right approach here. Some sample code would be very helpful. I am targeting Windows 7+ with recent GPUs.

Thanks for providing and improving the library.

[John Aeliya]

you can try with Keying (DX11.TextureFX)Chromakey DX11 for guest.v4p 7 (26.0 KB)

[roc1]

Sorry, I don't understand, please elaborate.

[Jean-Baptiste Kempf]

Sorry, I don't understand, please elaborate.

dx11 is an opaque format, representing the D3D11 hardware surface.

How are you getting this ?

[roc1]

I get this in the libvlc_video_format_cb callback, as the text in the first char* argument. I actually would like to get the internal surface so I can skip the copy from video memory to CPU memory and then back to video memory. Is that possible? The other issue is that regardless of what I return or set the chroma to, with VLC 3.0.x, the surfaces are not modified when the unlock callback is called. The same identical code works fine in 2.2.8. Why could that be?

[Jean-Baptiste Kempf]

Yes, but you need to read the code of the direct3d11 video output, to know what data is in: this is opaque.

[roc1]

Okay, thanks, I presume you are pointing me to:

modules\video_output\win32\direct3d11.c

but the API there is quite different from the library interface. If I can get a hold of the DX11 surface, I will play with until I figure out the opaque format. But for now, I am stuck on what to do in video_format_cb, video_lock_cb and video_unlock_cb, so I get this DX11 surface in the first place.

[roc1]

Also when trying to accept the DX11 by returning 1, I get this as debug spew:

Code: Select all

unsigned video_format_cb(char *chroma, unsigned *width, unsigned *height, unsigned *pitches, unsigned *lines) { const DWORD dx11 = MAKEFOURCC('D', 'X', '1', '1'); if (*(DWORD *)chroma == dx11) { return 1; } }

Code: Select all

main vout display debug: A filter to adapt decoder DX11 to display DX11 is needed A filter to adapt decoder DX11 to display DX11 is needed main filter debug: looking for video converter module matching "any": 24 candidates looking for video converter module matching "any": 24 candidates main filter debug: no video converter modules matched no video converter modules matched main vout display error: Failed to create video converter Failed to create video converter main vout display error: Failed to adapt decoder format to display Failed to adapt decoder format to display main vout display debug: removing module "vmem" removing module "vmem" main video output error: video output creation failed video output creation failed

[Rémi Denis-Courmont]

NEVER return a format that you do not know or understand. Either fail or override with a known format.