Trying to add support for NVMPI ffmpeg decoders

[gavin_darkglider]

Ok, a little background..... I am a nintendo switch linux dev, and am trying to get ffmpeg to play nice with the nvmpi ffmepg decoders found here https://github.com/jocover/jetson-ffmpeg

So, this is more of a pet project, and it does require a custom version of ffmpeg, so I dont expect support to go upstream until the nvmpi decoders are actually pushed upstream for ffmpeg, which might never happen.

Here is the issue..... unfortunatly, you have to force the correct decoder, this can be done here fairly easily: https://github.com/videolan/vlc/blob/master/modules/codec/avcodec/avcodec.c#L282

so place before that line:

Code: Select all

switch (i_codec_id) { case AV_CODEC_ID_H264: //h264 p_codec = avcodec_find_decoder_by_name("h264_nvmpi"); break; case AV_CODEC_ID_HEVC: //hevc p_codec = avcodec_find_decoder_by_name("hevc_nvmpi"); break; case AV_CODEC_ID_VP9: //vp9 p_codec = avcodec_find_decoder_by_name("vp9_nvmpi"); break; case AV_CODEC_ID_MPEG4: //MPEG4 p_codec = avcodec_find_decoder_by_name("mpeg4_nvmpi"); break; case AV_CODEC_ID_MPEG2VIDEO: //MPEG2 p_codec = avcodec_find_decoder_by_name("mpeg2_nvmpi"); break; }

Problem is, even with this patch, something is wrong, while vlc trys to force use of the decoders supplied, I am getting:

Code: Select all

[0000007f64c21590] main decoder error: buffer deadlock prevented [h264_nvmpi @ 0x7f67d3a4f0] video_get_buffer: image parameters invalid [h264_nvmpi @ 0x7f67d3a4f0] get_buffer() failed [0000007f64c21590] avcodec decoder error: avcodec_send_packet critical error

The same patch, sleightly modified works in ffplay and the decoder works in moonlight-qt, but in my testing with VLC, and Kodi, I keep getting these issues. Does anyone have any thoughts on how to set the correct image parameters, or a way to figure out why they would be incorrect to begin with?

[Lissanro]

Thank you for providing your patch for VLC, it helped me to force VLC to use NVMPI. Since this old post comes up high in Google results when searching for VLC with NVMPI support, I decided to share my solution here instead of creating a new topic so it would be easier to find.

In short, the error happens because

Code: Select all

avctx->pix_fmt

is not initialized to 0 in the nvmpi decoder. VLC does not work with nvv4l2dec for the same reason, since it also has the same bug. If somebody is interested in long story how I was able to debug this and find a fix, please read this post: https://forums.developer.nvidia.com/t/h ... er/75872/6 (there I also shared deb packages for mpv with nvmpi and nvv4l2dec support).

This is how to install VLC with ffmpeg, compiled with nvmpi support (ffmpeg also includes nvv4l2dec support). Tested with Ubuntu 18.04 and JetPack 4.5 on Jetson Nano B01 (it is somewhat similar to Nintendo Switch since it uses similar Tegra chip but has 128 GPU cores instead of 256 but this does not affect hardware video decoding performance).

First, uninstall vlc if you have it installed (usually by running `sudo apt remove vlc`).

To install VLC and ffmpeg with nvmpi support, run the following commands (copy and paste all lines in the terminal):

Code: Select all

wget http://Dragon.Studio/2021/02/vlc-nvmpi-qt515.zip && unzip -n vlc-nvmpi-qt515.zip && sudo apt -y --allow-change-held-packages --allow-downgrades install ./vlc-nvmpi-qt515/*.deb && sudo apt-mark hold $(find vlc-nvmpi-qt515 | grep deb | xargs -n1 basename | cut -d_ -f1)

Last line is important otherwise Ubuntu may offer to "upgrade" vlc or ffmpeg with versions without nvmpi. I had to use vlc version newer than included in Ubuntu 18.04 due to issues I encountered (I was unable to get it working with nvmpi). Qt 5.15 is necessary for vlc 4.0.0-dev. I compiled all included Qt debs myself from sources provided by this repository https://launchpad.net/~beineri/+archive ... 5.2-bionic since id did not include binaries for arm64 architecture, it took a lot of time and effort.

The zip file also includes patches I have used for ffmpeg and vlc in case somebody prefers to compile themselves.

Performance of VLC is somewhat worse than of mpv on Jetson Nano, but both are capable of playing HEVC 4K@30FPS videos smoothly.