several vlc instances using vdpau simultaneously

[sampsa]

Hi (Remi),

I have a problem when my application is using several vlc instances, all taking advantage of the vpdau hw acceleration.

In detail, I do the following:

1) I call XInitThreads()
2) I get an instance of libvlc_new()
3) I create a media player from that instance [media_player_new()]
4) I create a media from that instance [media_new()]

==> When playing live media (rtsp unicast/multicast), vdpau works fine:
- Library does not complaing about uninitialized X11
- I can see from Nvidia's proprietary linux program, in tab "GPU-0" => "Video Engine Utilization" becoming larger that "0 %" (**)

Now, when trying to play several live media simultaneously:

1) I call XInitThreads()
2) I create n libvlc_new() instances (take n=4 for starters)
3) I create a media player from each instance (= n media players)
4) I create a media from each instance (= n media)

==> When playing n live media, vdpau does not work correctly:
- Library does not complaing about uninitialized X11
- I can see from Nvidia's proprietary linux program, in tab "GPU-0" => "Video Engine Utilization" becoming larger that "0 %" (**)
- .. but however, the video is "choppy" and the vlc library is complaining all the time that "more than 5 secs too late video" .. finally the videos freeze

Question: is there some caveat here when using several vlc instances with vdpau from the same program..? Vdpau itself should be thread-safe, right?

On the contrary, when I launch from the command line n independent vlc client programs (say, again n=4), they all seem to use vdpau correctly (**) and there is no freezing of the video.

Any thoughts on this problem highly appreciated!

Regards,

Sampsa

P. S. This is in python, see also:
https://forum.videolan.org/viewtopic.php?f=32&t=136443

[Rémi Denis-Courmont]

Well, the hardware can only do so much at a time. VLC tries to fall back to software decoding if the GPU runs out of memory. But if it runs out of processing capacity before it runs out of memory, there is no way for VLC to know. The driver would have to report an error, which it apparently does not.

[sampsa]

Dear Remi,

Thanks for the answer.

The weird thing is that, we are running into this problem already with n=3..4. I thought that a modern nvidia gpu could handle quite many (say, like .. ten?) vdpau "threads".

Said that, I have to take this opportunity to ask your opinion / experience (you working in nvidia and with vdpau and all that), how much is the limit for a single gpu? (how many vdpau "threads"). Are the normal consumer-grade laptops / desktops equipped an nvidia gpu thought to be used only with a single vdpau thread maybe?

If I want to have, say, ten vdpau "threads" without problems, should I go to some extremely high-end cuda cards or what..?

Regards,

Sampsa

P. S. Or could it be that vlc somehow "messes up" the different video streams into a single vdpau "thread" .. ?

[sampsa]

Important Note!

Remi, it seems you didn't read the question carefully:

As I said in the original question, we CAN use several vdpau "threads" simultaneously, by launching several different, independent vlc clients.

.. in that case there is no problem. VLC uses 4+ vpdau "threads" and everything goes smoothly.

The problem only arises, when using several vlc instances in the SAME PROGRAM.

Regards,

Sampsa

[sampsa]

in modules/hw/vdpau/instance.c

It says ..

/** Finds an existing VDPAU instance for the given X11 display and screen.
* If not found, (try to) create a new one.
...

So it seems that vdpau instances are on "ration cards". That would explain the problem maybe..

[Rémi Denis-Courmont]

AFAIK, typical NVIDIA GeForce cards can handle two decodes at a time, at a resolution and rate that depends on the model. You might be able to do more decodes with professional cards or at lower resolution or lower frame rates.

The driver is supposed to be reentrant, and based on your description, it would seem that it is reentrant but poorly optimized. But my original answer still stands: VLC cannot know that.

[sampsa]

Dear Remi,

Thanks for the great explanation. I totally get it now ..
With the more "professional"/modern cards I guess the way to go is the "nvidia video decoder api", i.e. with NVDEC and that VPDAU is kind of obsolete, right?

What might be the status of NVDEC in in VLC?

Regards,

Sampsa

[Rémi Denis-Courmont]

I am not aware of any public API for NVDEC, and in any case, VLC does not support it.

In any case, you won't get higher rate limits with a different API on NVIDIA hardware.

[sampsa]

Final thanks Remi,

So, nvdec and vdpau both use the underlying special circuits for hw acceleration.. it just depends on the graphics card. ok. thanks.

Kind Regards,

Sampsa

[McMuffins4EVER]

I am not aware of any public API for NVDEC, and in any case, VLC does not support it.

In any case, you won't get higher rate limits with a different API on NVIDIA hardware.

The NVDECODE API is the old NVCUVID, which has been around since 2015 and was available in the SDKs from Nvidia. It was added to FFMPEG recently - libav gets it down the road - and based on Nvidia's own information, VDPAU was ended in favor of pushing the NVENCODE/NVDECODE APIs. At this point, VDPAU is incomplete on the Pascal cards - no 4K playback with my 1050 TI and my 4K monitor is no fun - and Nvidia hasn't made any real VDPAU moves in nearly two years, so I'd say it's likely dead. Currently, most Linux users can't use the new APIs with because they don't have the supported hardware, required CUDA software/new Nvidia drivers installed, and the updated FFMPEG or libav that can use them. Obviously, power-users like myself can use them with a bit of work, even though mpv is the only player that supports it at this time. That being said, the VLC team needs to work on it sooner or later in order for VLC to be used on new 4K Linux laptops and desktops that are now available from System76 and others. The new APIs are supported all the way back to Kepler, but only the GeForce 950/960 and 10-series cards (Maxwell 2.5 and Pascal, respectively) have full API support at this time, so you will need a fairly new GPU to begin implementing and testing these new APIs within VLC. The low-end GeForce 1030 will be released next month and it will be a media-oriented card that will be very inexpensive - $80 or less - and allow anyone to begin testing VLC with the new APIs in the experimental versions. I expect many Linux desktop users will buy it for upgrade purposes, even if they aren't running 4K. However, the 1050 or 1050 TI would closer match what is currently used in Linux systems equipped with Pascal GPUs, so anyone wanting to work with the new APIs might want to use one of those GPUs instead.

https://developer.nvidia.com/ffmpeg

https://developer.nvidia.com/nvidia-video-codec-sdk