Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

This forum is about all development around libVLC.
whammy38
New Cone
New Cone
Posts: 8
Joined: 13 May 2015 20:47

Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby whammy38 » 13 May 2015 22:19

Scenario: WPF app using a PInvoke wrapper around libvlc 3.0.0.0 for video playback. Videos can be live streaming (UDP multicast or RTSP), archived (.ts files over HTTP), or static files on the local file system.

Problem: Since our application allows seeking backwards (even for live streaming videos) we must buffer the video from the network locally. To accomplish this we have a 'splitter' that re-streams the live video to 2 ports on the loopback interface using a dedicated libvlc instance:

Code: Select all

public VlcSplitter(Uri mediaUri, UdpPortPool udpPortPool) { portPool = udpPortPool; recordPortLease = portPool.Borrow(); playbackPortLease = portPool.Borrow(); var options = string.Format( "--sout=#duplicate{{dst=std{{access=udp,mux=ts,dst=127.0.0.1:{0}}},dst=std{{access=udp,mux=ts,dst=127.0.0.1:{1}}}}}", RecordPort, PlaybackPort); context = new VlcContext(options); using (var media = new Media(context, mediaUri)) { mediaPlayer = new MediaPlayer(media, false); mediaPlayer.Play(); } }
VlcContext is the dedicated libvlc instance (created via libvlc_new with the given options). Our 'saver' uses the recording port to save the video to the file system in 10 minute segments. Our video player is a libvlc MediaPlayer that is created using its own instance of libvlc. It plays the live video off the 127.0.0.1:{PlaybackPort} stream coming from the splitter.

This works, but it comes with random but eventual deadlocks on native libvlc calls (libvlc_media_player_stop, libvlc_media_player_play, and libvlc_media_player_release have all deadlocked). I've confirmed that I don't get these deadlocks when the MediaPlayer on our video player never tries to play the stream from the splitter. I suspect this has something to do with there being 2 libvlc instances with 1 playing the stream output of another. Is what we're doing a legal way to use libvlc? If so what can we do to avoid these deadlocks? If not then what would be the best practice for our scenario?

P.S. I've seen lots of posts about deadlocks involving certain libvlc calls executing on threads processing windows messages. We are executing media player api calls (play, stop, release) on a thread that is definitely pumping windows messages, but the only time I've seen deadlocks is in the scenario described above (where it's playing the stream from the other libvlc instance) so I don't think this has anything to do with WPF.

whammy38
New Cone
New Cone
Posts: 8
Joined: 13 May 2015 20:47

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby whammy38 » 14 May 2015 02:17

I have some more info after running some more tests:

- Setting our MediaPlayers Media to the multicast video directly (instead of the loopback video from the splitter) also resulted in deadlocks when our splitter was also splitting that same multicast video.
- Doing the same thing as above but disabling the splitter resulted in no deadlocks.

The hypothesis I have now that is consistent with everything I've seen is that 2 instances of libvlc cannot be interacting with the same video stream. This covers the behavior I described in the 1st post where 1 instance is streaming out to 127.0.0.1:{SomePort} and another instance is trying to play the video from that same IP:Port. It also covers the case I described in this post where 2 instances were both doing something with the same multicast video.

Rémi Denis-Courmont
Developer
Developer
Posts: 15268
Joined: 07 Jun 2004 16:01
VLC version: master
Operating System: Linux
Contact:

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby Rémi Denis-Courmont » 14 May 2015 16:09

Based on the description, I would rather think that this is a Windows message problem. If you are absolutely sure that this is a VLC bug instead, then please provide thread symbolic strack traces, or native C code to reproduce the problem.
Rémi Denis-Courmont
https://www.remlab.net/
Private messages soliciting support will be systematically discarded

whammy38
New Cone
New Cone
Posts: 8
Joined: 13 May 2015 20:47

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby whammy38 » 15 May 2015 21:09

I spent yesterday setting up a native test app that reproduces the deadlock and ran a bunch of tests. I set up pretty much the same situation that we're setting up through our PInvokes... 2 libvlc instances, 2 media players, 1 is playing a multicast video and is duplicating and streaming it out on localhost, and the other instance is just playing one of the localhost streams. I set up the same callback situation that we have for our media players, but I found that in the native code the callbacks were never executed (so I left the function bodies empty). In our C# callback code we're not doing anything that looks like it could be re-entrant.

The following is the main for the Visual C++ console app. I followed the directions here to generate a static library from the libvlc dll (3.0.0-git-20140921-0402) to link against. I have my windows VLC Media Player sending out a video over multicast on udp://@239.8.6.8:5309 while I'm running this test console app.

Code: Select all

#include <vlc\vlc.h> #include <memory> void* frameBuffer; void* video_lock(void* opaque, void** planes){ return NULL; } void video_unlock(void *opaque, void *picture, void *const *planes){ } void video_display(void* opaque, void* picture){ } unsigned video_format(void **opaque, char *chroma, unsigned *width, unsigned *height, unsigned *pitches, unsigned *lines) { return 1; } void video_cleanup(void* opaqe){ } int main(int argc, char** argv) { libvlc_instance_t* instance; libvlc_media_t* media; libvlc_media_player_t* player; libvlc_instance_t* instance2; libvlc_media_t* media2; libvlc_media_player_t* player2; const char* const options = "--sout=#duplicate{{dst=std{{access=udp,mux=ts,dst=127.0.0.1:35554}},dst=std{{access=udp,mux=ts,dst=127.0.0.1:35555}}}}"; instance = libvlc_new(1, &options); media = libvlc_media_new_location(instance, "udp://@239.8.6.8:5309"); player = libvlc_media_player_new(instance); libvlc_video_set_callbacks(player, video_lock, video_unlock, video_display, NULL); libvlc_video_set_format_callbacks(player, video_format, video_cleanup); instance2 = libvlc_new(0, NULL); media2 = libvlc_media_new_location(instance2, "udp://@127.0.0.1:35555"); player2 = libvlc_media_player_new(instance2); libvlc_video_set_callbacks(player2, video_lock, video_unlock, video_display, NULL); libvlc_video_set_format_callbacks(player2, video_format, video_cleanup); libvlc_media_player_set_media(player, media); libvlc_media_player_play(player); libvlc_media_player_set_media(player2, media2); libvlc_media_player_play(player2); for (int i = 0; i < 20; i++) { if (i % 2 == 0) { libvlc_media_player_stop(player2); } else { libvlc_media_player_play(player2); } } libvlc_media_release(media); libvlc_media_player_release(player); libvlc_release(instance); libvlc_media_release(media2); libvlc_media_player_release(player2); libvlc_release(instance2); return 0; }


I would expect this to run to completion, and sometimes it does. Other times it gets deadlocked on media_player_stop or libvlc_release. It seems like it has a better chance of deadlocking in Release mode without the visual studio debugger attached, but I've seen it deadlock with the debugger attached as well. Most of the time it appears like it deadlocked but it just takes way longer to execute than you would think (on the order of 30-45 seconds), but there are definitely still cases where does actually deadlock.

This is pretty much all we're doing in our managed code, so hopefully we're just doing something stupid :)

Rémi Denis-Courmont
Developer
Developer
Posts: 15268
Joined: 07 Jun 2004 16:01
VLC version: master
Operating System: Linux
Contact:

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby Rémi Denis-Courmont » 16 May 2015 11:35

The UDP input gets stuck on stop. This has nothing to do with having 2 instances, or with the streaming output for that matter.
Rémi Denis-Courmont
https://www.remlab.net/
Private messages soliciting support will be systematically discarded

whammy38
New Cone
New Cone
Posts: 8
Joined: 13 May 2015 20:47

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby whammy38 » 17 May 2015 01:47

Thanks for your reply. That makes sense and is consistent with what I've seen. Before this we were only using libvlc for streaming HTTP videos and never saw any deadlocks.

Rémi Denis-Courmont
Developer
Developer
Posts: 15268
Joined: 07 Jun 2004 16:01
VLC version: master
Operating System: Linux
Contact:

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby Rémi Denis-Courmont » 17 May 2015 11:19

It looks the same as this problem: https://forum.videolan.org/viewtopic.php?f=13&t=123937 albeit with UDP rather than a DVB tuner. Stop never returns if no data is received. This is also tracked at https://trac.videolan.org/vlc/ticket/13979
Rémi Denis-Courmont
https://www.remlab.net/
Private messages soliciting support will be systematically discarded

whammy38
New Cone
New Cone
Posts: 8
Joined: 13 May 2015 20:47

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby whammy38 » 18 May 2015 02:20

I did notice that calling stop when the player was playing a bogus video seemed to always deadlock, but in the case of the sample code I provided there was multicast video for that IP and port being streamed on the network when I did all my tests and it still occasionally deadlocked. So I think there's still a race condition that causes a deadlock even when UDP video is being received.

Rémi Denis-Courmont
Developer
Developer
Posts: 15268
Joined: 07 Jun 2004 16:01
VLC version: master
Operating System: Linux
Contact:

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby Rémi Denis-Courmont » 18 May 2015 08:40

In that case, you will have to produce a threaded symbolic stack trace by yourself. I cannot reproduce the issue.
Rémi Denis-Courmont
https://www.remlab.net/
Private messages soliciting support will be systematically discarded

whammy38
New Cone
New Cone
Posts: 8
Joined: 13 May 2015 20:47

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby whammy38 » 18 May 2015 20:51

I don't have a debug build of the version we're using so there are no debug symbols, but here are the stacks of the 2 threads that are deadlocked (there are 17 threads total for the sample I provided).

Main Thread (stuck on line 73: libvlc_release(instance); )

ntdll.dll!NtWaitForMultipleObjects() Unknown
KernelBase.dll!WaitForMultipleObjectsEx() Unknown
libvlccore.dll!00000000720348a4() Unknown
libvlccore.dll!0000000072035422() Unknown
libvlccore.dll!0000000071fe258d() Unknown
libvlc.dll!000000007224afde() Unknown
> VlcDeadlockTest.exe!main(int argc, char * * argv) Line 73 C++
VlcDeadlockTest.exe!__tmainCRTStartup() Line 626 C
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown

Worker Thread

> ntdll.dll!NtWaitForSingleObject() Unknown
mswsock.dll!SockWaitForSingleObject() Unknown
mswsock.dll!WSPSelect() Unknown
ws2_32.dll!select() Unknown
libvlccore.dll!00000000720a81f1() Unknown
libvlccore.dll!0000000072009fa8() Unknown
libudp_plugin.dll!0000000071e51512() Unknown
libvlccore.dll!0000000072035322() Unknown
msvcrt.dll!_callthreadstartex() Unknown
msvcrt.dll!_threadstartex() Unknown
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown

I have full stack traces for all 17 threads here. It's a ton of text so I didn't want to include it all here. I'll work on getting debug symbols next.

Rémi Denis-Courmont
Developer
Developer
Posts: 15268
Joined: 07 Jun 2004 16:01
VLC version: master
Operating System: Linux
Contact:

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby Rémi Denis-Courmont » 18 May 2015 21:01

The UDP input stuck on select() does not look like a different problem to me.
Rémi Denis-Courmont
https://www.remlab.net/
Private messages soliciting support will be systematically discarded

whammy38
New Cone
New Cone
Posts: 8
Joined: 13 May 2015 20:47

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby whammy38 » 18 May 2015 21:40

Alright if you're convinced then I am too. Any idea when/if this will be fixed? A fix for this would allow us to unify all our video playback under libvlc so we'd be willing to pay a bounty.

Rémi Denis-Courmont
Developer
Developer
Posts: 15268
Joined: 07 Jun 2004 16:01
VLC version: master
Operating System: Linux
Contact:

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby Rémi Denis-Courmont » 19 May 2015 09:13

In so far as it shows the UDP input stuck in select and the app's thread stuck waiting for something outside the UDP input, it looks like the bug I posted above. Now I realized I was rather dense. Based on the original report and analysis done by user "elevenfound", LibVLC locks dead when the application stops the media "too quickly" - before any data is read. Thus strictly speaking, it can happen even if there are incoming UDP datagrams. That fits very well with the test case that you provided: it just starts and stops the playback.

You can PM me if you need help with that.

However, I am not aware of any deadlock that could occur later, after playback has already started. Within the context of your application, it might be a different bug :-(
Rémi Denis-Courmont
https://www.remlab.net/
Private messages soliciting support will be systematically discarded

whammy38
New Cone
New Cone
Posts: 8
Joined: 13 May 2015 20:47

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby whammy38 » 19 May 2015 20:25

We can probably determine whether our problem is caused by the tracked bug you linked. We were just going to prevent our player from ever calling Stop if it has not received an event that playback has started (I think there is a callback that will tell me this). If that didn't work I was going to refactor our PInvokes to always run synchronously on a dedicated TaskScheduler that is definitely not pumping windows messages to try to eliminate that variable as well.

For now we are just going back to using the paid license video library we have for udp while libvlc will handle http and rtsp for us... just too much risk to take on at this time. We'll eventually get the time to re-visit this and determine if our problem is a known bug or not, and when we do we'll let you know one way or the other. Thanks for your insight on this issue!

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Please Advise: libvlc 3.0.0.0 deadlock when MediaPlayer plays sout from another libvlc instance

Postby Jean-Baptiste Kempf » 19 May 2015 23:16

We can probably determine whether our problem is caused by the tracked bug you linked. We were just going to prevent our player from ever calling Stop if it has not received an event that playback has started (I think there is a callback that will tell me this). If that didn't work I was going to refactor our PInvokes to always run synchronously on a dedicated TaskScheduler that is definitely not pumping windows messages to try to eliminate that variable as well.

For now we are just going back to using the paid license video library we have for udp while libvlc will handle http and rtsp for us... just too much risk to take on at this time. We'll eventually get the time to re-visit this and determine if our problem is a known bug or not, and when we do we'll let you know one way or the other. Thanks for your insight on this issue!
Don't hesitate to come back to us when you have the time.

Btw, nightly builds have debug versions.
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.


Return to “Development around libVLC”

Who is online

Users browsing this forum: No registered users and 42 guests