Multiple simultaneous RTSP Streams - Swift

[moyoteg]

Hi guys,

I am trying to have a Video Player instance inside a UICollectionVewCell but it is crashing.
My goal is to have at least 2 simultaneous RTSP Streams on screen on an iPad.
Is this even possible?

Thanks!

[Jean-Baptiste Kempf]

It should be possible, yes. Do you have a lldb backtrace?

[moyoteg]

It should be possible, yes. Do you have a lldb backtrace?

Here it is:

thread #1: tid = 0x4c6d26, 0x057e19a8 libobjc.A.dylib`bool objc::DenseMapBase<objc::DenseMap<DisguisedPtr<objc_object>, unsigned long, true, objc::DenseMapInfo<DisguisedPtr<objc_object> > >, DisguisedPtr<objc_object>, unsigned long, objc::DenseMapInfo<DisguisedPtr<objc_object> >, true>::LookupBucketFor<DisguisedPtr<objc_object> >(DisguisedPtr<objc_object> const&, std::__1::pair<DisguisedPtr<objc_object>, unsigned long> const*&) const + 18, queue = 'com.apple.main-thread'
frame #0: 0x057e19a8 libobjc.A.dylib`bool objc::DenseMapBase<objc::DenseMap<DisguisedPtr<objc_object>, unsigned long, true, objc::DenseMapInfo<DisguisedPtr<objc_object> > >, DisguisedPtr<objc_object>, unsigned long, objc::DenseMapInfo<DisguisedPtr<objc_object> >, true>::LookupBucketFor<DisguisedPtr<objc_object> >(DisguisedPtr<objc_object> const&, std::__1::pair<DisguisedPtr<objc_object>, unsigned long> const*&) const + 18
frame #1: 0x057e18c4 libobjc.A.dylib`objc::DenseMapBase<objc::DenseMap<DisguisedPtr<objc_object>, unsigned long, true, objc::DenseMapInfo<DisguisedPtr<objc_object> > >, DisguisedPtr<objc_object>, unsigned long, objc::DenseMapInfo<DisguisedPtr<objc_object> >, true>::find(DisguisedPtr<objc_object> const&) + 36
frame #2: 0x057dffd2 libobjc.A.dylib`objc_object::sidetable_release(bool) + 96
frame #3: 0x057df361 libobjc.A.dylib`objc_release + 65
frame #4: 0x057e052d libobjc.A.dylib`(anonymous namespace)::AutoreleasePoolPage::pop(void*) + 371
frame #5: 0x039588a8 CoreFoundation`_CFAutoreleasePoolPop + 24
frame #6: 0x0429a9c5 UIKit`_prepareForCAFlush + 370
frame #7: 0x0427095b UIKit`_UIApplicationHandleEventQueue + 8293
frame #8: 0x0399ae5f CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 15
frame #9: 0x03990aeb CoreFoundation`__CFRunLoopDoSources0 + 523
frame #10: 0x0398ff08 CoreFoundation`__CFRunLoopRun + 1032
frame #11: 0x0398f846 CoreFoundation`CFRunLoopRunSpecific + 470
frame #12: 0x0398f65b CoreFoundation`CFRunLoopRunInMode + 123
frame #13: 0x07ba6664 GraphicsServices`GSEventRunModal + 192
frame #14: 0x07ba64a1 GraphicsServices`GSEventRun + 104
frame #15: 0x04276eb9 UIKit`UIApplicationMain + 160
frame #16: 0x000096d1 PlayerVM`main + 145 at AppDelegate.swift:12
frame #17: 0x061e3a25 libdyld.dylib`start + 1

thread #2: tid = 0x4c6da9, 0x065237de libsystem_kernel.dylib`kevent64 + 10, queue = 'com.apple.libdispatch-manager'
frame #0: 0x065237de libsystem_kernel.dylib`kevent64 + 10
frame #1: 0x061a7034 libdispatch.dylib`_dispatch_mgr_invoke + 257
frame #2: 0x061a6d84 libdispatch.dylib`_dispatch_mgr_thread + 60

thread #8: tid = 0x4c6db0, 0x06522d5e libsystem_kernel.dylib`__workq_kernreturn + 10
frame #0: 0x06522d5e libsystem_kernel.dylib`__workq_kernreturn + 10
frame #1: 0x064e734b libsystem_pthread.dylib`_pthread_wqthread + 1289
frame #2: 0x064e4f56 libsystem_pthread.dylib`start_wqthread + 34

thread #9: tid = 0x4c6dbd, 0x065223ea libsystem_kernel.dylib`__psynch_cvwait + 10
frame #0: 0x065223ea libsystem_kernel.dylib`__psynch_cvwait + 10
frame #1: 0x064e8538 libsystem_pthread.dylib`_pthread_cond_wait + 757
frame #2: 0x064ea276 libsystem_pthread.dylib`pthread_cond_wait$UNIX2003 + 71
frame #3: 0x00cc5906 PlayerVM`Thread [inlined] LoopInput(p_playlist=<unavailable>) + 26 at thread.c:467
frame #4: 0x00cc58ec PlayerVM`Thread(data=0x0de318b4) + 1308 at thread.c:519
frame #5: 0x064e7780 libsystem_pthread.dylib`_pthread_body + 138
frame #6: 0x064e76f6 libsystem_pthread.dylib`_pthread_start + 155
frame #7: 0x064e4f7a libsystem_pthread.dylib`thread_start + 34

thread #10: tid = 0x4c6dbf, 0x0651b4d6 libsystem_kernel.dylib`semaphore_wait_trap + 10
frame #0: 0x0651b4d6 libsystem_kernel.dylib`semaphore_wait_trap + 10
frame #1: 0x061a308c libdispatch.dylib`_dispatch_thread_semaphore_wait + 25
frame #2: 0x0619b415 libdispatch.dylib`_dispatch_barrier_sync_f_slow + 480
frame #3: 0x061a0f56 libdispatch.dylib`_dispatch_barrier_sync_f + 78
frame #4: 0x0619bce0 libdispatch.dylib`_dispatch_barrier_sync_slow + 235
frame #5: 0x0619bbef libdispatch.dylib`dispatch_barrier_sync + 61
frame #6: 0x0000a708 PlayerVM`-[VLCEventManager startEventLoop](self=0x00000000, _cmd="startEventLoop") + 1738 at VLCEventManager.m:215
frame #7: 0x00009fd0 PlayerVM`EventDispatcherMainLoop(user_data=0x0f009380) + 33 at VLCEventManager.m:94
frame #8: 0x064e7780 libsystem_pthread.dylib`_pthread_body + 138
frame #9: 0x064e76f6 libsystem_pthread.dylib`_pthread_start + 155
frame #10: 0x064e4f7a libsystem_pthread.dylib`thread_start + 34

* thread #7: tid = 0x4c6f1c, 0x0622d160 libsystem_c.dylib`strlen + 16, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
frame #0: 0x0622d160 libsystem_c.dylib`strlen + 16
frame #1: 0x06288811 libsystem_c.dylib`strdup + 20
* frame #2: 0x00ce5ba3 PlayerVM`InputSourceInit(p_input=<unavailable>, in=<unavailable>, psz_mrl=<unavailable>, psz_forced_demux=<unavailable>, b_in_can_fail=<unavailable>) + 35 at input.c:2207
frame #3: 0x00ce0c24 PlayerVM`Init(p_input=<unavailable>) + 1316 at input.c:1212
frame #4: 0x00ce3c78 PlayerVM`Run(obj=0x0de1cb24) + 24 at input.c:520
frame #5: 0x064e7780 libsystem_pthread.dylib`_pthread_body + 138
frame #6: 0x064e76f6 libsystem_pthread.dylib`_pthread_start + 155
frame #7: 0x064e4f7a libsystem_pthread.dylib`thread_start + 34

[moyoteg]

.
Okay so I can have multiple RTSP streams no problem.
.
But for some reason I cannot determine.
Whenever I try to make a CollectionViewCell be drawable or contain a drawable video view the application crashes.

.

[fkuehne]

Do you create a subview in the content view of the cell to draw the video? Do you stop rendering before the cell is being recycled?

[fkuehne]

In fact, this post reminded me of code I did in 2014. You can see a demo video here: https://vimeo.com/174839654

[moyoteg]

Felix!
.
I got it to work! Danke!!
.
The issue that I am running into now is that I am dropping frames and so some of the streams are not showing?

[153c3274] avcodec decoder error: more than 5 seconds of late video -> dropping frame (computer too slow ?)

.
How can I improve performance? I have to hopefully be able to run this on iPad2s which are not very powerful...
.
Thanks in advance!

[fkuehne]

Which version of MobileVLCKit are you using?

[moyoteg]

Using MobileVLCKit (2.2.2)

[fkuehne]

Consider using a 3.0 pre-release then to use the hardware decoder. Should be quite a bit faster as long as you run iOS 8 and later and the video track is in H264.

[awr]

In fact, this post reminded me of code I did in 2014. You can see a demo video here: https://vimeo.com/174839654

Hi Felix, I'm trying to do something similar and the videos are displaying as expected. However, there seems to be a bug where only one of the videos resizes on rotate. Can you confirm if your example has the same issues.

Relevant code (I'm using a Xamarin wrapper around MobileVLCKit, version 2.2.0 RC):

Code: Select all

public partial class VideoCollectionViewCell : UICollectionViewCell, IVLCMediaPlayerDelegate { VLCMediaPlayer mediaPlayer; public VideoCollectionViewCell (IntPtr handle) : base (handle) { BackgroundView = new UIView { BackgroundColor = UIColor.Orange }; ContentView.BackgroundColor = UIColor.White; mediaPlayer = new VLCMediaPlayer(); mediaPlayer.Delegate = this; mediaPlayer.Drawable = ContentView; mediaPlayer.Media = new VLCMedia(NSUrl.FromString("rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov")); mediaPlayer.Play(); } [Export("mediaPlayerStateChanged:")] public void MediaPlayerStateChanged(Foundation.NSNotification notification) { Console.WriteLine("State changed to: {0}", mediaPlayer.GetState()); } }

Relevant controller code:

Code: Select all

public partial class VideoCollectionViewController : UICollectionViewController { List<string> videoUrls = new List<string>(); UICollectionViewFlowLayout FlowLayout; public VideoCollectionViewController (IntPtr handle) : base (handle) { videoUrls.Add(""); videoUrls.Add(""); videoUrls.Add(""); } public override void ViewDidLoad() { base.ViewDidLoad(); FlowLayout = new UICollectionViewFlowLayout(); FlowLayout.MinimumLineSpacing = 0; FlowLayout.MinimumInteritemSpacing = 0; this.CollectionView.CollectionViewLayout = FlowLayout; } public override void ViewDidLayoutSubviews() { base.ViewDidLayoutSubviews(); var size = CalculateSize(); // function that returns size depending on screen orientation. FlowLayout.ItemSize = size; } public override void ViewWillTransitionToSize(CGSize toSize, IUIViewControllerTransitionCoordinator coordinator) { base.ViewWillTransitionToSize(toSize, coordinator); } }

And the result:
Portrait


Landscape (notice only 1 of the vids resized correctly)

[moyoteg]

In fact, this post reminded me of code I did in 2014. You can see a demo video here: https://vimeo.com/174839654

I am using autoayout.

if you are using it, here is how you do it:

override func willRotateToInterfaceOrientation(toInterfaceOrientation: UIInterfaceOrientation, duration: NSTimeInterval) {
self.collectionView.collectionViewLayout.invalidateLayout()
}

Hi Felix, I'm trying to do something similar and the videos are displaying as expected. However, there seems to be a bug where only one of the videos resizes on rotate. Can you confirm if your example has the same issues.

Relevant code (I'm using a Xamarin wrapper around MobileVLCKit, version 2.2.0 RC):

Code: Select all

public partial class VideoCollectionViewCell : UICollectionViewCell, IVLCMediaPlayerDelegate { VLCMediaPlayer mediaPlayer; public VideoCollectionViewCell (IntPtr handle) : base (handle) { BackgroundView = new UIView { BackgroundColor = UIColor.Orange }; ContentView.BackgroundColor = UIColor.White; mediaPlayer = new VLCMediaPlayer(); mediaPlayer.Delegate = this; mediaPlayer.Drawable = ContentView; mediaPlayer.Media = new VLCMedia(NSUrl.FromString("rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov")); mediaPlayer.Play(); } [Export("mediaPlayerStateChanged:")] public void MediaPlayerStateChanged(Foundation.NSNotification notification) { Console.WriteLine("State changed to: {0}", mediaPlayer.GetState()); } }

Relevant controller code:

Code: Select all

public partial class VideoCollectionViewController : UICollectionViewController { List<string> videoUrls = new List<string>(); UICollectionViewFlowLayout FlowLayout; public VideoCollectionViewController (IntPtr handle) : base (handle) { videoUrls.Add(""); videoUrls.Add(""); videoUrls.Add(""); } public override void ViewDidLoad() { base.ViewDidLoad(); FlowLayout = new UICollectionViewFlowLayout(); FlowLayout.MinimumLineSpacing = 0; FlowLayout.MinimumInteritemSpacing = 0; this.CollectionView.CollectionViewLayout = FlowLayout; } public override void ViewDidLayoutSubviews() { base.ViewDidLayoutSubviews(); var size = CalculateSize(); // function that returns size depending on screen orientation. FlowLayout.ItemSize = size; } public override void ViewWillTransitionToSize(CGSize toSize, IUIViewControllerTransitionCoordinator coordinator) { base.ViewWillTransitionToSize(toSize, coordinator); } }

And the result:
Portrait


Landscape (notice only 1 of the vids resized correctly)