Capture frame from VLCKit on macos

iOS, iPad, iPhone, tvOS specific usage questions
ahmoylaw
New Cone
New Cone
Posts: 2
Joined: 29 Nov 2023 05:42

Capture frame from VLCKit on macos

Postby ahmoylaw » 29 Nov 2023 06:08

I'm new to VLCKit and have been playing with VLCKit on macos for few days already but still couldnt figure out on how to capture frame from video.

As far as i know, VLCKit only allows snapshot into file only but i need to process every frame and prefer in memory processing.

I have tried methods below but none of it will give me a correct NSImage, the capture Image is just blank.

1. Capture NSImage from NSView (by using NSView onto the VLCMediaPlayer.drawable)
2. Capture NSImage from VLCVideoLayer (assign created VLCVideoLayer and call setVideoLayer)
3. Replace addVoutLayer and removeVoutLayer methods inside VLCViewLayer (but unfortunately it seems like nobody will call it)

Any helps will be very much appreciated!

(sorry for my bad english)

Thanks!

fkuehne
Developer
Developer
Posts: 7264
Joined: 16 Mar 2004 19:37
VLC version: 0.4.6 - present
Operating System: Darwin
Location: Germany
Contact:

Re: Capture frame from VLCKit on macos

Postby fkuehne » 01 Dec 2023 10:31

What is your use-case exactly? do you want to manipulate the raw frames directly on the GPU?
VideoLAN
Felix Paul Kühne
Medic. VLC developer for appleOS since before you were born.
Blog: https://www.feepk.net

ahmoylaw
New Cone
New Cone
Posts: 2
Joined: 29 Nov 2023 05:42

Re: Capture frame from VLCKit on macos

Postby ahmoylaw » 02 Dec 2023 04:54

I need to capture every fame from a video stream (rtsp), feed it to my library (can be in NSImage format) to process for objects recognition and also detect any movement (mainly for human).

Thanks.

MystiqueWanderer
Blank Cone
Blank Cone
Posts: 49
Joined: 21 Nov 2023 13:02

Re: Capture frame from VLCKit on macos

Postby MystiqueWanderer » 04 Dec 2023 14:30

I think that instead of relying on VLCKit's high-level APIs, you could try to intercept the draw calls at a lower level. This could be done by subclassing NSView or VLCVideoLayer and overriding the relevant drawing methods. In these overridden methods, you can access the raw pixel data being drawn. But, I should warn you that this approach requires a good understanding of macOS's graphics systems.

fkuehne
Developer
Developer
Posts: 7264
Joined: 16 Mar 2004 19:37
VLC version: 0.4.6 - present
Operating System: Darwin
Location: Germany
Contact:

Re: Capture frame from VLCKit on macos

Postby fkuehne » 05 Dec 2023 12:59

You need access to the raw frames. For video decoded in software, we have a solution in version 3 of libvlc/VLCKit. For hardware decoding, you need the forthcoming version 4 to prevent major performance degradations.
VideoLAN
Felix Paul Kühne
Medic. VLC developer for appleOS since before you were born.
Blog: https://www.feepk.net

MystiqueWanderer
Blank Cone
Blank Cone
Posts: 49
Joined: 21 Nov 2023 13:02

Re: Capture frame from VLCKit on macos

Postby MystiqueWanderer » 05 Dec 2023 13:33

Oh, really? I didn't know about that. But thanks a lot for this info. I'll definitely try it.


Return to “VLC for iOS, iPadOS and Apple TV”

Who is online

Users browsing this forum: No registered users and 21 guests