Capture frame from VLCKit on macos

[ahmoylaw]

I'm new to VLCKit and have been playing with VLCKit on macos for few days already but still couldnt figure out on how to capture frame from video.

As far as i know, VLCKit only allows snapshot into file only but i need to process every frame and prefer in memory processing.

I have tried methods below but none of it will give me a correct NSImage, the capture Image is just blank.

1. Capture NSImage from NSView (by using NSView onto the VLCMediaPlayer.drawable)
2. Capture NSImage from VLCVideoLayer (assign created VLCVideoLayer and call setVideoLayer)
3. Replace addVoutLayer and removeVoutLayer methods inside VLCViewLayer (but unfortunately it seems like nobody will call it)

Any helps will be very much appreciated!

(sorry for my bad english)

Thanks!

[fkuehne]

What is your use-case exactly? do you want to manipulate the raw frames directly on the GPU?

[ahmoylaw]

I need to capture every fame from a video stream (rtsp), feed it to my library (can be in NSImage format) to process for objects recognition and also detect any movement (mainly for human).

Thanks.

[MystiqueWanderer]

I think that instead of relying on VLCKit's high-level APIs, you could try to intercept the draw calls at a lower level. This could be done by subclassing NSView or VLCVideoLayer and overriding the relevant drawing methods. In these overridden methods, you can access the raw pixel data being drawn. But, I should warn you that this approach requires a good understanding of macOS's graphics systems.

[fkuehne]

You need access to the raw frames. For video decoded in software, we have a solution in version 3 of libvlc/VLCKit. For hardware decoding, you need the forthcoming version 4 to prevent major performance degradations.

[MystiqueWanderer]

Oh, really? I didn't know about that. But thanks a lot for this info. I'll definitely try it.