Creating a new overlay filter using metadata from RTSP

[solud]

I've started a new project that involves a video device that streams using RTSP along with a standards-compliant metadata stream; it is standards-compliant in the sense that RTSP players like VLC/live555 will just drop the data unless they understand what to do with it. I have integrated libVLC into my software but I now need libVLC to be able to use this metadata, which is just a bunch of metrics about the scene, and be able to paint an overlay onto the video stream on-the-fly. The metadata comes in its own UDP packet, easily found using a packet sniffer.

Before I go messing around with VLC code I want to lay out how I think this plan might proceed. I'm assuming that I'm going to want to create a new URL scheme like CUSTOM:// so that VLC will know that I'd be accessing my metadata-infused stream. Then I think I'd need to copy over the code that the RTSP:// scheme uses while adding the extra bits that would parse the metadata packets and then paint the overlay.

Does anyone have experience in doing this with VLC? Where should I start looking?

Thank you!

[solud]

Console output from Live555's openRTSP tool shows that there are two tracks for this stream: video/H264 and control/X-HREF. The second track isn't setup by openRTSP as it outputs "Unable to create receiver for "control/X-HREF" subsession: RTP payload format unknown or not supported."

From what I've looked at so far, I'll need to modify the live555 bits of VLC to setup this track. It looks to me that this would be started in \modules\demux\live555.cpp.

Am I on the right track? (no pun intended)

[Sébastien Escudier]

this is a message from the live555 library, so you would also need to modify this library.