general RTSP protocol question

[thetoolwiz]

Hi, I hope this question isn't totally off-topic. I'm just not sure where to go for help. (Suggestions are welcome!)

A guy came to me and offered me some work taking a bit of code he'd written and "get it working".

Basically, it's a simple RTSP client and he's gotten it to the point where it plays a video stream (most of the time).

Now, please understand that I have no prior experience with RTSP, RTP, or video streaming at all. I'm just a programmer who's being paid to figure this out. I've gone through a bunch of RFCs and a dozen or more websites, but there's just not much I've found to help programmers in this arena.

The problem with this app is, there's no audio yet. That's what he wants me to get working, I guess.

I've been using VLC to open and test various links that I've found around the internet, and I have to say, working links seem to be rare as hens' teeth.

A list of 100 was published back in 2005 or so, and it has been making the rounds since then. Virtually none of them are working now, in VLC anyway. I've found a couple that work, so I'm using them for testing. (Got any others I can test? I'd particularly like to find some live rtsp feeds to test. There are TONS of live feeds in web pages, but they're all wrapped up in crazy flash or JS frames.)

I've tried looking through the VLC source code, but I'm not having a lot of luck finding what I need.

Basically, here's what I'm trying to understand.

When an app like VLC connects to an RTSP server, it initiates a session.

It sends out an OPTIONS request, then a DESCRIBE request, then a SETUP request to get a SESSION-ID.

Finally, it sends PLAY, PAUSE, and TEARDOWN requests using the SESSION-ID.

The thing I'm not sure of is whether the video and audio are muxed into the same RTP stream, or if they're delivered separately?

If they're separate, then at what point do you extract them? Do you need separate SESSIONS? Or the same session but separate tracks? or ... what?

All the examples I've found seem to be oversimplified and only show getting a single video stream.

I'd like to find an example of how you go about connecting to a server via RTSP and getting a video (H.264/MPEG4) and mono (or stereo) audio streamed from it.

(I know that's what VLC does, but maybe there's something that explains how it works at the protocol level. Because trying to dissect the RFCs to implement a working client is a humonguous time sink....)

Thanks
-David

[Rémi Denis-Courmont]

No, see RFC2326 for details.

[thetoolwiz]

Wow, I have not seen that RFC referred to anywhere. It's got a lot of info I've been trying to find.

Thanks!
-David

[thetoolwiz]

After further studying, I'm still in a quandery.

I've able to demux the two data streams and feed the video stream to the video codec, and for some streams it displays an image stream. So that's some progress.

But what I can't seem to find is any discussion on how to set up the audio codec or identify the format of the audio data packets following the '$' in the RTP stream.

There's also a 'config=' parameter in the m= section for the audio channel. But as far as the RFCs are concerned this is either "codec-specific" or "out-of-band" material that isn't discussed anywhere.

Where can I find help for this kind of thing?

This is for a GUI-based app that has a codec built into it, not a command-line app that saves to a file.

(A client wants a custom embedded player for monitoring multiple live camera feeds. However, there's additional processing going on with each feed, so it's not like a simple multi-security camera setup.)

-David