H264 RTP Streaming + Packetization

[Jerrie85]

Hello all,
I've a few questions hehe for my first post
I've been interested in streaming H264 over RTP, and was wondering if VLC can stream H264 over RTP? I have a source H264 file, and i was wondering what the command line syntax was to play this h264 file over RTP to a destination computer?

I've been trying it myself, and i haaven't had too much luck - so i wrote a H264 framer and streamed it to VLC client myself, and it works ok, except for one problem

my h264 stream NAL units dont have any SEI timing information with them - hence, i dont know the playback rate to play it back on

any vlc developers, do you know if the program looks at the RTP timestamps ALL the time for decoding timing information when it comes to playing back H264 streams? Or is there a way i can make this PTS timestamp from nothign? Like perhaps the slices themselves?

Thanks

[The DJ]

VLC will ALWAYS look at the RTP timestamp to base the PTS information on. I don't believe the SEI unit is relevant here. I think it contains buffering information, not time info....

[Jerrie85]

yup SEI messages have a struct called picture_timing info, check the JVT spec - its from this that i can extrapolate the PTS

its ok, i fixed it; i was lucky enough that my video coming out had 1 slice per frame, so i knew every slice meant a new 1/30th of a second increment..works fine in VLC now

the only problem VLC has is, it cant seem to recognize my SPS parameter set when i inject it the first time (SPS is the first NAL unit in my video)...
unfortunately, video doesn't create another SPS unit again, so what i do is i inject the SPS again into the stream a little bit after the start...VLC doesn't like this and crashes!

[The DJ]

What version of VLC are you using?

0.9.0 (which currently is a bit in a broken state)
and 0.8.6 have a new updated h264 packet framer that should allow this just fine. It should also pick up the initial SPS/PPS.

It's interesting that you talk about slices and SEI. We have been trying to get H264 RTSP from VLC to QuickTime working, but are running into issues with multiple slices. Do you have any information about how this works/should work ?
I'm very interested in testing our H264 support together with you. I can make the changes in VLC that are required, I'm just a bit stuck atm with my knowledge on how h264 works. I think you could help exactly with that.

[Jerrie85]

hurray DJ! i've fixed almost all of my problems;
timing wise, i've just allocated 1/30th for PTS, and it works like a charm;
H264 decoding on VLC is just magic using RTSP streaming

as for your questions, yup i'd be glad to help

one of my next tasks is to get my streamer + h264 framer to work with quicktime and i think for that, my initial research suggests encapsulating NAL units into atoms and stuff (and hint tracks etc)

for your multiple slices, the individual slices have a slice group id, which will tell you a) how many slices are in the current frame, and using this, u can set their PTS (if i have 3 slices, i set them all to the same PTS)
and if u have an SEI picture msg, that will tell u more detailed timing information

i'm interested in your work on streaming to quicktime, how have you accomplished it thus far?

Thanks

[The DJ]

https://trac.videolan.org/vlc/browser/t ... _out/rtp.c

This file contains a RTP muxer for h264. It works mostly, but there are issues with multiple slices. QT will only show the first slice, no matter what we do.

[Jerrie85]

when it comes to showing multiple slices, are you ensuring the PTS of all the multiple slices are the same? If you find that 3 slices belong to the same frame, they have to get the same timing information otherwise, most programs screw it up i think

also, make sure your not dropping your PPS/SPS when you come across one...transmit them too

if you SEI timing msgs, then you can get even more detailed timing info from otherwise

i've gotten mines to work with quicktime today, but mine's is a very simple case where i have single slice = single frame, no SEI timing msgs

with quicktime, your SDP params have to be set in stone

[koo]

hello jerrie85!

if you dont mind, can i get the source code that you write?

i need your help.

my work is streaming h264 using rtp.

when i start a work, i have no knowledge both rtp and h264..

i did study h264 and rtp and find a vlc .

but vlc is so slow in my embded system.

i did try customize it , but hard to me.

plz help me.

sorry for my english,it is not my mother language.

thank you.

[Jerrie85]

Hello koo,
I unfortunately cant release source code due to company policies, but i can guide you; my framework uses the live.com livemedia library as the basis for streaming over RTP

my framer is very simple; based on the RFC spec, it finds discrete nal units (delimited by 0x000001), cuts it up using FU-A fragmentation, appends headers, timestamps and sends it to livemedia to send

thats all there is to it!

the rtp packetizer file that DJ sent also has exquisite details on how he does it, which is exactly what i mentioned before

[koo]

hello jerrie85

i should try it.

thank you.

[AvcRtp]

Hi, DJ and Jerry

I am new to videolan but managed to put 0.8.6 version on my redhat linux WS rel 4 box. I try to play a H264 TS file and it works. I saw the topic about H264 RTP packitize but could not find a way to active it from VLC gui. could you please help on this issue?

Thanks

[mrep]

iam a newbie to VLC, i just stumbled on this discussion thought of asking some doubts. iam trying to stream H264 video to a VLC player from RTSP/RTP server. I have a RTSP/RTP server,RTSP client and VLC player.
When the client sends the requests for a video file. The server divides the file into different RTP packets, right now i just have it simple as to extract NAL units with 0x000001 as delimiter. When i check the size of each NAL unit, some has size as small as 12 bytes and some has as big as 29520. I don't know whether iam doing something wrong here. To each NAL unit, i would append the RTP header.
Also, after i create these packets how would i stream it to the VLC client. Iam kind of stuck here.
Can anyone tell me if iam doing something wrong and help me how to stream these data and correct if iam wrong.

Eagerly waiting for a reply....

Thanks.

[mrep]

can anyone post a sample code for streaming the RTP packets to a VLC client?
badly stuck over here, greatly appreciate if anyone could help me out
Thanks a bunch....

[Jerrie85]

what are you stuck on? are you using VLC or some other RTP/RTSP library

[mrep]

Iam not using any RTSP/RTP library. I have written a RTSP/RTP server that would stream the video on client's request. For streaming, i look for the x000001 delimiter to get the NAL units and packetize these frames and send it over the RTP. Iam just trying to use VLC media player as a client that would would receive the stream from the server. After streaming the packets i donno how to receive the packets using a VLC Media Player.This is where iam stuck.
Also as i have asked before while extracting the NAL units using the delimiter x000001. The size of each NAL unit or the frame has varying sizes. some are as small as 12 bytes and some like 76k bytes. SO wanted to know, whether iam doing something wrong during packetizing.

Thanks.

[Jerrie85]

Please look at RFC 3984 and read it throughly - thats your bible
look specifically at single NAL unit packetization mode and FUA mode
single NAL unit mode is only if your NAL unit is < MTU packet size
otherwise, u'll have to fragment and then you have to use FUA

[mrep]

Thanks for the reply.
I followed the RFC 3984 to generate RTP packets.
My RTP Packets look like this.
RTPHeader
-----------
F|NRI| Type
NAL Unit
-------------
When i parse the video, i delimit NAL units based on the 0x000001. Should i include the startcode prefix too in the RTP packets.
AT the receiver end what do i need to do to play it on VLC media player?

Thanks again....

[Jerrie85]

first of all, that mode that you showed me is single nal unit mode - you CANT do that for all NAL units, (espicially IDR and SLICE ones, they're too big!)

you never include the start code prefix, and what your doing is right for single nal unit mode

first figure out what your max packet size is going to be...mine is like 1500 bytes

to play it in VLC, your RTSP server should have a url it can play, like
rtsp://192.168.202.33:8554/stream

[mrep]

Hi i have my developed my own RTSP Server and to packetize i have followed RFC 3984(single NALU mode and FU-A). When i acess my server using the url rtsp://<ip>:<port>/stream. I get rtsp OPTIONS msg at the server side. I send a response for this command but the server just hangs waiting for another request from the VLC client. There seems to be no ther request from the VLC. Do i have to take care of somethin here?

Thanks

[YaoPeter]

Hi Dj,

I just found the problem: rfc3984 states that the rtp marker bit should indicate end of an access unit, but not end of an NAL unit. VLC sets this wrongly in rtp_packetize_h264. Quicktime relies on this bit heavily. I've sent a bug report to the vlc-devel list to ask for a patch.

https://trac.videolan.org/vlc/browser/t ... _out/rtp.c

This file contains a RTP muxer for h264. It works mostly, but there are issues with multiple slices. QT will only show the first slice, no matter what we do.

[doandm]

when U stream H264 RTP Streaming suceccfully,which media player do use to play that h264 rtp streaming ?
VLC can not play that stream

[saulo]

Hi
Does anyone know if the RTP h264 demuxer is already implemented?? If so, in what version and modules??

Thank you!

[Jean-Baptiste Kempf]

In developpment versions.