VLC RTP/MPEG-TS live stream time display

[amartin]

Hi,

I run VLC playing a RTP + MpegTS + H264 stream (provided by a simple Gstreamer pipeline) represented by an SDP, but the displayed time is always 00:00.

First of all, I have already test it with different VLC versions. Now I am using VLC 2.0.3 where the issue persists.

I am live streaming a UDP + MPEGTS + H264 with the following pipeline:
gst-launch --gst-debug=mpegtsmux:5 videotestsrc do-timestamp=true is-live=true ! video/x-raw-yuv, framerate=(fraction)30/1 ! ffmpegcolorspace ! timeoverlay font-desc="Verdana bold 50px" ! ffmpegcolorspace ! queue ! x264enc byte-stream=true bitrate=500 key-int-max=5 tune=zerolatency ! video/x-h264 ! mpegtsmux name=mux m2ts-mode=false pat-interval=3000 pmt-interval=3000 ! video/mpegts ! video/mpegts ! rtpmp2tpay pt=33 ! udpsink port=5004 host=127.0.0.1 sync=false enable-last-buffer=false

According to the VLC staff VLC always uses the MPEG-TS PCR when dealing with MPEG-TS, including over RTP, ignoring the RTP timestamp.

So the timing capability lays on the PCR data of the MPEG-TS. I have already checked that the PCR data is inside the stream with "mpegtsmux" pluging of gstreamer. It implements the code described in the following post:
http://stackoverflow.com/questions/6199 ... r-from-pts

I have also checked that PCR data is inside the stream with a MPEG-2 Transport Stream packet analyser (http://www.pjdaniel.org.uk/mpeg/)

I checked that the TimeStamps are there with the next pipeline:
gst-launch --gst-debug=2 udpsrc uri=udp://localhost:5004 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)MP2T-ES, payload=(int)33" ! gstrtpjitterbuffer latency=200 ! application/x-rtp ! rtpmp2tdepay ! video/mpegts ! mpegtsdemux name=demux ! video/x-h264 ! queue ! decodebin name=dec ! ffmpegcolorspace ! timeoverlay font-desc="Verdana bold 50px" valignment=1 ! directdrawsink

I tried also to send the PCR inside the Adaptation field with the first packet of each new TimeStamp (30fps = TS increments of 3000 for RTP 90000khz) by hacking mpegtsmux plugin... displayed time "00:00"

VLC open the SDP file that contains the following data:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 127.0.0.1
t=0 0
a=tool:GStreamer
a=type:broadcast
a=range:npt=now-
m=video 5004 RTP/AVP 33
a=rtpmap:33 MP2T/90000
a=control:mpegts-transport

The time is not displayed by the VLC (always 00:00). The expected result is a displayed time equal to the one "watermarked" inside the video. This means if the stream is running for 30seconds when VLC player connects it, the VLC displayed time should be "00:30".

I have also tried to modify "a=range:npt=now-" to other values, but the result is the same... "00:00"

Some idea/tip about what's wrong? / How to fix it?

Thank you in advance.

[Jean-Baptiste Kempf]

The only way to fix it would be to work on the code of the TS demuxer, at this point.

[amartin]

After preliminary commit review related to PCR there are some commits that could be related to this issue:

1) Fixed a big rtp demuxer: the problems with TS over RTP (for TS with small pcr/pts delay that is, not created by VLC)
http://mailman.videolan.org/pipermail/v ... 49246.html
2) Enable seek base on PCR in the TS demuxer only when the stream can seek fast
http://mailman.videolan.org/pipermail/v ... 09538.html
3) Simplify/fix checks enabling the use of pcr for seeking/ positioning in the TS demuxer
http://mailman.videolan.org/pipermail/v ... 09439.html

1) This commit could highlight that it is necessary bigger pcr/pts delay (send PCR is calculated with a offset from the current PTS)
Note that there should be a time-offset between the PTSs of the multiplexed streams and the PCR, it's usually in the range of a few hundred ms, depending on the stream. In this case the solution is just increase this delay inside mpegtsmux plugin.

2) The change is "stream_Control( p_demux->s, STREAM_CAN_FASTSEEK, &can_seek );". I do not know what stream_Control does. Does it establish in someway that only fast seekable streams can be seek? and just for them current time and duration is faced? If this is true (i will go in further detail in the source code), the time of a live stream could not be displayed at all.

3) This change suggest to "Seek based on percent not time" "Seek and position based on a percent byte position, not a PCR generated time position. If seeking doesn't work propery, turn on this option."

I will explore each path.

Thank you

[amartin]

I achieved it, but I supposed I broke the correct behaviour for other streams.

After going into further details inside the source code:

A) the issue is Live555 demux deals with RTSP/RTP/UDP live streams, but when the "MainLoopInterface" that updates the variables used by the interfaces perform the "demux_Control" call for "DEMUX_GET_TIME" the Live555 demux manages them putting aside the PCR information brought by inner muxer MPEG-TS (in this case). These calls are not transfered to inner TS demux to exploit PCR information that is properly managed.
I am not able to find how to access to "psz_name" or "b_muxed" in "MainLoopInterface" to check inner muxed streams to switch off/disable "es_out_SetTimes".

B) "DStreamThread" function performs DEMUX_GET_POSITION both for position and time ¿bug? (instead of DEMUX_GET_TIME) but the retrieved figures are used just for stats.
Here I am able to access to "psz_name" in to check inner muxed streams to enable/perform "es_out_SetTimes" call with PCR MPEG-TS data.

C) Only TS streams, where TS demux can retrieve "i_first_pcr" and "i_last_pcr" from the stream, are managed with time seek.
So live streams are always managed by percentage (percentage/position is enabled at many places in the code if "i_first_pcr" and/or "i_last_pcr" are not present, not taking into account the VLC advanced preferences setup).
I think that the first and last PCR values are available only for TS files not MPEG-TS live streams. Moreover, in TS demux the time answered to DEMUX_GET_TIME is relative from first_pcr, instead of just return current_pcr (live time from the server)

To sum up these are just some clues for people that require this feature now.
My feeling is that TIMING from MPEG-TS live streams should be reviewed because nowadays it is mainly focused on offline TS files. The PCR reading and management is in the code but it is not used at all in the VLC display.
From my VLC code knowledge is quite hard to successfully modify the code because I was not able to access structures that keep inner information about containing muxed streams.

[Rémi Denis-Courmont]

Use the native RTP input from VLC version 2.1. Only it gets timing from the MPEG-TS demuxer.

[amartin]

Even in this situation I think issue C) would persist

Use the native RTP input from VLC version 2.1. Only it gets timing from the MPEG-TS demuxer.

[Rémi Denis-Courmont]

I don't even understand what that problem is supposed to mean.

VLC has started as a MPEG-TS broadcast receiver, and it has always worked very well at that.

[amartin]

The problem:

*Server: Live streaming UDP + RTP + MPEGTS + H264 to 127.0.0.1 (localhost) port 5004.

*Client: VLC with SDP described in a previous post.

--Issue: Time displayed 00:00--
In the VLC GUI at the bottom left and right corner (begin and end of seek bar) it is displayed always and all the time 00:00 although PCR MPEGTS timestamps are in the stream.

A)
VLC updates the variables used by the interfaces
input.c -> static void MainLoopInterface( input_thread_t *p_input )
ask for time:
if( demux_Control( p_input->p->input.p_demux, DEMUX_GET_TIME, &i_time ) )
update variables:
es_out_SetTimes( p_input->p->p_es_out, f_position, i_time, i_length );

Request is managed by Live555 demux
live555.cpp -> static int Control( demux_t *p_demux, int i_query, va_list args )

The result is 0, Live555 demux does not manage PCR timestamps, but Live555 knows they are there
live555.cpp -> static int SessionsSetup( demux_t *p_demux )
tk = (live_track_t*)malloc( sizeof( live_track_t ) );
if( !strcmp( sub->codecName(), "MP2T" ) )
{
tk->b_muxed = true;
tk->p_out_muxed = stream_DemuxNew( p_demux, "ts", p_demux->out );
}

B)
Time from PCR MPEGTS timestamps are never retrieved
stream_demux.c -> static void* DStreamThread( void *obj )
if( demux_Control( p_demux, DEMUX_GET_POSITION, &newpos ) )
newpos = 0.;
if( demux_Control( p_demux, DEMUX_GET_LENGTH, &newlen ) )
newlen = 0;
if( demux_Control( p_demux, DEMUX_GET_POSITION, &newtime ) ) // <- BUG?
newtime = 0;
Instead of ask for time it ask for position for the TS demux
Note: I moved the call to es_out_SetTimes from MainLoopInterface to DStreamThread

C.1)
Percent seek feature is set to ON "always", setting OFF time seek
ts.c -> static int Open( vlc_object_t *p_this )
bool can_seek = false;
stream_Control( p_demux->s, STREAM_CAN_FASTSEEK, &can_seek ); //<-- For RTP and UDP can_seek is always FALSE
//rtp.c -> static int Control (demux_t *demux, int query, va_list args) does not define "case" for STREAM_CAN_FASTSEEK just for DEMUX_CAN_SEEK
//udp.c -> static int Control( access_t *p_access, int i_query, va_list args ) returns *pb_bool = false;
if( can_seek ) // <- always false. BUG?
{
GetFirstPCR( p_demux ); // It is never retrieved
CheckPCR( p_demux ); // It is never retrieved
GetLastPCR( p_demux ); // It is never retrieved
}
if( p_sys->i_first_pcr < 0 || p_sys->i_last_pcr < 0 ) <-- It is always true
{
p_sys->b_force_seek_per_percent = true;
}

C.2) Once B) is fixed
Time is always related to the beginning of the playing action instead of just formatting timestamps
ts.c -> static int Control( demux_t *p_demux, int i_query, va_list args )
case DEMUX_GET_TIME:
pi64 = (int64_t*)va_arg( args, int64_t * );
if( (p_sys->b_dvb_meta && p_sys->b_access_control) ||
p_sys->b_force_seek_per_percent ||
p_sys->i_current_pcr - p_sys->i_first_pcr < 0 )
{
if( DVBEventInformation( p_demux, pi64, NULL ) )
{
*pi64 = 0;
}
}
else
{
*pi64 = (p_sys->i_current_pcr - p_sys->i_first_pcr) * 100 / 9; // WHY THIS WAY? always from 00:00
// *pi64 = (p_sys->i_current_pcr) * 100 / 9; <-- This is for me more correct, displaying server time (timestamps), no playing time
}
return VLC_SUCCESS;

It is now more clear?