Gigaset M740 AV DVB-T Transport Streams not processed

[T.Stroter]

Hello,

VLC 0.8.4 seems to have problems to decode the transfer streams generated by the Siemens Gigaset M740 AV DVB-T box.
On MacOS X (10.4) and Windows XP VLC does not display the files (see the attached logfiles). After processing the MPEG TS with MPEG StreamClip 1.5 on the Mac (removed the first frames) VLC was capable to show this stream :->.

I guess, VLC is a little bit critical about MPEG TS that do not start with the correct header data (just a guess) - Other programs like the "Elecard MPEG2 Decoder" on Windows or VLC on Linux seems not to have such a problem.


To verify:
you might use the following files from the FTP upload for verification:
a) GigasetM740AV-10s.ts (10 seconds, cropped)
b) GigasetM740AV.ts (1 minute)

Best regards,

Thomas Stroeter




Background:
I'am evaluating developing a streaming solution for the quite popular Gigaset M740 box. The MPEG TS files are currently saved as single files on a network share. It would be nice, to use the vlc to view these files on different computers delivered by a webapplication without much converting :->








Logfile VLC 0.8.4 Siemens GigasetM740AV-10s.ts (cropped by MPEG Streamclip 1.5)
--------------------------------------------------------------------------------------------------------

main debug: CPU has capabilities AltiVec FPU
main debug: looking for memcpy module: 2 candidates
main debug: using memcpy module "memcpyaltivec"
main debug: waiting for thread completion
main debug: thread 41980416 (playlist) created at priority -47 (src/playlist/playlist.c:183)
main debug: waiting for thread completion
main debug: thread 42162176 (preparser) created at priority -47 (src/playlist/playlist.c:205)
main debug: looking for interface module: 1 candidate
main debug: using interface module "hotkeys"
main debug: interface initialized
main debug: thread 42747392 (interface) created at priority -47 (src/interface/interface.c:178)
main debug: looking for interface module: 3 candidates
main debug: using interface module "macosx"
main debug: interface initialized
main debug: thread 42759680 (manager) created at priority -47 (src/interface/interface.c:152)
main debug: adding playlist item `GigasetM740AV-10s.ts' ( /Users/stroeter/Desktop/GigasetM740AV-10s.ts )
main debug: creating new input thread
main debug: waiting for thread completion
main debug: thread 43335680 (input) created at priority 37 (src/input/input.c:230)
main debug: `/Users/stroeter/Desktop/GigasetM740AV-10s.ts' gives access `' demux `' path `/Users/stroeter/Desktop/GigasetM740AV-10s.ts'
main debug: creating demux: access='' demux='' path='/Users/stroeter/Desktop/GigasetM740AV-10s.ts'
main debug: looking for access_demux module: 1 candidate
main debug: creating access '' path='/Users/stroeter/Desktop/GigasetM740AV-10s.ts'
main debug: looking for access2 module: 4 candidates
vcd debug: trying .cue file: /Users/stroeter/Desktop/GigasetM740AV-10s.cue
access_file debug: opening file `/Users/stroeter/Desktop/GigasetM740AV-10s.ts'
main debug: using access2 module "access_file"
main debug: pre buffering
main debug: received first data for our buffer
main debug: prebuffering done 1408981 bytes in 0s - 133653 kbytes/s
main debug: creating demux: access='' demux='' path='/Users/stroeter/Desktop/GigasetM740AV-10s.ts'
main debug: looking for demux2 module: 39 candidates
main debug: using demux2 module "ts"
ts debug: DEMUX_SET_GROUP 0 0x0
main debug: `/Users/stroeter/Desktop/GigasetM740AV-10s.ts' successfully opened
ts debug: PATCallBack called
ts debug: new PAT ts_id=6144 version=4 current_next=1
ts debug: * number=66 pid=4384
ts debug: pid[4385] unknown
ts debug: pid[4386] unknown
ts debug: pid[4387] unknown
ts debug: eof ?
main debug: EOF reached
macosx debug: input has changed, refreshing interface
main debug: closing input
ts debug: pid list:
ts debug: - pid[0] seen
ts debug: - pid[4384] seen
ts debug: - pid[4385] seen
ts debug: - pid[4386] seen
ts debug: - pid[4387] seen
ts debug: - pid[8191] seen
main debug: unlocking module "ts"
main debug: unlocking module "access_file"
main debug: thread 43335680 joined (src/input/input.c:386)
main: nothing to play





Logfile#2: original/uncropped transport stream from Siemens Gigaset M740AV DVB-T Reciever

main debug: CPU has capabilities AltiVec FPU
main debug: looking for memcpy module: 2 candidates
main debug: using memcpy module "memcpyaltivec"
main debug: waiting for thread completion
main debug: thread 42031104 (playlist) created at priority -47 (src/playlist/playlist.c:183)
main debug: waiting for thread completion
main debug: thread 42161664 (preparser) created at priority -47 (src/playlist/playlist.c:205)
main debug: looking for interface module: 1 candidate
main debug: using interface module "hotkeys"
main debug: interface initialized
main debug: thread 42746880 (interface) created at priority -47 (src/interface/interface.c:178)
main debug: looking for interface module: 3 candidates
main debug: using interface module "macosx"
main debug: interface initialized
main debug: thread 42763776 (manager) created at priority -47 (src/interface/interface.c:152)
main debug: adding playlist item `GigasetM740AV.ts' ( /Volumes/HOMEGROUP;MEDIA/GigasetM740AV.ts )
main debug: creating new input thread
main debug: waiting for thread completion
main debug: thread 43269632 (input) created at priority 37 (src/input/input.c:230)
main debug: `/Volumes/HOMEGROUP;MEDIA/GigasetM740AV.ts' gives access `' demux `' path `/Volumes/HOMEGROUP;MEDIA/GigasetM740AV.ts'
main debug: creating demux: access='' demux='' path='/Volumes/HOMEGROUP;MEDIA/GigasetM740AV.ts'
main debug: looking for access_demux module: 1 candidate
main debug: creating access '' path='/Volumes/HOMEGROUP;MEDIA/GigasetM740AV.ts'
main debug: looking for access2 module: 4 candidates
vcd debug: trying .cue file: /Volumes/HOMEGROUP;MEDIA/GigasetM740AV.cue
access_file debug: opening file `/Volumes/HOMEGROUP;MEDIA/GigasetM740AV.ts'
macosx debug: input has changed, refreshing interface
main debug: using access2 module "access_file"
main debug: pre buffering
main debug: received first data for our buffer
main debug: prebuffering done 262136 bytes in 0s - 501 kbytes/s
main debug: creating demux: access='' demux='' path='/Volumes/HOMEGROUP;MEDIA/GigasetM740AV.ts'
main debug: looking for demux2 module: 39 candidates
main debug: using demux2 module "ts"
ts debug: DEMUX_SET_GROUP 0 0x0
main debug: `/Volumes/HOMEGROUP;MEDIA/GigasetM740AV.ts' successfully opened
ts debug: PATCallBack called
ts debug: new PAT ts_id=6144 version=4 current_next=1
ts debug: * number=66 pid=4384
ts debug: pid[4385] unknown
ts debug: pid[4386] unknown
ts debug: pid[4387] unknown
main debug: control type=0
main debug: control: stopping input
main debug: closing input
ts debug: pid list:
ts debug: - pid[0] seen
ts debug: - pid[4384] seen
ts debug: - pid[4385] seen
ts debug: - pid[4386] seen
ts debug: - pid[4387] seen
ts debug: - pid[8191] seen
main debug: unlocking module "ts"
main debug: unlocking module "access_file"
main debug: thread 43269632 joined (src/input/input.c:386)

[The DJ]

VLCs behaviour is correct. the files are not according to spec.
ProjectX analyzer says:

Thursday, January 5, 2006 4:26:05 PM CET
ProjectX 0.90.3.00 (31.12.2005)

<<< session infos >>>

-> working with collection 0

-> save normal log file
-> write all video data
-> write all other data
-> patch c.d.flagged infos of pictures
-> add sequence end code
-> set resolution in SDE
-> PVA: strictly specs. for audio streams
-> VOB: determine diff. Cell timelines
-> TS: ignore scrambled packets
-> TS: enhanced search for open packets
-> TS: join file segments (of Dreambox®)
-> TS: generate PMT stream dependent
-> get only enclosed PES/TS packets
-> concatenate different recordings
-> ensure 1st PES-packet start with video
-> generate PCR/SCR from PTS

-> write output files to: '/Users/hartman/Desktop'

-> Input File 0: '/Users/hartman/Desktop/GigasetM740AV.ts' (6,191,960 bytes)
-> Filetype is TS (generic PES Container)
-> demux
-> Service ID 0x2
-> PMT 0x1120 refers to these usable streams:
Video:
PID: 0x1121(#1)
Audio:
PID: 0x1122(ger)(#2)
PID: 0x1123(fre)(#4)
Teletext:
PID: 0x1124(ger_i100 )(#3)
Subpict.:
n/a

--> PID 0x0 (PAT) (0 #1) -> ignored
--> PID 0x1120 (PMT) (188 #2) -> ignored
ok> PID 0x1121 has PES-ID 0xE0 (MPEG Video) (3384 #19)
ok> PID 0x1122 has PES-ID 0xC0 (MPEG Audio) (13536 #73)
ok> PID 0x1123 has PES-ID 0xC0 (MPEG Audio) (20680 #111)
-> video basics: 544*576 @ 25fps @ 0.6735 (4:3) @ 7000000bps, vbvBuffer 112
-> starting export of video data @ GOP# 0
!> dropping useless B-Frames @ GOP# 0 / new Timecode 00:00:00.000
!> incomplete ts packet at EOF detected @ 6191780
packs: 32855 100% 6191960

-> Video: fr/ ct/ 1p/ cg/ og/ dg -> 478/ 1/ 0/ 40/ 0/ 0
-> Video length: 478 frames @ 00:00:19.120
-> GOP summary: min. 20, max. 24 fields; contains interlaced frames
-> avg. nom. bitrate 2138376bps (min/max: 1294400/3204800)
-> set first sequenceheader bitrate to 3204800bps
---> new File: /Users/hartman/Desktop/GigasetM740AV.m2v

--> MPEG Audio (0xC0) on PID 0x1122
-> check CRC of AC-3 / MPEG-Audio L1,2
-> delete CRC in MPEG-Audio Layer1,2
-> add frames
Audio PTS: first packet 05:55:35.489, last packet 05:55:54.809
Video PTS: start 1.GOP 05:55:35.962, end last GOP 05:55:55.082
-> adjusting audio at video-timeline
-> src_audio: MPEG-1, Layer2, 48000Hz, stereo, 192kbps, CRC @ 00:00:00.000
!> 7 frame(s) (168ms) added @ 00:00:18.960
audio frames: wri/pre/skip/ins/add 797/0/0/0/7 @ 00:00:19.128 done...
---> new File: '/Users/hartman/Desktop/GigasetM740AV.mp2'

--> MPEG Audio (0xC0) on PID 0x1123
-> check CRC of AC-3 / MPEG-Audio L1,2
-> delete CRC in MPEG-Audio Layer1,2
-> add frames
Audio PTS: first packet 05:55:35.577, last packet 05:55:54.897
Video PTS: start 1.GOP 05:55:35.962, end last GOP 05:55:55.082
-> adjusting audio at video-timeline
-> src_audio: MPEG-1, Layer2, 48000Hz, jstereo, 128kbps, CRC @ 00:00:00.000
!> 3 frame(s) (72ms) added @ 00:00:19.056
audio frames: wri/pre/skip/ins/add 797/0/0/0/3 @ 00:00:19.128 done...
---> new File: '/Users/hartman/Desktop/GigasetM740AV[1].mp2'

summary of created media files:
.Video (m2v): 478 Frames 00:00:19.120 '/Users/hartman/Desktop/GigasetM740AV.m2v'
Audio 0 (mp2): 797 Frames 00:00:19.128 0/0/0/7 '/Users/hartman/Desktop/GigasetM740AV.mp2'
Audio 1 (mp2): 797 Frames 00:00:19.128 0/0/0/3 '/Users/hartman/Desktop/GigasetM740AV[1].mp2'
=> 5,875,840 bytes written...
-> we have 4 warnings/errors.

So one of the TS packets in the stream is broken. causing VLC to error out with an EOF error.

[T.Stroter]

Hello Derek, thank for your fast check - I'am not an expert in TS encoding, so is it possible, that it is the firstor last package of the stream that causes VLC not display the stream? Or more conctrete what exactly must be fixed in the stream to make VLC display it?

[The DJ]

I looked a bit more at the file, and i think i found the issue.
the EOF is just the normal end of file. It's not the problem. the unfinished TS frame before the EOF is not really nice, but isn't a big issue either.
The real problem is that the file does not have a PMT table/cannot find the PMT table. I'm not sure why yet.

[The DJ]

We spent quite some time to look at your issue

Code: Select all

ts debug: new PAT ts_id=6144 version=4 current_next=1 ts debug: * number=66 pid=4384

So this says the PMT for program 66 has PID 4384
However the sample has only one PMT (@ byteoffset 188 like ProjectX said.) We decoded the PMT with hexdump by hand, and the PMT there says it describes program 2.
VLC is much more strict in these kinds of things, since it also needs to handle streams with multiple programs for instance, which can have multiple PMTs with the same PID describing different programs.

So the conclusion is that the program# in either the PAT or the PMT is incorrect causing a mismatch between the two, which is correctly handled by VLC. it's either 66 or 2, but not both. The system that produces these streams needs to be fixed.

DJ

[T.Stroter]

Thank you DJ for that detailed check !

I will try to check, if the wrong program# is stored in the original TS on the box or caused by the converting program. (I guess the first one would be the case, because the files directly taken from the box arent displayable either - that would mean that the Siemens Gigaset developer have to fix a bug...)

Thomas

[The DJ]

And if so, then it's VERY important they know about this, so we don't have a trilion devices with bugged TS.
Seeing how many ppl test DVB-H with VLC, i'm surprised they didn't notice it so far.

[petgun]

And if so, then it's VERY important they know about this, so we don't have a trilion devices with bugged TS.
Seeing how many ppl test DVB-H with VLC, i'm surprised they didn't notice it so far.

i'm not surprised...it is ignorance...similar to 'Neutrino-TS'...they know about it viewtopic.php?t=9828&highlight= but nothing changed until today

[T.Stroter]

OK, now the bug is reported to the NOW Portal as Bug#763 http://now-portal.c-lab.de/tracker/?fun ... 3&atid=134
and the Gigaset info forum http://www.m740.de/forum/showthread.php ... #post39151

[petgun]

OK, now the bug is reported to the NOW Portal as Bug#763 http://now-portal.c-lab.de/tracker/?fun ... 3&atid=134
and the Gigaset info forum http://www.m740.de/forum/showthread.php ... #post39151

...please let me/us know if/when they fix it.

[The DJ]

@T.Stroter:

I would like to reiterate that VLC will NOT be more fault tolerent in this area. VLC needs to be able to handle MPEG-TS streams with multiple programs in it, and having a correct PAT/PMT combination is critical for this support. We cannot compromise and support incorrect PAT/PMT combinations as that would break our support for multiprogram TS.

Also, this is the only issue preventing VLC from playing it. VLC just ignores the streams, because the program isn't properly selected. It should be easy to fix.

[T.Stroter]

BTW: is there a good specification/reference on the web how to encode the PAT/PMT packages? I googled a lot references how the MPEG-2 TS packages are encoded but no details about the encoding of the PAT/PMT packages.

[The DJ]

It's in the MPEG2-TS systems ISO - specification. (ISO specs are not publicly available). But libdvbpsi knows how to produce them.

These files have the tables and the functions to read/write them.
libdvbpsi/src/tables/pat.h
libdvbpsi/src/tables/pmt.h

[T.Stroter]

Thanks DJ for the info - whoever else needs a quick fix for the wrong encoded TS files on the M740 STB:

I have placed a small JAVA utility on the M740 Forum:

http://www.m740.de/forum/showthread.php ... ge=2&pp=10

Regards,

Thomas Stroeter

[T.Stroter]

OK, to close this request - here is my result of this discussion

homepage:
http://cowfishserver.sourceforge.net

download:
http://sourceforge.net/projects/cowfishserver

Code: Select all

Cowfish 0.7.1 released ====================== You can use the Cowfish Streamingserver to view movies recorded by the Gigaset 740/750 STBs on a VideoLan Client without any conversion. Features: * Web based GUI - Use your WebBrowser to access the server. * You can navigate (skip forwards/backwards) the stream during playback. * Delete a record (only on the server computer) * Multiuser support: Several users can watch movies at the same time. * Display the EPG information of the records. * Freeware / Open Source program :-), hosted at SourceForge * Supports init- and exit scripts to mount smb network shares on MacOS or Linux. * Pure Java Server - runs on Windows, MacOS X and Linux - very simple to start and install. Restrictions: The server must access the data via a windows network share or a local directory. FTP is NOT SUPPORTED, yet. Enjoy!