Can't simply stream MPEG-TS (DVB Capture) without remuxing

[tomamoto]

I work in a cable lab and we often need to loop content that is not easy to acquire directly via satellite at any given time for testing services. Currently, all of our streaming is done off a Sencore platform, which, to keep it brief, I'd like to replace with an open-source linux platform that I will build myself.

As a basis for my platform, I want to prove that I can use VLC, from command line, without any remuxing, to take DVB captures (MPEG2/TS) that I already have and stream them over UDP multicast (RTP is out of the question as we have a many different modulators in our test-headend that simulate the actual setups of our customers and UDP is the only format that they universally accept).

Here is the basic command I am working with running VLC 2.2.2 on CentOS7:

cvlc /video/WOBI.ts --sout '#standard{access=udp,dst=233.8.8.8:888}'

And it kind of works, but there are some peculiarities that seem to be associated with the remuxing process. Chief among them is the fact that we seem to be losing something in the PMT/PAT (pardon amy vagueness as it's a result of my own ignorance). Here's my StreamXpert report on the VLC output when I analyze that same multicast stream:



Now here's what I'm using as a baseline. This is Sencore's output of that exact same file:



Things to Note

• There are more streams than just the Audio and Video.
• This capture definitely contains 2x MPEG2-Audio Streams and 1x AC3 Audio stream, so why does the VLC mux'd version show the MPEG2 audios streams as MPEG-1 Audio streams?
• The program/service number gets set to 1 i the VLC steam, even thought it's definitely 200 in the original capture.

Note: I've provided the mediainfo output at the bottom so you can see for yourself the details on the TS file I'm working with.

I want to just push the raw TS back out in the same form I captured it. Apparently, once upon a time, VLC had a --ts-out "Fast udp streaming" option that did exactly what I wanted, but it was removed for some reason in more recent versions. I've already looked into DVBlaster (doesn't take plain files as input) and multicat (doesn't output to raw UDP and also doesn't seem to take TS files that it didn't generate itself because of the whole .aux files thing) and I'm at a loss as to how I should be going about this. Am I missing some critical option when I set up my stream? Is there a more suitable open-source solution aside from VLC?

And I'd rather not dig up old versions of VLC to get the --ts-out option. I did a whole lot of searching and wasn't even able to find out WHY that was deprecated, but I want to ultimately work the with python interface of VLC and I don't want to struggle with documentation because I'm using an unsupported 5+ year-old version of the binaries.

Any help with this issue would be vastly appreciated! I know this is a lot of detials to ask what should otherwise be a simple question, but I wanted to make as much available as possible to anyone who might be able to help me. Thanks!





Mediainfo:

Code: Select all

$ mediainfo /video/WOBI.ts General ID : 1 (0x1) Complete name : /video/WOBI.ts Format : MPEG-TS File size : 100 MiB Duration : 3mn 0s Start time : UTC 2016-04-28 21:15:00 End time : UTC 2016-04-28 21:17:50 Overall bit rate mode : Constant Overall bit rate : 4 654 Kbps Network name : SERSAT Original network name : Astra Satellite Network 19,2'E Frequency : 3682000000 OrbitalPosition : 40.5W Video ID : 2160 (0x870) Menu ID : 200 (0xC8) Format : MPEG Video Format version : Version 2 Format profile : Main@Main Format settings, BVOP : Yes Format settings, Matrix : Custom Format settings, GOP : Variable Format settings, picture structure : Frame Codec ID : 2 Duration : 3mn 0s Bit rate : 4 101 Kbps Maximum bit rate : 3 500 Kbps Width : 704 pixels Height : 480 pixels Display aspect ratio : 4:3 Frame rate : 29.970 (30000/1001) fps Standard : NTSC Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Interlaced Scan order : Top Field First Compression mode : Lossy Bits/(Pixel*Frame) : 0.405 Time code of first frame : 13:18:20:15 Time code source : Group of pictures header GOP, Open/Closed : Open Stream size : 88.3 MiB (88%) Audio #1 ID : 2120 (0x848) Menu ID : 200 (0xC8) Format : MPEG Audio Format version : Version 1 Format profile : Layer 2 Mode : Joint stereo Codec ID : 4 Duration : 3mn 0s Bit rate mode : Constant Bit rate : 128 Kbps Channel(s) : 2 channels Sampling rate : 48.0 KHz Compression mode : Lossy Delay relative to video : -200ms Stream size : 2.76 MiB (3%) Audio #2 ID : 2122 (0x84A) Menu ID : 200 (0xC8) Format : MPEG Audio Format version : Version 1 Format profile : Layer 2 Mode : Dual mono Codec ID : 4 Duration : 3mn 0s Bit rate mode : Constant Bit rate : 64.0 Kbps Channel(s) : 2 channels Sampling rate : 48.0 KHz Compression mode : Lossy Delay relative to video : -8ms Stream size : 1.38 MiB (1%) Audio #3 ID : 2124 (0x84C) Menu ID : 200 (0xC8) Format : AC-3 Format/Info : Audio Coding 3 Mode extension : CM (complete main) Format settings, Endianness : Big Codec ID : 129 Duration : 3mn 0s Bit rate mode : Constant Bit rate : 128 Kbps Channel(s) : 2 channels Channel positions : Front: L R Sampling rate : 48.0 KHz Frame rate : 31.250 fps (1536 spf) Compression mode : Lossy Delay relative to video : -112ms Stream size : 2.76 MiB (3%) Menu ID : 5200 (0x1450) Menu ID : 200 (0xC8) Duration : 3mn 0s List : 2160 (0x870) (MPEG Video) / 2120 (0x848) (MPEG Audio) / 2122 (0x84A) (MPEG Audio) / 2124 (0x84C) (AC-3) Service name : WOBI 200 Service type : digital television

[tomamoto]

Found one more useful thing that might lead me somewhere, but I'm still working on understanding it. Found this line in the release notes for 2.1.6 to 2.2.0:

--ts-out option has been removed, it has been superceded by --demux demuxdump --demuxdump-access udp --demuxdump-file 127.0.0.1:1234

Instead of of --ts-out, I tried pasting this in, replacing the 127.0.0.1:1234 with my test multicast destination of 233.8.8.8:888 and it seems to work, in the sense that UDP data is streaming, but there's definitely something wrong.

cvlc /video/WOBI.ts --demux demuxdump --demuxdump-access udp --demuxdump-file 233.8.8.8:888

When I analyze this, not only am I not able to identify a useable TS or any PIDs, but the detected bitrate is a staggering 1.4Gbps! That's definitely not right...

[kmf31]

In the preferences: stream output => access output => udp, you find two parameters: caching value and group packets which may help but I am actually not sure if they also apply to udp in combination demuxdump but I believe they should. In principle they apply to udp output together with sout (your first variant of streaming).
The commandline versions of them are:

UDP stream output (udp)
--sout-udp-caching <integer [-2147483648 .. 2147483647]>
Caching value (ms)
Default caching value for outbound UDP streams. This value should be
set in milliseconds.
--sout-udp-group <integer [-2147483648 .. 2147483647]>
Group packets
Packets can be sent one by one at the right time or by groups. You
can choose the number of packets that will be sent at a time. It
helps reducing the scheduling load on heavily-loaded systems.

You simply try to increase the sout-udp-caching to 1000, 2000 or more ms and you may group much more than 1 udp-packet together. However, this will really help is another question.


Apart from it is my understanding that this kind of "raw" streaming is not reliable, less reliable than the method with sout but which indeed considerably modifies the stream by many details (vlc does these modifications).

In addition with sout output there is a problem for certain type of h264 video streams (see here: https://forum.videolan.org/viewtopic.php?f=4&t=132666 ).

However you seem to use only mpeg2-video and you should therefore not be concerned by this ?

In my personal experience the vlc-2.0.x branch (latest version here is vlc-2.0.9) is more reliable for udp streaming of DVB. I you can compile/install this version in another tree in your system (e.g. /usr/local/VLC-2.0.9/ as --prefix option in ./configure for vlc) I recommend that you try this. However, you will also need to install/use older versions of ffmpeg and libdvbpsi (best also in special trees in order to avoid conflicts with existing versions). Ideally you would need a Linux version of ~3 years ago with libraries of that time.

The ts-out option has also a file variant "tsdump-file" which is also useful in old vlc versions and there are some particular cases (if the source is rtsp and not dvb) where the new version with demuxdump etc. is not usuable while tsdump or ts-out are.


Apart from this if everything is done on the same machine (and not on different machines on the network) you can also apply the strategy to use a pipe (also called fifo) which is equivalent to a file instead of a stream (but without actually using the space of a big file). You simply create a pipe (which is like a file) with the command mkfifo at some place, use one vlc-instance to write to this pipe as it were a file (also with demuxdump but with file instead of udp option) and then use a second vlc instance to read simultaneously this pipe (again in the same way as it were a file). Of course this less pratical than streaming (no multicast) but it may help investigate/understand better your problem.
Of course you can also write a big file (with demuxmup with file option) and use another vlc instance to read this file in order to understand better what happens.

[tomamoto]

Thanks for the tips, kmf31! I'll look into those options and see if I can craft a more reasonable stream.

In the meantime, yes, the majority of video we test with is MPEG2, though we still need to run tests with MPEG4/h264 from time to time (even if many of the decades-old settop boxes some of our customers use don't even have mpeg4 decoders).

I'll also look into the vlc 2.0.x branch as a possible option. I understand there isn't a lot of a demand for what I'm trying to do, but I really hope I can come up with a solution that uses open-source software to fill in the gaps of my own knowledge. I've already tried doing this directly from ffmpeg and run into the same problems. Even using -c copy, it still seems to drop all the streams that aren't recognized as audio/video and loses all the PIDs and and program numbers in the process.

[Rémi Denis-Courmont]

As far as I am aware, the old ts-out hack would not address your use case any better than the demuxdump does. Neither of those will do pacing; they only work if the input itself is paced (e.g. DVB tuner), or if the output does not need pacing (e.g. recording to file).

To stream all programs, you need --sout-all. Passing raw MPEG-TS around without demuxing it is largelly out of scope of a media player though. Dvblast may be better suited.

[tomamoto]

As far as I am aware, the old ts-out hack would not address your use case any better than the demuxdump does. Neither of those will do pacing; they only work if the input itself is paced (e.g. DVB tuner), or if the output does not need pacing (e.g. recording to file).

To stream all programs, you need --sout-all. Passing raw MPEG-TS around without demuxing it is largelly out of scope of a media player though. Dvblast may be better suited.

I feared this might be the case. DVBlast would be ideal if it could handle plain files as input, but I imagine it doesn't because that would require a lot more moving parts to figure out pacing and all that. I might hit up the DVBlast mailing list to see if anyone can advise me on this, but it looks like it's back to the drawing board for me. I'm not particularly surprised there is little open source support for this. It's technology that'd mainyl be useful for cable headends, and that isn't really an industry that's well-represented in the Open Source world cause of, you know, absolute greed.

Anyway, thanks much for your help! I'll keep poking around the internet and see what I can find. Worst case scenario, I could try to build what I need from (mostly) scratch. I'm not really an experienced C developer, but it's never too late to learn...

[tomamoto]

Compiled the somewhat obscure open source project OpenCaster and I'm already able to achieve what I'm looking for from the command line, so at least I'm one step of the way there. Just leaving this in case anyone ever digs up this old thread trying to do the same thing as I. Cheers, and thanks again guys! I'm a big VLC fan and have been using it on a daily basis for near a decade now, but this particular situation just happened to call for a different set of tools. It was my own ignorance that brought me here in the first place, but I'm glad to have learned something.