Call for a Better Solution

[jlundy82]

The short story: I have been trying for several months to build a DIY "Slingbox" that would transmit live TV over the internet. Some parts of this have been difficult, but ultimately it has been relatively easy to assemble the necessary components: old stripped-down computer, cheap capture card/hardware encoder, universal remote control accepting web commands, scheduling scripts to print out nice clickable TV schedules, etc. HOWEVER, the almost impossible task has been to actually STREAM the data over the internet.

For those who like it short and sweet, here's what I'm driving at: we absolutely need to come up with a simple guide for home users to reliably stream video, using VLC, to other computers. From what I've seen, this forum is littered with requests for how to do this, and frankly, I imagine the moderators are getting tired of answering the same questions about why RTSP won't work behind a NAT router, or how HTTP progressive download is different from real streaming, or how one sets the buffer to time x for y transport protocol, or simply how one goes about streaming in the first place.

THERE HAS GOT TO BE A WAY TO DO THIS! In principle, it's a little tricky, but it's not impossible. The software on official Slingboxes cannot be THAT complicated, and VLC is already frustratingly close to having a good solution. There are a whole host of websites offering custom made programs to reliably transmit video. If we work together, we must be able to find some kind of generalizable solution for home users wanting to stream.

The content of this solution is going to have to work under the following conditions: the server will be behind a NAT router, the computer will be cheap and only moderately powered, the encoding will be done by a cheap hardware encoder, the stream will be decent enough to read the current score in a football game, and the stream will reliably traverse firewalls.




The long story: Here's all that I've tried (or considered), and all that has failed.

- RTSP behind a NAT router. I'll admit I don't totally understand the full complexities of TCP, UDP, etc. However, it seems evident that your router needs to have a certain level of understanding about what's being transmitted, in order to send RTSP. NO REASONABLY-PRICED ROUTER seems capable of doing this, and YOU CANNOT AVOID THIS PROBLEM USING DMZ. If you try to do this, the best you are going to receive is a few RTSP headers. I understand that if the RTSP could be "interleaved" it would get around this problem. From what I gather, interleaving means that you put all the components of the RTSP stream into a single path, with their various bits of information interleaved so that they can get through the router's narrow restrictions. However, the development team is busy, and I don't gather that they will be introducing interleaved RTSP-over-HTTP any time soon.

- MSSH. This is the only "streaming" method that actually will get out from behind my home router, and into my computer through other routers. However, I use the scare quotes around the word "streaming" intentionally. This method is CRAPPY (it's from Microsoft -- shocker). Unless the video is completely unrecognizably degraded, it won't hold onto a connection. PLUS, even on my LAN, it starts to build latency into the stream, which basically makes it no better than simple HTTP progressive downloading.

- RTMP. There's no way to do this with VLC. Plus, there is no inexpensive FLV hardware encoder. That means that the cheap computer you want to run your media server is going to have to grab video from your capture card, software encode the video, and then act as the server for transmitting the video. THIS WILL EASILY OVERWHELM MOST CHEAP COMPUTERS. Furthermore, even if your computer is up to it, it's not a straightforward task to set this up. The only open source program to serve RTMP is Red5 and it's not user friendly in its documentation. I don't think Red5 will grab video from the capture card either, which means you need some botched processing chain that looks like: Hardware Encoder -> VLC Software Encoder -> Red5 -> Internet.

- Apple's HTTP Streaming. This almost looks enticing, especially since, theoretically, it means getting your stream onto an iPhone or iPad. However, this method requires a hardware encoder that will encode into H.264 (doable). Plus, from what I see, you need a mac to get the media segmenter. PLUS, this is going to introduce LATENCY. Upwards of 30s after a short while (from what I've heard on the web). That means using this method to stream TV is just not feasible. If you have a DVR, imagine the frustration of virtually "clicking" your remote control, and then waiting 10s for the resulting action to show up on your computer screen. Forget fast-forwarding through commercials.

- HTTP Progressive Download. The old standard works, kind of. I've adapted the example code offered by VLC for their web interface. I updated flowplayer to the latest version, and added the aforementioned clickable schedules. This method will, on a decent connection, get video from one computer to another. However, its obvious downside is the inevitable latency it builds every time there's a hiccup in the connection. Within 7 minutes, the lag between reality and what's displayed might be 10s. This is perfectly OK for VOD; but for live streaming this is just not feasible.

- Miscellaneous. I've also looked into Unreal Server, VX30, the Java IBM MP4 Toolkit, Darwin SS, and probably many more I can't even remember at this point in time.

If we work together, there must be a solution that will not only work for me, but which will serve as a guide for the many people hoping to do something similar.

[Rémi Denis-Courmont]

You CANNOT stream from behind a NAT. There is by definition no way to do it, unless you forward a port from the NAT to the server (which is not always possible, and often times requires manually intervention), or there is a relay on the open Internet (which costs a lot to maintain).

[jlundy82]

Hi Rémi,

Thanks so much for the quick reply. I'm glad we have some progress already!

Let me ask for a point of clarification on your response. In the past I've forwarded ports to my home server. I do this the standard way; i.e. going into the router's HTML interface, and selecting certain ports to forward to the internal IP address of my home server. This is how I was able to access an MMSH stream from the WAN (i.e. internet). This is also how I was able to receive a few RTSP headers from the WAN. However, when my VLC client sent a DESCRIBE request, I got an error message saying there was an invalid response from my home server.

From what you've said in past messages, I was under the impression that this simple form of port forwarding wasn't good enough for RTSP/RTP. Rather, what you need is a router that supports an Application Layer Gateway (or ALG) for RTSP/RTP. From what I can gather, this is a more advanced form of port forwarding. A quick check on the internet indicated to me that there are no home routers with support for an RTSP ALG (e.g. you're not going to find a router at Best Buy that supports RTSP/RTP ALG).

So, given that there aren't any home routers with support for the proper ALG, here's what questions remain:

1) Is there a form of streaming (besides MMSH and interleaved RTSP-over-HTTP) that can work behind a NAT router with simple port forwarding?

2) Is there a combination of open source (or cheap) programs available that can get an RTSP stream to pass through a NAT router?

3) Does anyone know if there is a way to configure one's home environment to allow the home server to be "in front of" the router? I.e. Cloud (Internet) -> Home server -> Router ?

I'm very appreciative of any responses about how to make this work. I'll be continuing to work through it and I'll post any information I find (including attempts that work, and also attempts that don't work).

[Rémi Denis-Courmont]

1) Is there a form of streaming (besides MMSH and interleaved RTSP-over-HTTP) that can work behind a NAT router with simple port forwarding?

HTTP.

In theory RTSP/2.0 with RTSP-ICE could work, but that's a really huge beast to implement on both clients and servers, and not even standardized yet.

2) Is there a combination of open source (or cheap) programs available that can get an RTSP stream to pass through a NAT router?

No.

3) Does anyone know if there is a way to configure one's home environment to allow the home server to be "in front of" the router? I.e. Cloud (Internet) -> Home server -> Router ?

Many routers have a "DMZ" configuration for this, but the details depends on the model.

[jlundy82]

Rémi,

I really do appreciate your responses. From your comments on this forum, and also your website (http://www.remlab.net/op/vod.shtml), you seem to be one of the few people capable of answering this thorny question.

Also, as promised, I wanted to keep everyone updated on my progress, so that this might (one day) be a helpful guide to other amateur home streamers.

Here's what I've now found out:

1) I basically forwarded all the ports on my router, and this seems to have gotten me one step closer to getting RTSP to work from behind my router. Specifically, I've been able to move through all the RTSP headers (e.g. DESCRIBE, SETUP, PLAY, etc.) up to the GET_PARAMETER header. At that point though, two things happen. First, the log indicates that my VLC client is trying to get through to my internal IP address, where before it was trying to access my external address. Second, my internet at home ceases to work unless I reboot the server (I suspect some kind of loop problem?). I'll try setting my server to be a DMZ, and see if this makes a difference.

2) I'm intrigued by the RTSP-ICE comment. I recognize it may be terribly time consuming to execute, BUT, if I can figure it out, maybe I can pass on information that would make it easier for others. In my case, I'm looking to stream TV to myself, so even if the setup is a bit involved at least I may only need to do it for one server and one client (my laptop). The only code I could find for ICE is PJNATH. Unfortunately, despite comparatively good documentation, I haven't been able to build the windows binary.

3) It appears that there are some people who believe it's possible to get a stream through the NAT (see the links below). I know its probably frustrating, but is it possible to explain, briefly, why it's so hard to get a client to access RTSP streaming through a NAT router? Essentially, I've now opened all ports on my router, but that doesn't seem to be good enough. If every port (both TCP and UDP) are being forwarded to the router, why is it that RTSP/RTP cannot make it to a client on the web?

http://support.microsoft.com/kb/827562
http://homepage.ntlworld.com/robin.d.h. ... m.html#nat
http://hirntier.blogspot.com/2008/10/st ... h-nat.html

Thanks to anyone (esp. Rémi) who can answer these questions.

[jlundy82]

Ok, all, here's an update:

My latest attempt is to do NAT traversal using a VPN. If I can get OpenVPN up and running, then hopefully this will establish a virtual LAN, and by doing this it will make the problem of accessing the WAN go away. If I can get this work, I'll post the results here.



****UPDATE*****

10/09/10

****UPDATE*****


I've tried the VPN solution at a number of different public WIFI locations and the results are pretty bad. Only one of them (ATTwifi at a McDonald's) even allowed me to use UDP. Everywhere else basically limits your connection strictly to HTTP on port 80 (i.e. no other ports, no other protocols). I still have yet to try this out on a home network. In theory, I might get a better connection doing this because I can plug in to ethernet, I have a better chance of UDP being open, and I'll have better bandwidth than the meager connection offered by public wifi.

However, given my tests at the one location that did work, it doesn't look good. I was able to successfully create a VPN connection between the home server and my client laptop; however, the speed of the connection was so poor that my VLC client could not manage to get an RTP/RTSP stream up and running. I had hoped that the VPN would only minimally affect the connection speed (it is transferring over UDP after all); but it appears that it does significantly slow down the connection.

WHAT DOES THIS ALL ADD UP TO? The Internet is a f***'ed up thing.

On the surface of it, creating a DYI slingbox seems simple. The hardware in an official slingbox is simple to recreate, a home-server can serve up video just as well as the proprietary software of Sling media, and a client computer with VLC has all the hardware and software required to capture and play this stream.

Then come the routers. Basically, routers take the resplendent traffic of the internet and put a straitjacket on it, such that it is reduced to a banal means for carrying emails and HTML pages. To break through the restrictions of routers means one of two options: send everything through a tiny hole in their firewall (if you can find one), or transmit data over HTTP. Both solutions cripple streaming video to terrible conditions. Of course, corporations manage to break through routers, by employing a network of exceedingly complicated methods that bypass the router's limitations. These mostly work by employing "firewall punching" -- i.e. using a third party server to coordinate the streaming connection. These solutions are pretty well outside the reach of an amateur home streamer.

I'll try the VPN approach on a friend's router, and see if this works any better than public wifi. I doubt it's going to work, but if it does, at least it will be a small victory. WHICH I NEED, AFTER MONTHS OF WASTING TIME TRYING TO DEVISE A METHOD TO MAKE THIS WORK!

[tilleke]

Hi

I can understand your frustation. I am in the same mood.

About VPN, I found this software for Android, VLC Stream & Convert, and a support-page to overcome problems using VPN. Here is the link:

http://traveldevel.com/vlc-stream-conve ... g-over-vpn

I am not saying this is resolving problems but maybe the guide can get you moving in the right direction.

rgds,
tilleke

[jlundy82]

tilleke,

Thanks for the tip (and for the consolation). It's pretty crazy to see that this person could sustain an RTSP stream using VPN. Maybe there's hope yet. I've been busy over the last month, and had kind of given up on this; but now I see someone with success I'll have to give it another try.

On a side note, the following post also seem really interesting: viewtopic.php?f=4&t=64827

This person seems to have coded a program that can intercept and alter the outgoing RTSP stream, so that the appropriate IP address is given to a client on the other side of the server's NAT. If this code were distributed, this would seem to be a much better solution than a VPN. Using this approach would be much more straightforward.