Open Source Live Video Mixer/Internet TV Production Software

[ngd9]

***Edit 28/6/10 - This has now been founded as an open source project. Please visit www.dustv.org and forum.dustv.org for more details on the project.***

Hi,

Ok massive post - sorry! I'm new to the board and was hoping I could gauge interest and also get some help and advice on a project I'm starting that will most likely be making heavy use of libVLC.

I'm a final year undergraduate Communications Engineer and I have decided to use my final year project to begin development on an open source software suite that’ll hopefully, once finished, allow the creation of professional style live Internet TV broadcasts by anyone with access to a video camera and a PC. I’ve hunted around but can’t find anything open source similar to this that currently exists, although if there is please let me know as I may have to rethink my project! I’m also fairly new to VLC and have no experience with libVLC or any of the technicalities of video codecs, streaming protocols, setting up open source projects etc - I have a lot to learn!! I may be talking rubbish in parts so please let me know if I am...please be kind !

I’m still in the early planning stages but at the moment the idea is to develop several pieces of software to perform the main functions of a TV production studio using consumer laptops, PCs and video cameras as the hardware and LANs and the Internet as the communications systems. The software suite would require the following:

OB (Outside Broadcast) Streaming Node - SN
Used to take audio and video from a video camera (probably via firewire), transcode and stream it via LAN/Internet to the “TV Production Server”. I will probably initially just use VLC on a laptop to do this but plan to later develop a customised program using libVLC.

TV Production Server - PS
This would be the main development of the project and will hopefully include the following functions:
1. Audio/Video Mixer - mix multiple live audio/video streams coming from the SNs into a single output stream for broadcast.
2. Graphics - animated graphics able to be overlaid onto the output stream (e.g. Logos, ticker bars, name titles etc.) for professional looking output.
3. VT (Video Tape Deck) – mixing of locally stored videos into the output stream. This could also be used to insert commercial breaks etc.
4. Recording – Allow recording of the output stream for archive/On Demand services
5. Broadcast Server – Stream the output to the world...!

These would all be installed on a main server PC in the studio with the idea that individual computers on the studio network could be used to control the different functions so that one laptop would control mixing, another graphics, another VT etc. so it can be laid out like a professional studio.

Here’s a diagram of a possible setup for a student TV channel or something:


I would assume most video streaming/encoding etc. would be done using libVLC however the main problem will be how to mix the different video streams as I doubt this would be possible at codec level? One idea I had would be to mix the video at the Operating System’s graphics level using OpenGL or similar and then capture this using a similar technique to screencasting to create the output stream. This would then allow many different codecs, resolutions etc. to be in the input streams without having to worry about compatibilities. I guess this would also make overlay of graphics a lot easier? Unfortunately, again, I have never programmed OpenGL/libVLC so don’t know how feasible this is? Any help/advice/comments would be great!!

Any ideas on what you think conceptually and technically about the project and also advice on if it’s worth setting up as an open source project yet etc. etc. would be a massive help!

Thanks for reading if you got this far!
Nick

[ngd9]

Hi,

For anyone who comes across this post and is interested in the project, I have decided to use the GStreamer media framework as it is much more suited to this type of project (low latency, more flexible architecture etc.). I will hopefully set up a website when more work has been done and post it here.

Thanks,
Nick

[lionsharemedia]

Great idea,
I stumbled on this post and am very interested, would like to assist with testing. I think the direction you take with gstreamer is the right way as there is already a lot of work done.
Good luck
Keep me posted

[ngd9]

Thanks! I may be looking for a small number of testers early next year so I'll let you know nearer the time. I'd be particularly interested in long distance testing over the internet, making a mock TV news program or something using live video feeds from testers around the UK and beyond.
Nick

*Apologies to admins as I guess this thread is now in the wrong forum!

[neckro]

next year is here can i be a alpha too ?

[ngd9]

Hi neckro, thanks for the offer!

Just to update on my progress...I'm coming to the end of my University research project and am currently writing the final report...fun fun! I've developed some prototype software for testing purposes, however it is not really of any great use for anything other than my research at the moment as it was very rushed and will have a fair few bugs. However it shouldn't take too much of an effort to get it to alpha, especially if others will be interested in contributing

Anyway here are some screenshots to show where it's at:

The Mixer...aka "Dust Maker"


The Streamer...aka "Dust Sender"

• - The first shows the mixer software with 9 MPEG4 (Xvid) SD video channels (720x576) being decoded from RTP payloads streamed via LAN. This test was run on an Intel i7 machine with Ubuntu 10.04 Alpha 1 and if "top" is to be believed only used around 12% of the CPU total...so plenty of extra juice spare for implementing effects, audio etc and should work on slower PCs. Bottom sliders etc. currently not implemented. Video playback is generally pretty smooth, however some work will have to be done to ensure good frame synchronisation.

• - The second shows the streaming software running on a laptop. In this test it was just streaming the MPEG4 "Big Buck Bunny" video via RTP to the mixer software.

Anyway unfortunately I don't have time to get the project moving at the moment as I'm currently writing the report and have finals in May. I will try and get the ball rolling in June/July to setup a website, Git etc. and maybe post my Uni project report which will give a lot more detail into the specifics. Hopefully then if people are interested and willing to contribute we can turn this into something really useful! Until then...thanks for the patience!

Cheers,
Nick

*Edit 8/4/10 - Updated thread title to better reflect topic

[stu843]

Hi I would be very interested in seeing the completed program and code and perhaps trying to develop this project further. I run Reading University Student Television (www.ruon.tv) and I think that this piece of software may allow us to do some interesting projects. I am correct in thinking that potentially this software could run on PC and MAC? (gstreamer appears to compile on both)

I am also currently writing my third year project report for my degree course so I understand how busy you are, good luck!

Stuart

[neckro]

/me is in standby mode

[ngd9]

Hi Stuart,

Sorry for the delayed response...project deadlines

Thanks for the interest...yours sounds like exactly the type of use I was aiming it for. Yeh I've written it using GStreamer, Glib and GTK+ all of which compile for Linux, Windows & Mac. It should theoretically be fairly simple to compile on these with little or no code modification. The only library which isn't cross platform is the BSD sockets library which I have used for network communications. This should work on Mac, but Windows uses Winsockets instead so this module would have to be re-written to compile on Windows.

I must stress though that the program will need a fair bit of work before it is production ready. It is currently written for quite specific prototyping uses, and some of the code is messy and uncommented due to the massive time pressure I had to get it usable! That said, with a couple of contributors (especially any GStreamer gurus) it could only be a few months before we have something basic that could be used to make simple productions. Beyond that, I have already done some simple tests with transitions, alpha mattes and graphic overlays using the GStreamer command line "gst-launch" which proved very promising and shouldn't be too difficult to implement in the program in the long run.

I finish exams early June so will set up a website and source repository then and start promoting it (maybe with you, nekro and others' help?) and try and get it moving as a project. I'm sure with a good team of contributors it could be developed into a very powerful piece of software!

Cheers,
Nick

[stu843]

That's fine, I am now in the middle of my exams and trying to avoid revising, hence the quick reply!

Anyway a website/forum/launchpad would be all very useful if you were to make this an OS project. Then we could get off the VLC forums as it isn't really to do with VLC!

I have to admit I have never used Gstreamer before, but I have done a bit of programming in C before on my course, I have also done a module this year on video engineering which may be of use. I finish my exams on the 13th of May and so will be able to help any time after that. I have had quite a bit of experience at web development, having helped develop the RUON website amongst others so if you need any help there then give me a shout. It will be interesting to see where this project goes, my only concern is the synchronising of the video sources as across the internet there could be some large delays. Programs such as procaster and make.tv are all web based and can do a similar thing, but they are not open source/adaptable and don't provide an option to download the stream to your PC.

Let me know when you have something up and running!

Stuart

[ParadoxG]

Hi Guys,

I am interested in this project in two aspects: I would be an ideal client of this project, and I would enjoy contributing to this project.

At my church we do live video streaming and presentations. Sometimes we would like to have both a powerpoint presentation and a live feed from the camera in the same video stream. We don't have a video mixer yet and have been looking for video mixing software.

I just finished my exams for my last year of computing science. I have no video experience, but some software development experience. I also have a web server that could provide web space, forums, and a repository.

~ParadoxG

[ngd9]

Sorry for the lack of response...I've spent too much time on this project this year and not enough on lectures so am in a bit of a poor state with revision!! Thanks for all the offers of help, I'll get back in contact after exams are over in early June and we'll get this ball rolling...I'll get the source up and hopefully we can migrate over to our own forum and continue discussions there!
N

[neckro]

any news ?

[tzuktz]

Hi,

I am very interested in such a project.
I am looking to build one these days so a collaboration could be very much useful for all of us

Any news?

Cheers

[ParadoxG]

I'm still here watching this thread.

~ParadoxG

[ngd9]

Hey, apologies again for the lack of response. exams are over and finally I have a little time to get this moving and hopefully be more responsive...I've bought the domain dustv.org so that's where the project will be living. I'm gunna set up a website and forum over the weekend so hopefully we'll be able to move over there next week and start properly discussing where to take this. I'll also get the source code online for the prototype

[neckro]

that will be great.. cheers

[ngd9]

Ok, it's been a fair bit of work and taken a bit longer than expected, but a basic website and forum are now up at http://www.dustv.org/ so if people can head on over and sign up to that forum we can continue discussion over there .

I would also appreciate feedback on the website and the content. Any suggestions on improvements for the design or text would be greatly appreciated.

I have also put the prototype source up at http://www.dustv.org/prototype.html for anyone interested. Do remember it was for research and therefore is extremely buggy and will crash a lot if not used in the specific way designed for my research I have written some instructions for the adventurous who want to give compiling it a go. I will post instructions for its use on the Dust forum.

See you all over there!
Nick

[Vincent Vega]

Hi, I don't for sure if i'm in the right place for this question. If i get digital television i would be able to watch shows on demand right? I don't mind paying extra for it. If it saves me the trouble of downloading it. On this site they tell me it's possible to watch movies on demand but is it possible to watch the big series on demand as wel? (site is in dutch)
http://www.internetentv.nl