Auto-sync subtitles

Feature requests for VLC.
rocky
Blank Cone
Blank Cone
Posts: 61
Joined: 13 Jan 2007 12:21
Contact:

Auto-sync subtitles

Postby rocky » 16 Oct 2012 05:20

We all know that sometimes, the subtitles we download for a movie are out of sync with the audio. Like other media players, VLC offers the possibility to "move" the subtitles to sync them with the audio, but doing so is tricky.
My suggestion is to develop some technology that auto-detect the voices, and to add a button that tries to sync the subtitles with them. It sounds difficult, but a lot of work already exists in the field of speech recognition, which could be helpful. The code could sweep through the whole audio until it finds some words that are clearly recognized (not necessarily the first words in the audio), then look for those words in the subtitles file, and sync them.
If effective, the feature would once again distinguish VLC from the competition. Also, it would solve the annoying problem of finding the right subtitiles / syncing them with the audio, and eventually, it could help to provide the option of auto-downloading the right subtitles for any movie.

DGMurdockIII
Big Cone-huna
Big Cone-huna
Posts: 534
Joined: 14 Sep 2006 16:46
VLC version: y
Operating System: windows 10 64bit Pro
Contact:

Re: Auto-sync subtitles

Postby DGMurdockIII » 21 Oct 2012 13:49

i agree the subtitle syncing feature kinda still need to be improved alot but i do agree that you have improved the subtitle support alot over the past 2 years but subtitle syncing still need more polish

sccy
New Cone
New Cone
Posts: 6
Joined: 26 Oct 2012 17:18

Re: Auto-sync subtitles

Postby sccy » 26 Oct 2012 18:19

Auto-sync is not a mature technology. Use hotkeys G and H for now.

rocky
Blank Cone
Blank Cone
Posts: 61
Joined: 13 Jan 2007 12:21
Contact:

Re: Auto-sync subtitles

Postby rocky » 05 Nov 2012 18:24

Thanks for the answers.
I post this reply just to clarify something. The speech recognition software doesn't need to be perfect. All it has to do is recognize with some confidence three or four words in the entire audio. By syncing those few words, the rest of the subtitles would be synced too. And if the software misses every now and then, it wouldn't really matter (we would just have to download another subtitle, or sync it with G and H, as usual).

asteriskk
New Cone
New Cone
Posts: 1
Joined: 11 Feb 2013 22:51

Re: Auto-sync subtitles

Postby asteriskk » 11 Feb 2013 22:53

You can also use an online tool such as http://www.sync-subtitles.com to sync subtitles.

XweAponX
Blank Cone
Blank Cone
Posts: 19
Joined: 15 Sep 2009 00:13
VLC version: 2.0.5 and 1.1.11
Operating System: Windows 7 and 8

Re: Auto-sync subtitles

Postby XweAponX » 12 Mar 2013 21:10

I suppose with all of the different types of Subs, this would be difficult to implement. There is a script addon that allows you to search for the exact version of the release you are playing - It downloads Subs "on the fly" right into the folder where you have the content stored. I found a windows installer for it, but it is downloadable as a script from some links here:

viewtopic.php?f=29&t=71535

Because with all of the content I have saved, I've found out - I MUST have the exact release for the subs I've downloaded, if you try to use any other subs, they will never sync up even using the hotkey G-H solution. I've also made my own subs for certain media I have, you can do it "by hand" or you can use NCH VideoPad Video Editor to add them in.

naderghanbari
New Cone
New Cone
Posts: 1
Joined: 20 Jul 2015 19:08

Re: Auto-sync subtitles

Postby naderghanbari » 20 Jul 2015 19:11

Is there any activity on this? I agree that using manual sync feature is very useful but in most cases auto syncing (at least just readjusting the delay or offset for english, french, and german) is very easy. I tried to do it (on a separate project) with some success and I feel confident enough to work on it but I'm not sure whether people actually need it or not.

Overdrivr
New Cone
New Cone
Posts: 2
Joined: 04 Apr 2019 14:39

Re: Auto-sync subtitles

Postby Overdrivr » 04 Apr 2019 14:48

Hi folks,

I know this is an old post but I wrote a tool that uses machine learning to do this. It finds the optimal subtitle's delay and framerate using a Machine Learning speech detection model. It even downloads the subtitle automatically for tv series (movies soon as well), by using the filename to guess the subtitle to fetch. It works well, at least on many tv series I've tried synchronization is very good. It can fix delays of up to a few hundred seconds (for instance when the tv series has an introduction summary but the sub has not), with an accuracy of ~ 80ms. It's not synchronizing each sentence individually thus far, but that's something that I'll might implement in the future.

It's still recent, so there might be some gotchas to figure out. Give it a try and let me know if something's not working, or help fixing it - the tool is free and open-source :)

Available here: https://captionpal.org

Cheers,

unidan
Developer
Developer
Posts: 1493
Joined: 25 Mar 2018 01:00

Re: Auto-sync subtitles

Postby unidan » 04 Apr 2019 15:20

Hi Overdrivr, it's been a while, very nice project!

There also have been other people doing this, like https://github.com/smacke/subsync :)
Do you plan to integrate this as a module inside VLC? It would be really nice to have but keep in mind that VLC is a (L)GPLv2 project. It would be ok if compiled as an external DLL but would remove every use case needing (L)GPLv2 with a static build, like VLC for android. It would be awesome if you could provide it under at least GPLv2 too.

I can try to provide some help on the integration if you would like it.

Overdrivr
New Cone
New Cone
Posts: 2
Joined: 04 Apr 2019 14:39

Re: Auto-sync subtitles

Postby Overdrivr » 16 Apr 2019 18:03

Hey, thanks for the feedback ! Appreciate it :)

Yes there's plenty of folks who've done similar stuff ! This technique works very well. But as far as I know those apps are mostly proof of concept command line applications that require python and pip install, not actual application you can install and that has buttons for non-technical folks. And I don't think any of them downloads the sub directly from the filename (well, I guess there must be some that does but don't know about it).

I'm not sure about the VLC integration yet. It would require some extensive effort and maintenance (builds for every VLC-supported platform, CI pipelines, conversion to C code, etc) that I'm not sure I can commit to. As it is now, maintenance is pretty simple and everything's already pretty much automated, so maintenance is a breeze. If I can ensure a reliable synchronization on a larger scale (when enough people will be using this regularly), then maybe I'll reconsider.

One thing to consider is that VLC is also used to display video streams (with PopcornTime for instance), but synchronization requires a good chunk of the audio track to work so I'm not 100% certain it would cover this important case.

Anyway, any feedback or contribution on the app is very welcomed :)

cbeltrangomez
New Cone
New Cone
Posts: 1
Joined: 17 Apr 2019 04:08

Re: Auto-sync subtitles

Postby cbeltrangomez » 17 Apr 2019 04:11

When I try to download the windows version of https://captionpal.org/

I got this:

This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Details>
No such object: captionpal-releases/0.12.0/captionpal-Windows-10-10.0.14393-SP010.exe
</Details>
</Error>
( function(){ window.SIG_EXT = {}; } )()


Return to “VLC media player Feature Requests”

Who is online

Users browsing this forum: No registered users and 14 guests