How about using VLC as a powerful language learning tool

[gordyliang]

Many people watches movies/series to learn different languages, and there are tons of sources of this kind of material. Most of the time the learners depend on the caption as a help of learning. The mode of watching the video for language learning should be quite different than that of just entertaining. I think the VLC player can be enhanced to support the language learning mode.

The features may include:
- Replay the last sentence.
- Forward/Reverse by sentence.
- Navigate and browse of the captions.
- Shortcut to link to dictionary
- Save favorate sentences, and review
- Save the new words that you add. Link back to the movie and the sentences by words.
... ...

Actually there are some existing software with the above function, but it is not free, and it is not open, and not able to utilize all possible materials from the internet for learning. By enhancing VLC with this, all language learners will be benfited.

I am a programmer and I can contribute to this. Want to know the feedback of all you guys.

[gordyliang]

Just curious isn't this interesting to anybody?

[whitedeer3.14]

sounds like a great idea to me... especially because i have kids who are learning to read

[sls]

I would LOVE this!

[Jean-Baptiste Kempf]

This is far from being trivial, since the subtitles are not exported.

[gordyliang]

I know it is not a trivial, but I really want to try.

What did you mean "the subtitles not exported". I mean for the videos with independant subtitle file, we can use it and build the above navigation features I metioned. If the video is with subtitles in its own, for sure it can not be utilized as an navigation.

[Jean-Baptiste Kempf]

the subtitle decoder does not pass the raw data.

[gordyliang]

the subtitle decoder does not pass the raw data.

I don't quite understand. Sorry. You mean the decoder does not pass the raw data of what?

[Jean-Baptiste Kempf]

of subtitles.

[gordyliang]

Let's make it more simple now to just the two basic feature I mentioned.

Say, if we have a text based subtitle file with the video file there. If we add two buttons in the interface, one is to skip to next sentence, and the other to reverse to the last sentence. Isn't possible that when the button is clicked:
- the interface module knows what is the current timestamp of playing.
- it queries from the subtitle module, which by looking into the subtitle data, gets the timestamp of the previous or next sentence.
- The interface module gets to know the new timestamp now, so it gets to that point by controling the decoder.

I just think about it with my instint. I did not read the code so just guest it should work.

[gordyliang]

I am trying to understand the related code and figure out the way to achieve the features I mentioned.

My rough idea is to enhance the QT interface module (or in the future, clone and develop an indepedent one).
* Add a text box showing all the subtitles, and highlight the current sentence.
* The interface module should have a way to access the parsed subtitle data.
A workaround solution may be just to create a global subtitle data structure, and when the subtitle is opened and loaded in the subtitle module, make a copy into this global data. So that the interface module can access it easily. This should work but may have broken down the current VLC framework.
Another approach could be to enhance the libvlc library so that it can expose the subtitle data.
* The interface should have a way to know the current timestamp of the media. This should be already there although I did not find out the code.
With the above essential things, all need to do is to implement the subtitle related features in the enhanced interface.

So could you advice how the following key points can be implemented?
- How to enhance the libvlc library to expose the subtitle data to the interface modules.
- How the interface modules get the current timestamp of the playback.

[gordyliang]

* The interface module should have a way to access the parsed subtitle data.

After a digging the code, I found out the following piece of code shall be able to access the subtitle data from the interface module (like qt4):

THEMIM->getInput() -> p->slave[0]->p_demux->p_sys

Please comment if this is a correct approach. Thx

[Jean-Baptiste Kempf]

* The interface module should have a way to access the parsed subtitle data.

After a digging the code, I found out the following piece of code shall be able to access the subtitle data from the interface module (like qt4):

THEMIM->getInput() -> p->slave[0]->p_demux->p_sys

Please comment if this is a correct approach. Thx

This is incorrect and will fail.

[gordyliang]

* The interface module should have a way to access the parsed subtitle data.

After a digging the code, I found out the following piece of code shall be able to access the subtitle data from the interface module (like qt4):

THEMIM->getInput() -> p->slave[0]->p_demux->p_sys

Please comment if this is a correct approach. Thx

This is incorrect and will fail.

I understand the restriction now. Looks like the data loaded by the modules can not be easily shared with other modules.

I tried several approach to escalate the subtitle data, and finally find the follow approach which can work. However it is a "dirty" solution though.

What I did:
- add fields i_subtitles and subtitle to the structure type input_thread_t.
- when calling "open" in subtitle.c, assign the values back to the fields of p_input (input_tread_t) that I created.
- in gui module files like input_manager.c, access the subtitle data through p_input->subtitle.

How could I send a code delte to you guys to have a quick look whether this is a good enough approach?

Thx.

[Jean-Baptiste Kempf]

What you should do is probably a new text_renderer module that could output the raw data.

[gordyliang]

What you should do is probably a new text_renderer module that could output the raw data.

Do you mean that when rendering the subtitle, output the raw data of that subtitle at the same time?

By doing this, can the interface module access all subtitle data randomly? What I want to do is that the interface module can access the subtitle data randomly no matter what is the current position. By doing this, many flexible features can be achieved.

[gordyliang]

I adjusted the way to share the subtitle data and it makes more sense now. And I added the function of prev/next sentence, repeat current sentence, based on the subtitle data.

The brief idea is: define the data structure of "main subtitle" in the input thread internal structure. Add APIs to set and get the main subtitle data. In subtitle module, set the main subtitle, in the interface module, get the main subtitle. Added several events for it as well.

Will send you a patch for review soon.

[gordyliang]

Patch sent.

[Hermosa.tabby]

I could see a reason to have more than one subtitle active. Sometimes users who speak multiple languages watch movies together. Some users understand about 50 percent of one and just need to fill in the words. That seems easy enough. It would not be all the features you seek, but it would be a benefit to anyone who learns a language with movies. I'm fluent in spanish thanks to immersion and movies

[saiprasad]

I would like to contribute to VLC by adding language learning tool. I see some of the features already added by gordyliang. I want to know how I can manipulate subtitle text that is displayed( removing/ adding some text to it). I am able to compile the VLC in ubuntu as of now.

[staratel20]

Think about this features for long time. +1