GPS data in Text tracks

[sherington]

Hello,

I want to extract GPS data that is encoded inside a Text track inside an MP4. I would like to be able to get at that data in real-time as the video is played, so I can display it, chart it or whatever.

As an aside, I can view the encoded data using the vlc application by enabling the subtitle track. I can also use "avconv" to extract the subtitle track to a text file and I have successfully decoded all the GPS data contained within that track, so I know in principle this should be possible.

My problem now is to synchronise that with the video so the current position and heading can be plotted on an interactive map. My ideal solution would be to be able to register a callback through libvlc to notify my application each time new subtitle data is available. What I would want from libvlc would be the "raw" subtitle text - I expect my own application to do the decoding of the proprietry GPS format given the raw "text".

So what I am hoping for are some pointers as to where I should start with vlc to implement such a feature - do I need to write a new vlc plugin of a particular type, should I be looking at adding a callback to an existing piece of vlc, or something else? Is there a good example to follow from any existing plugin?

[Jean-Baptiste Kempf]

Yes, you need to write a new plugin to extract text data, like vmem, but for subtitles.

[sherington]

I'm not sure if I have done this the right way, but this is what I did...

- Create a new "text_renderer" module to send the SPU text via a callback, the callback accepts the subtitle text and an opaque pointer (in keeping with how vmem and amem are implemented).
- Add new API to libvlc to set this callback.

This actually works - the subtitle text containing the encoded GPS data is sent via the callback to my application at the correct time and I can decode it successfully and display the GPS data side-by-side with the video.

I have two outstanding issues with it:

First is that I don't know about is how to programatically select/enable my new text renderer module - I can select it with the "--text-renderer <module-name>" switch when I create the libvlc instance, but is there a way to do this in code inside my set_callback function?

Second is that ideally I would want to use this for subtitles only, but since the new module is a general text renderer it's also used for things like displaying the video title. Can I do anything about that?

[Rémi Denis-Courmont]

I think it is somewhat obvious, but if the GPS data is marked as plain text, there is no way for the player to discriminate it from normal subtitles.

If you want to deal with GPS data cleanly, it must be coded as a separate/new data/codec type in the MP4 file.

[uglide]

I'm not sure if I have done this the right way, but this is what I did...

- Create a new "text_renderer" module to send the SPU text via a callback, the callback accepts the subtitle text and an opaque pointer (in keeping with how vmem and amem are implemented).
- Add new API to libvlc to set this callback.

This actually works - the subtitle text containing the encoded GPS data is sent via the callback to my application at the correct time and I can decode it successfully and display the GPS data side-by-side with the video.

Hi, I faced with same problem. Can you provide your solution?

[ryantheseer]

Hi sherington or uglide: have either of you succeeded in making a plugin for GPS data in text files? I'm looking to do something similar, except that instead of GPS data, I would like to make use of custom auxiliary data that needs to be synchronized with the video image frames.

[Jean-Baptiste Kempf]

Hi sherington or uglide: have either of you succeeded in making a plugin for GPS data in text files? I'm looking to do something similar, except that instead of GPS data, I would like to make use of custom auxiliary data that needs to be synchronized with the video image frames.

So far, I've not seen any plugin actually working