Is there an event to catch lyrics cues?

[Mikehell]

I'm trying to write an application that can sync things to music. I was thinking that I could just put the cues into a lyrics file and then catch them as they pass through. However. I can't seem to find and events to trap them. Is this possible? Or perhaps a better way to do what I need?

[Jean-Baptiste Kempf]

This is not implemented, sorry.

[Mikehell]

Man. That sucks. Can read different lyrics formats, but you can't do anything with 'em but display 'em.

Will any other .net library do this? I know WMP can have script commands (exactly what I need) but the music would need to be re-encoded to wma, a dying (or already dead?) format.

[unidan]

Hi, nevetheless event wouldn't work for sync as it is asynchronous, you would need callbacks.
But you can open a ticket for feature request on https://trac.videolan.org/vlc

[Mikehell]

Hi, nevetheless event wouldn't work for sync as it is asynchronous, you would need callbacks.
But you can open a ticket for feature request on https://trac.videolan.org/vlc

Why wouldn't it work? I use event listeners all the time. An event gets raised, gets caught, and you perform some task. Yes, there may be somewhat of a delay, but nothing that can't be overcome. There's got to be a listener already catching the subtitles anyway, no?

[trueboss926]

+1 for this feature request!

[unidan]

Why wouldn't it work? I use event listeners all the time. An event gets raised, gets caught, and you perform some task. Yes, there may be somewhat of a delay, but nothing that can't be overcome. There's got to be a listener already catching the subtitles anyway, no?

Event are asynchronous, you don't have any guarantee of when it would be processed. There could be seconds before you catch the lyrics event. Events are there to describe state change. For synchronization purpose, you want synchronized callbacks instead.

If you created a ticket, don't hesitate to link it there so that people can easily find it

[Mikehell]

Why wouldn't it work? I use event listeners all the time. An event gets raised, gets caught, and you perform some task. Yes, there may be somewhat of a delay, but nothing that can't be overcome. There's got to be a listener already catching the subtitles anyway, no?

Event are asynchronous, you don't have any guarantee of when it would be processed. There could be seconds before you catch the lyrics event. Events are there to describe state change. For synchronization purpose, you want synchronized callbacks instead.

If you created a ticket, don't hesitate to link it there so that people can easily find it

Sounds like you don't understand the issue. I'd assume that currently the software is listening for an event for lyrics. When one comes along, it throws an event and passes the lyric along. Then the function catches it, and runs code to display it. In what world would this take seconds? This can be done in javascript with no noticable delay.

synchronous will stop the music and wait till the function has completed.

Think Christmas lights synced to music. The music plays, it catches cues, it turns on lights. I'd assume DMX works along this same type of thing. What happens if, somehow, by some freak occurrence, a cue gets there a few seconds late? A string of lights goes off at the wrong time. Nobody (except the guy that wrote it) will probably even notice. We're not talking about controlling industrial equipment and building cars. We're not doing brain surgery with it. We want it to make silly lights go on and off. In my case, water to shoot out of nozzles like a cheap crappy version of Bellagio.

Ticket is: https://trac.videolan.org/vlc/ticket/22088

[sherington]

I don't know if this is splitting hairs or not, but I think it's more like the audio and video callbacks rather than being event-based. Maybe that's what unidan is getting at.

For example the video callback says, essentially, "render this frame now", so similarly you'd want an SPU callback that says "render this lyric/subtitle now".

The net effect may be the same, but it's not event-based.

Anyway, I'd like to see such a feature too.

[unidan]

Sounds like you don't understand the issue. I'd assume that currently the software is listening for an event for lyrics. When one comes along, it throws an event and passes the lyric along. Then the function catches it, and runs code to display it. In what world would this take seconds? This can be done in javascript with no noticable delay.

No, I understand, but events are not immediately executed, they are stored in the input thread, and processed afterwards. But the input thread is a big synchronous part of VLC and is also responsible for executing the access and demux functions, which can take time and thus delaying the sending of the event to your application AFTER it has been emitted by the output module.

The point is that it might not be your need to have something "perfectly" synchronous, but it won't fit the general use case for synchronization, for which you need accurate time cues so that you don't derive. Callbacks semantic is "I'm going to play this", event semantic is "Some time ago, we played this". Implementing two different method including one which is basically broken for synchronization doesn't look like a good design. Event-based one would work if you include a "Some time ago, at this timestamp, we played this" and if you're only syncing with a clock, not the stream itself, and even then it might be quite sensitive to jitter.

synchronous will stop the music and wait till the function has completed.

No, subtitles/lyrics are handled by the video output which is in a different thread than audio output. Even then, the audio output is filling buffers for the audio server, not directly modulating the audio material. Also they are a lot of function called in the audio output and it doesn't stop the music.

In any case, you would have your application and VLC in different threads, so the callback is just meant to notify your application that you have reached a synchronization point. It provides you a time event that you can sync on. If you don't do a lot of work and especially no blocking IO, you could even do what you need in the synchronization point.

I don't know if this is splitting hairs or not, but I think it's more like the audio and video callbacks rather than being event-based. Maybe that's what unidan is getting at.

that's exactly my point

For example the video callback says, essentially, "render this frame now", so similarly you'd want an SPU callback that says "render this lyric/subtitle now".

The net effect may be the same, but it's not event-based.

Anyway, I'd like to see such a feature too.

It would be great but I'm not sure how this should be handled. Maybe for this use case, a tagged timecode track would be a better fit, which is basically MIDI.

[sherington]

It would be great but I'm not sure how this should be handled. Maybe for this use case, a tagged timecode track would be a better fit, which is basically MIDI.

My own use-case for this is to extract GPS data which is encoded as a subtitle track, so I can plot a location on a map while the video plays.

It would be good enough for me if I could get a callback with the text each time an SPU were rendered. But then I'd like to get that text callback even if the subtitles are not displayed because I wouldn't want the raw GPS data string to be visible.

These things are never as simple as they seem.

[Mikehell]

Sounds like you don't understand the issue. I'd assume that currently the software is listening for an event for lyrics. When one comes along, it throws an event and passes the lyric along. Then the function catches it, and runs code to display it. In what world would this take seconds? This can be done in javascript with no noticable delay.

No, I understand, but events are not immediately executed, they are stored in the input thread, and processed afterwards. But the input thread is a big synchronous part of VLC and is also responsible for executing the access and demux functions, which can take time and thus delaying the sending of the event to your application AFTER it has been emitted by the output module.

The point is that it might not be your need to have something "perfectly" synchronous, but it won't fit the general use case for synchronization, for which you need accurate time cues so that you don't derive. Callbacks semantic is "I'm going to play this", event semantic is "Some time ago, we played this". Implementing two different method including one which is basically broken for synchronization doesn't look like a good design. Event-based one would work if you include a "Some time ago, at this timestamp, we played this" and if you're only syncing with a clock, not the stream itself, and even then it might be quite sensitive to jitter.

synchronous will stop the music and wait till the function has completed.

No, subtitles/lyrics are handled by the video output which is in a different thread than audio output. Even then, the audio output is filling buffers for the audio server, not directly modulating the audio material. Also they are a lot of function called in the audio output and it doesn't stop the music.

In any case, you would have your application and VLC in different threads, so the callback is just meant to notify your application that you have reached a synchronization point. It provides you a time event that you can sync on. If you don't do a lot of work and especially no blocking IO, you could even do what you need in the synchronization point.

I don't know if this is splitting hairs or not, but I think it's more like the audio and video callbacks rather than being event-based. Maybe that's what unidan is getting at.

that's exactly my point

For example the video callback says, essentially, "render this frame now", so similarly you'd want an SPU callback that says "render this lyric/subtitle now".

The net effect may be the same, but it's not event-based.

Anyway, I'd like to see such a feature too.

It would be great but I'm not sure how this should be handled. Maybe for this use case, a tagged timecode track would be a better fit, which is basically MIDI.

I think I understand now. It's the buffer that's the problem. Weird, if Microsoft can figure this --please stay polite-- out with WMP I'd think anybody could.