audio seek sample wise and video seek frame wise

[slajar]

Hey,


I am new to libvlc. The API looks awesome and I really like to use it in future developments.

Well, I am working on a audio mixing tool that is also playing songs backwards and scrubbing it all around the block That's why I am having my own audio prebuffering concept. That means I would need a possibibility to seek in the audio part sample by sample to ensure my Prebuffer gets the right data. What I have seen is a float precision. This is not really deterministic. Is there another way?

The same problem do I have for the video images. I would like to retrieve them frame by frame and I am having the same prebuffer (liek the audio one) there as well to ensure resond times are pretty fast. I would need some kind of a frame stepper. Okay with ms precision and the defined fps it is possible to do so, but it is not really a good software architecture.

Any answer is highly appreciated.

kind regards
Matthias

[Rémi Denis-Courmont]

Time seeking is in microseconds; we only use floating point for ratio seeking (and even then, you'll need a rather long file to exceed the 48-bits precision).

That said, VLC is not really frame or sample-accurate regardless.

[slajar]

this is sad to hear

It is not possible to use a floating point value in this case. Rounding will still cut in some cases 1 single sample and you will hear crackles. I'll have to go underneath.
Maybe you can give me some directions and pin point to integrate a feature like that?

regards
Matthias

[Rémi Denis-Courmont]

I don't really see how you would loose a sample with microseconds integer precision. I have never seen audio above 192kHz, except for DSD, and obviously not video.

[slajar]

Okay I see, you are right I haven't calulated it before:

44100Hz ^= 41100 samples per second ^= 44,1 samples per ms ^= 0,0441 samples per micro second

--> this should be sufficient until 500.000Hz sample rate. Above that we have an oversampling problem (shannon).

From you perspective it is save to recaluclate sample positions form those micro second positions? There is no rounding etc.?

[slajar]

It's been a while since I opened this thread but now I have started to try this in code

I am using libvlc_audio_set_callbacks to get the raw data from file and libvlc_media_player_set_position to specify the particular position.

Well, it seems that the media must be at playing state to get the raw data through the callbacks. I tried to pause and unpause the playing state to get just 4096 byte blocks (blockwise) from libvlc.
Nevertheless this seems not to work correctly. I there another way to retrieve exact raw buffers from a file?

TIA
Matthias

Btw, I am still not really convinced that libvlc_media_player_set_position has enough precision to specify exact sample positions

[Rémi Denis-Courmont]

So what format needs more than microseconds precision?

[slajar]

I actually don't see microsecond precision in the API and I just see milli second precision. You proposed to use the float (percentage) handling with libvlc_media_player_set_position and just to believe it will work in micro second precision Okay, I will believe it for now.

My question is now, how can I hop over to the next buffer. Since the mediaPlayer is playing free do I will need to seek back? To prevent vlc from decoding unnessecary data I tried the procedure above pausing, playing waiting for new data and pausing. This feels pretty nasty and the performance seems also not pretty good and there is probably a bug in my implementation since I cannot get a seemless stream.

The API seems pretty awkward at this point. Maybe I can use vlc input plugins directly somehow?

[slajar]

Okay, let me describe my question a bit more different.

Is there a sample code where I can see how to step through an audio file buffer-wise? Best would be backwards

I know this example: https://gist.github.com/TimSC/4121862

There is just one thing missing and that is the time setter

[slajar]

It seems, I found some sample code in KDE:
http://code.metager.de/source/xref/kde/ ... reader.cpp

Well, This code seems pretty complicated to work around missing APIs in vlc. I can understand that vlc's API is stream based and that is why direct hopping to a particular position is pretty complicated but as far as I know every audio API has two ways to first file based and second stream based.

Is there no other way to do this than to use a complicated workaround like this? I am very sure that all file based codecs internally use block wise decoding and sample precision seek. This API has just not evolved to the higher level libvlc-API. Am I right?

....
10 Minutes later
....
Damn... Hhm, I just found that the code from KDE is NOT a workaround for my situation They are doing the oppisite way. They use VLC to play from phonon. Actually, there you can see VLC expects exactly the API I expect from VLC a read-Callback with a buffer and a sample-correct seek-Callback

VLC has all these fantastic plugins wrapped into one API, why can't I just use it?

[slajar]

no idea?

[slajar]

Since there was no answer here, I started to read trough the vlc sources. There are tons of source code files for different layers and event passing

First I started on top from the libvlc_ functions and I found that they are pretty far away from the mixer code while passing these agrv from command line to the particular threads. Well, this procedure is pretty complicated to understand.That's why I did it the other way around. Starting from an audio codec in my case a52.c I searched for usages of the DecodeBlock function and finally I found stream.c in src\input\stream.c and there I found the magic

The point is that vlc seems to use only stream_Block( ) for each file buffer that's read from the actual file. The file pointer is beeing automatically incremented. For play back and streaming this is good on the other hand I want to get particular raw audio on a random position, so a position woudl be needed. Actually, there is a function called AStreamSeekStream() in stream.c and it seems to do exactly what I need. Unfortunately, I didn't find a way where this function is called from the "outside".

After understanding this input-thread concept a bit more I am very sure that I don't want to use the input-thread and the mixer-thread in my application at all.

So my new questions are:
1. Can I disable mixer and input threads?
2. Can I access stream.c functions through an API call?

[slajar]

Not one Idea or a direction? I am willing to contribute my changes but I need some pin Points. Just a little help / overview on the basic structure of Libvlc and libvlccore would be helpful.