Frame-stepping forwards and backwards

[lobster]

The frames are inside the GPU. So you would need to keep 20 frames (btw, why 20 and not 40 or 60) of 6MB each in an extra memory pool on the CPU side and do one or two extra copying of ALL frames in VLC? You realize the impact in performance, no?

So, the only way is to seek back quickly. And that's hard.

What about building an index while playing the movie - similar to how ffms2 does it. That makes stepping easy and requires small amount of memory.

[Jean-Baptiste Kempf]

The frames are inside the GPU. So you would need to keep 20 frames (btw, why 20 and not 40 or 60) of 6MB each in an extra memory pool on the CPU side and do one or two extra copying of ALL frames in VLC? You realize the impact in performance, no?

So, the only way is to seek back quickly. And that's hard.

What about building an index while playing the movie - similar to how ffms2 does it. That makes stepping easy and requires small amount of memory.

Sure, but it's not trivial. Not to mention that it means all your seeks must be frame-accurate.

Anyway, VLC is open source, patches are welcome.

[whunter]

And that, my friends, is why open source projects stagnate. Those who want to implement new features don't know where to start, and those who know where to start don't like to help.

I'm having flashbacks to the bad old unix days...

[drati]

Hi

For Nth+1 time,
It can be done with local files quite easily (just seek frame that is N-1 where N is current frame)

I'm playing from local files and I can't find out how to do this. How do I find the current frame and set the current frame to a different one.

Thanks in advance,
Nuno Almeida

[complexzeros]

I have a quick patch for backward stepping. It is very very very very ugly and will not be merged. But if needed contact me (if you are a programmer, of course).

How can I get a copy of this patch?

I only need to be able to back-step for I-frame only MPEG-2 videos. I'm also willing to work on implementing this if someone could point me in the right direction as to where to start.

[Jean-Baptiste Kempf]

mail me

[complexzeros]

What if we don't always save a buffer of previous frames during normal play, but build it on demand? For example, assume the user's video is paused.
1. User asks VLC to step back one frame.
2. The position of the current frame is saved (using PTS or byte location?): call it frame x.
3. Then you seek backwards enough to find an I-frame,
4. decode forward until until you reach frame x (saving each frame as it is decoded),
5. display frame x-1.

Then you could step back several more frames before having to re-buffer. Are there any flaws with this high-level idea?

[complexzeros]

Well, in the end we decided to have our hired programmer try and build us a basic player, based on ffmpeg, which could framestep forward and backward. But indeed, like mentioned in the previous post, it poses many problems! Nevertheless, we do now have a player with, albeit limited, possibilities for backward frame stepping. But I can imagine it would confuse users when it works for some files/formats but not for others.. Maybe make it an 'experimental / expert' function?

Which formats does it work for?

[vlcforumusername]

I would love to have this feature as well. I came across this lua extension...

http://addons.videolan.org/content/show ... ent=156396

But the damn thing only works when the dialogue window is open (you cant use the keyboard shortcuts if you're in full screen). It does 'work' though.

Could we maybe just fix this extension so the hotkeys work?

[complexzeros]

That extension doesn't actually step backwards frame by frame. It just tries to seek back by a small increment which will only be the previous frame if you're lucky and have a cooperative format.

[vlcforumusername]

Wouldn't it be a matter of trivial math to seek back to the previous frame? I mean you know where you're at, and you know the time between frames soo...

In any case, just seeking back "by a small increment" would be enough for most people. Why can't this extension be fixed to work with hotkeys?

[Jean-Baptiste Kempf]

Wouldn't it be a matter of trivial math to seek back to the previous frame? I mean you know where you're at, and you know the time between frames soo...

How do you know the time between frames?

[TheBigTime001]

Indeed, calculations from keyframes would tend to be very inaccurate. Just examining a movie's keyframes in Avidemux, there is absolutely no consistency between different medias. A DVD could range between 1 to over 10 frames between keyframes, and from the Internet, there are so many poorly encoded videos that this method of calculation is doomed to fail. You could build a frame-by-frame cache, up to x frames ago, but as has been pointed out, this has the potential to be extremely RAM-intensive. He didn't even get in to UHD video's requirements, and for good reason. That could require over 1 GB of RAM for the caching. Avidemux uses a frame-by-frame index, AFAIK, but it takes up to 30 seconds to a minute or more to load the file, depending on the size and complexity. That kind of long load time is absolutely not necessary unless the user wishes it. Maybe pre-indexing of the file is potentially an option, but it would likely not be used, as I doubt that the vast majority of people would want to wait that long for files to load in VLC. Especially because settings that aren't supposed to be on by default have an odd tendency to wind up on when users play around in the advanced settings (In my first year of VLC I made many such mistakes). So it comes down to both long loading times being necessary for any practial accuracy and potential UI issues (many settings that are in the advanced settings aren't well documented and can confuse users). Sorry for the long wall of text, it's just my 5 years of VLC experience talking.

[TheBigTime001]

Correction: to index the file takes up to a minute, not loading it.

[vlcforumusername]

How do you know the time between frames?

In the video properties the fps is displayed. Divide by fps... right?

And as for the 'buffering is too expensive' argument - With modern hardware, seeking to any point in a video is nearly instant. So why buffer anything? When you want to step back one frame, just simulate a hop directly to that point, as if the user had manually moved the scrubber.

Basically, I think all people want is a option to set the 'Very short backwards jump' to 1/fps. Currently the lowest you can set it to is 1 full second.

[TheBigTime001]

I think there is a confusion here about how keyframing works. Normal (non-index-based) seeking is done on a keyframe-by-keyframe basis, which is why there is only one second of frame-by-frame backseeking at a time; VLC has to buffer the data (all 24 to 30 or 60 frames worth per second, potentially all the way up to 4096x2304 per frame) to overcome this. Index seeking would work but would require an initialisation period of up to a minute. I guess maybe an option to index a file would be OK, so long as it was on-demand. I personally would rather wait the 30 seconds or so inside VLC and then have full capabilities (even if only transiently) than have to load the file in Avidemux to do this. I just would rather it not potentially inconvenience the vast majority of users when offered.

[Jean-Baptiste Kempf]

How do you know the time between frames?

In the video properties the fps is displayed. Divide by fps... right?

Ahah, you are a bit naive. Most of the time, fps displayed is wrong

[vlcforumusername]

...but some of the time it's right.

It just seems like it would be simple to allow users to specify a fractional time step in the 'backwards jump' setting (currently locked to 1 full second). Even IF the video properties are wrong, there usually aren't more than a handful of formats (24fps, 30fps, etc...). You could likely get it right by taking a blind guess.

[vlcforumusername]

So is there any reason why we couldn't just use a fractional number in the advanced 'backwards jump' time? Or better yet, a 'estimated 1 frame' option, even if it isn't always accurate, it would be better than what we have now.

[TheBigTime001]

Fractional numbers aren't necessarily the best idea, because inevitable imprecisions allow for the possibilty of error. 1/30th of a second wouldn't be too hard, that's just 0.0333(repeating) seconds, so accuracy would be ok. So also with 60 frames a second (0.016667 seconds or so per frame). But with odd framerates, cumulative imprecisions could occur (because even float numbers are inevitably truncated at some point). So you'd have to have compensation routines built into such code. Estimated 1 frame wouldn't be all that practical either, because a video can use just about any keyframing pattern possible. With so many ways to keyframe, all of which would have to be taken into consideration and detected, this would be quite a substantial effort (and also constantly updated as someone might find a new keyframing pattern not covered). Perhaps on-demand buffering is possible, as follows: only store 1 second's worth of frames in the buffer at a time. If the user seeks a frame outside of the buffer, then discard the old buffered data and replace it with either the previous or next set of frames. So instead of accumulating massive RAM usage, you would purge the old set of frames from this buffer/cache when you were done with them. Just adding my two cents for now.

[vlcforumusername]

Yes exactly, there is never any reason to buffer more than a second.

So is anyone actually working on a feature like this?

[Jean-Baptiste Kempf]

No.

[fishstick]

: )

No is an answer.

But why? Jurisdictional?

VLC is already prereading videodata. Why not let users define a freeze and step back? Accordingly to free available memory in machine VLC takes back (prereading) a few seconds? That can't be that hard if you all are great at making VLC such a hit : )

And if all you excellent programmers still persist on the No, please tell us users why.

Thanks! (from a heavy user)

[Jean-Baptiste Kempf]

Usually, it's because noone seems to think it's important enough to work on it. As people are volunteers, noone does it.

The issue here, is that it's a quite hard feature, more difficult that most people think.

[pixtur]

ROFL. It's super tempting to write a rant about coder design and ignoring completely valid user requests. Sorry, but I have to get this out.

This missing feature is the primary reason why I'm using the QuickTime-Viewer most of the time. But sometimes (under Windows and for .AVIs) I'm stuck with VLC. I don't --please stay polite-- care how hard it is. You're having it backwards: If the user want's to see the previous frame, jump to the previous iFrame and decode forward. Of course it's not as efficient as possible, but nowadays computers are so fast, that it would get the job done. Sorry, but an answer like this drives me insane. You might be the smartest developer on the planet, but PLEASE I beg you, PLLEEEASE read "Don't make me think" but Steve Krug. You're wasting your time by writing code that misses its potential.

A mentality like this will drive VLC down the route to Gimp, and becoming an icon that normal people use to make fun of open source projects.