Frame-stepping forwards and backwards

[Rik1975]

Being able to step through a movie frame by frame, forward and backward, would be extremely useful for me. I am into video analysis, and I use VirtualDub a lot for frame-by-frame analysis, but that software has the disadvantage that it does not play nearly as many formats as VLC media player. If VLC would support frame-stepping, me and my colleagues would likely start using VLC a LOT more.

- Shortcut keys (preferably left and right cursor keys) for frame stepping would be super.
- Frame number display would go great with that.
- A frame-accurate slider would really top it off.


Anyone who puts this in is my hero!

Good luck on this fantastic project.

Rikkert

PS:
I understand frame-stepping or sliding to a (previous) non-key frame fast could be a bit of a hassle, but isn't that a nice challenge? If decoding from closest previous key frame is too slow, because a movie has little key frames, perhaps auto-buffering of reference frames periodically could be a fast solution? Just my 2 cents as a non-programmer.

[VLC_help]

Forward frame step is in VLC. Backwards one isn't and most likely there won't be one.

[Jean-Baptiste Kempf]

and shortcut keys are there for frame-stepping

[thomasjohnsen]

and shortcut keys are there for frame-stepping

Correction: A shortcut key is there for frame stepping.
Only one shortcut key, e for next frame. Too bad when I need it both ways...

[borisentrup]

Hello all, I've the same request.
I'm a beginner and I'm looking for a program taking snapshots and it would be also helpful for me to step in previous frame. Any help will be useful, or maybe anyone can recommend me other progams.

You're making a good job. Keep going!!!

[borisentrup]

@Rikkert: maybe it's worth a look to virtualdub.

[callstel]

I think a lot of people would like this "back-frame" function, including myself. It's almost impossible to capture the correct frame without this function. Surely it can't be THAT difficult to implement?

[Jean-Baptiste Kempf]

I think a lot of people would like this "back-frame" function, including myself. It's almost impossible to capture the correct frame without this function. Surely it can't be THAT difficult to implement?

It is THAT hard to implement.

[softskiller]

Could this be realised with a plugin?
Now I always have to use an external viewer like virtualdub, when I want to analyse single frames.

[Jean-Baptiste Kempf]

I believe this needs to be done in the core of VLC.

[orazio]

I need this essential function too. It has been asked for years, let's hope a good programmer will do the job!!!

[klawdek]

I put my vote in for the frame backwards feature.

I have a heck of a time trying to get to the right point when i try to back up to see something.

I would also love to have a finer control when moving forward or backward. I remember years ago having a multimedia player with a control that looked and worked like a jog and shuttle control. I would love to see something like that in VLC.

[rylleman]

Yes please!

It's essential in production enviroments.

I've asked for this before with only reply "can't be done"... - Are you sure? have anyone tried or are you just repeating same dogmatic truth?

[VLC_help]

For Nth+1 time,
It can be done with local files quite easily (just seek frame that is N-1 where N is current frame), but it won't work in many streaming scenarios (unless you buffer frames, but that won't be memory efficient with HD files). Also it will be SUPER slow with local playback if e.g. H.264 files have one I frame and 250 P frames after that. Also proper seeking is broken/unimplemented with certain demuxers, so outcome is that it would work for some formats OK, but there would be so many cases it wouldn't work at all or would just cause confusion with users.

If you need frame accurate moving, use video editors and I frame only video coding.

[Rik1975]

Well, in the end we decided to have our hired programmer try and build us a basic player, based on ffmpeg, which could framestep forward and backward. But indeed, like mentioned in the previous post, it poses many problems! Nevertheless, we do now have a player with, albeit limited, possibilities for backward frame stepping. But I can imagine it would confuse users when it works for some files/formats but not for others.. Maybe make it an 'experimental / expert' function?

[rylleman]

I went over to using Snappy video player for reviewing my clips which already in it's 0.2 version has backward frame stepping. Not for all formats and sometimes a little shaky. But it is working. As a professional user this is a major requirement for my work tool.
I would really like to see this implemented (for the nth+1 time) in VLC. It doesn't have to be perfect and work for all formats. But having it will rise the player miles up for professional usage.
I also don't think it will confuse the average user. Have it obscured in standard settings, not having any shortcut etc. But have it as an option for those who desire it.
Professionals who will use this feature are apt to use formats that are better suited for it so for them I don't think it will be a huge issue either.

No, I do not know how to code myself. That's why I'm nagging here on this forum. I could however set up a bounty for this somehow if desired by developers.

[Jean-Baptiste Kempf]

I have a quick patch for backward stepping. It is very very very very ugly and will not be merged. But if needed contact me (if you are a programmer, of course).

[rylleman]

I have a quick patch for backward stepping. It is very very very very ugly and will not be merged. But if needed contact me (if you are a programmer, of course).

Wow!
Although I'm no developer I would very much like to test this.
I can probably compile with the patch.

[Rik1975]

Ah, that's great!

I would love to see the code. I'm no advanced programmer, but I do have some programmers down the hall who could have a look at it, and see if they can compile it.

I'll also try and contact the programmer we hired some time ago to make our frame-stepping player and ask him how he solved some of the problems... Perhaps there could be some cross pollination.

[Khain]

Hello,
I am very interested in this piece of code to set up the frame-steping backward.

If you can send it to me

Best regards.

[JSLover]

It is THAT hard to implement.

...I see it like this...have a pref: Number of frames to keep in memory for frame skipping, default to say 10 or 20. As VLC plays the file forward it would keep each frame (in memory or in a cache on disk), it wouldn't matter what format the source video was in (if VLC can play/decode the video, it can remember that frame), take one frame at a time (as you are playing the file normally) & "remember" them (basically do the same as the "snapshot" function, just to memory or a specific disk location {a cache, separate from user-initiated snapshots}). Regardless of muxer/codec/whatever...or being a local file or not. I understand "seeking" backward is "THAT hard" (even tho the seek bar does go backward?)...but this would not be seeking, it's just "remembering (caching) what you already decoded".

I don't care if storing 20 frames of HD is bad on the memory/disk...it would be worth it for this feature. You don't know how many times I've tried to stop on a specific frame & my only choice is to have faster reflexes (trying to stop right before the frame) or stop early (way before the frame) & press E 9000 times to get to the frame I want. I would love to just Pause as soon as I see the frame I want (after it just happened) & "back up" a little. I would start with my prefs set to caching 10 frames back & 10 frames forward. If I get within 5 frames from the end of my back buffer (depending on my pref, above), VLC should anticipate that I might need more back frames: while I'm still looking at whatever frame I'm on, it should (in the background) seek back (if it can -- I understand "seeking" is not always possible) 5 or 10 secs & generate more "back frames"...it should not try to seek exactly "5 frames back", just seek back however far (5-10 secs), step/play the video forward (still in the background/in memory, not on screen) & see how many frames you got (comparing to the "oldest" one already in the back buffer). Add any frames found to the "back buffer". If you didn't get enough frames (according to the pref), seek farther & remember (only temporarily) that the default seek wasn't enough for this stream. If you can't seek, that's fine: the user can simply increase their buffer pref & restart the stream.

[Jean-Baptiste Kempf]

Sorry, but you have a very naive idea on how frames are rendered in VLC.
Patches are welcome though...

[JSLover]

Sorry, but you have a very naive idea on how frames are rendered in VLC.

...yes, I'm not a VLC dev, so I have no idea how frames are rendered. BUT I can imagine: you "somehow" decode the "frame" from all the file formats VLC supports, this part I don't care about...then you "draw" that frame to the screen, this part I do care about. As soon as you "draw" it to the screen, it might as well be a "screenshot"/"bitmap" (or other more compressed format, like png). That is what I'm saying: AFTER VLC's already decoded the frame, then save a copy to a buffer. Yes, I do know about "key frames", I do know that "not all frames are complete". But that's still simple: every keyframe, you start a new reference buffer frame: draw to screen, draw to a buffer (or draw only to the buffer & "copy" that to the screen {if copying is faster than "drawing" twice}). On the next frame, a non-keyframe, draw to screen (which properly covers the image already on screen), copy the last buffered frame to a new buffer & draw the partial frame on the copy of the buffer, so then the buffer would have a full "keyframe", for each frame. You can run the buffer (copy it to the screen) forwards or backwards (without seeking the original source), since they are essentially just images/screenshots now.

Patches are welcome though...

...I have nowhere to start. & I can't compile VLC. You said you had a patch that was "very very very very ugly", if you posted that online somewhere, people would have a "place to start". I'd like to see the method you chose that, even you, thought was very ugly.

Instead of telling me "you have a very naive idea on how frames are rendered in VLC"...I'd like you to tell me exactly why taking a "snapshot" of each rendered frame would NOT work. I think my idea is fine. Prove me wrong. No, I have no idea how VLC renders frames, but I'm talking about after it's on screen...or at least, after it's decoded & ready to be drawn. Once it's decoded & ready to be "drawn"...simply draw to the screen & also to a "frame buffer".

Other than not knowing how I should write it: I think my overall idea is sound. Think of it this way: if I played a video in VLC & stepped thru each frame, manually taking a snapshot of each frame...I'd have a directory full of full-frame snapshots (they would all be keyframes at this point). Then if I tell VLC to play that directory, it would appear to be a copy of the video (without sound, of course), in this case VLC could "frame backward", since each frame is a separate playlist item. So, in my "naive" way, that's all I'm suggesting: make VLC "do that" faster than a person could do manually. Make VLC either buffer each frame, as it's about to be drawn -- or simply draw to the screen & run the "snapshot" function, after each frame. Drawing the frames to the screen is already in VLC (duh). Taking a snapshot of the current frame is already in VLC. So, marry those 2.

Just to be clear, I am suggesting 2 different ways to solve this problem...

• BEFORE drawing to the screen, save a copy of what's about to be drawn to a buffer. Making sure each buffer is a full keyframe.
• AFTER drawing to the screen, run the built-in "snapshot" function (this option might be slower, but cannot not work)

[Jean-Baptiste Kempf]

I'd like you to tell me exactly why taking a "snapshot" of each rendered frame would NOT work. I think my idea is fine. Prove me wrong. No, I have no idea how VLC renders frames, but I'm talking about after it's on screen...or at least, after it's decoded & ready to be drawn. Once it's decoded & ready to be "drawn"...simply draw to the screen & also to a "frame buffer".

The frames are inside the GPU. So you would need to keep 20 frames (btw, why 20 and not 40 or 60) of 6MB each in an extra memory pool on the CPU side and do one or two extra copying of ALL frames in VLC? You realize the impact in performance, no?

So, the only way is to seek back quickly. And that's hard.

[madsjuul]

To be able to frame step Backwards would be very usefull for me.
I hope to see this feature implemented in VLC at some Point
Kind Regards
Mads Juul