Frame-stepping forwards and backwards

Feature requests for VLC.
Rik1975
New Cone
New Cone
Posts: 5
Joined: 22 Jul 2010 17:01

Frame-stepping forwards and backwards

Postby Rik1975 » 22 Jul 2010 17:19

Being able to step through a movie frame by frame, forward and backward, would be extremely useful for me. I am into video analysis, and I use VirtualDub a lot for frame-by-frame analysis, but that software has the disadvantage that it does not play nearly as many formats as VLC media player. If VLC would support frame-stepping, me and my colleagues would likely start using VLC a LOT more.

- Shortcut keys (preferably left and right cursor keys) for frame stepping would be super.
- Frame number display would go great with that.
- A frame-accurate slider would really top it off.


Anyone who puts this in is my hero! :geek:

Good luck on this fantastic project.

Rikkert

PS:
I understand frame-stepping or sliding to a (previous) non-key frame fast could be a bit of a hassle, but isn't that a nice challenge? If decoding from closest previous key frame is too slow, because a movie has little key frames, perhaps auto-buffering of reference frames periodically could be a fast solution? Just my 2 cents as a non-programmer.

VLC_help
Mega Cone Master
Mega Cone Master
Posts: 25661
Joined: 13 Sep 2006 14:16

Re: Frame-stepping forwards and backwards

Postby VLC_help » 22 Jul 2010 17:43

Forward frame step is in VLC. Backwards one isn't and most likely there won't be one.

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Frame-stepping forwards and backwards

Postby Jean-Baptiste Kempf » 22 Jul 2010 19:31

and shortcut keys are there for frame-stepping
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

thomasjohnsen
New Cone
New Cone
Posts: 2
Joined: 29 Jan 2010 18:17

Re: Frame-stepping forwards and backwards

Postby thomasjohnsen » 14 Sep 2010 07:22

and shortcut keys are there for frame-stepping
Correction: A shortcut key is there for frame stepping.
Only one shortcut key, e for next frame. Too bad when I need it both ways...

borisentrup
New Cone
New Cone
Posts: 2
Joined: 15 Oct 2010 15:31

Re: Frame-stepping forwards and backwards

Postby borisentrup » 15 Oct 2010 15:52

Hello all, I've the same request.
I'm a beginner and I'm looking for a program taking snapshots and it would be also helpful for me to step in previous frame. Any help will be useful, or maybe anyone can recommend me other progams.

You're making a good job. Keep going!!!

borisentrup
New Cone
New Cone
Posts: 2
Joined: 15 Oct 2010 15:31

Re: Frame-stepping forwards and backwards

Postby borisentrup » 15 Oct 2010 17:39

@Rikkert: maybe it's worth a look to virtualdub.

callstel
New Cone
New Cone
Posts: 1
Joined: 17 Feb 2011 16:39

Re: Frame-stepping forwards and backwards

Postby callstel » 17 Feb 2011 16:46

I think a lot of people would like this "back-frame" function, including myself. It's almost impossible to capture the correct frame without this function. Surely it can't be THAT difficult to implement?

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Frame-stepping forwards and backwards

Postby Jean-Baptiste Kempf » 17 Feb 2011 18:03

I think a lot of people would like this "back-frame" function, including myself. It's almost impossible to capture the correct frame without this function. Surely it can't be THAT difficult to implement?
It is THAT hard to implement.
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

softskiller
New Cone
New Cone
Posts: 3
Joined: 15 Jul 2010 21:47

Re: Frame-stepping forwards and backwards

Postby softskiller » 08 Mar 2011 12:05

Could this be realised with a plugin?
Now I always have to use an external viewer like virtualdub, when I want to analyse single frames.

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Frame-stepping forwards and backwards

Postby Jean-Baptiste Kempf » 13 Mar 2011 11:44

I believe this needs to be done in the core of VLC.
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

orazio
New Cone
New Cone
Posts: 1
Joined: 14 Mar 2011 13:14

Re: Frame-stepping forwards and backwards

Postby orazio » 14 Mar 2011 13:17

I need this essential function too. It has been asked for years, let's hope a good programmer will do the job!!!
:) :) :)

klawdek
Blank Cone
Blank Cone
Posts: 17
Joined: 10 Nov 2010 11:26

Re: Frame-stepping forwards and backwards

Postby klawdek » 02 Apr 2011 05:55

I put my vote in for the frame backwards feature.

I have a heck of a time trying to get to the right point when i try to back up to see something.

I would also love to have a finer control when moving forward or backward. I remember years ago having a multimedia player with a control that looked and worked like a jog and shuttle control. I would love to see something like that in VLC.

rylleman
Blank Cone
Blank Cone
Posts: 11
Joined: 24 Feb 2009 09:07

Re: Frame-stepping forwards and backwards

Postby rylleman » 05 Apr 2011 10:05

Yes please!

It's essential in production enviroments.

I've asked for this before with only reply "can't be done"... - Are you sure? have anyone tried or are you just repeating same dogmatic truth?

VLC_help
Mega Cone Master
Mega Cone Master
Posts: 25661
Joined: 13 Sep 2006 14:16

Re: Frame-stepping forwards and backwards

Postby VLC_help » 05 Apr 2011 13:27

For Nth+1 time,
It can be done with local files quite easily (just seek frame that is N-1 where N is current frame), but it won't work in many streaming scenarios (unless you buffer frames, but that won't be memory efficient with HD files). Also it will be SUPER slow with local playback if e.g. H.264 files have one I frame and 250 P frames after that. Also proper seeking is broken/unimplemented with certain demuxers, so outcome is that it would work for some formats OK, but there would be so many cases it wouldn't work at all or would just cause confusion with users.

If you need frame accurate moving, use video editors and I frame only video coding.

Rik1975
New Cone
New Cone
Posts: 5
Joined: 22 Jul 2010 17:01

Re: Frame-stepping forwards and backwards

Postby Rik1975 » 02 Apr 2012 15:22

Well, in the end we decided to have our hired programmer try and build us a basic player, based on ffmpeg, which could framestep forward and backward. But indeed, like mentioned in the previous post, it poses many problems! Nevertheless, we do now have a player with, albeit limited, possibilities for backward frame stepping. But I can imagine it would confuse users when it works for some files/formats but not for others.. Maybe make it an 'experimental / expert' function? ;)

rylleman
Blank Cone
Blank Cone
Posts: 11
Joined: 24 Feb 2009 09:07

Re: Frame-stepping forwards and backwards

Postby rylleman » 02 Apr 2012 15:47

I went over to using Snappy video player for reviewing my clips which already in it's 0.2 version has backward frame stepping. Not for all formats and sometimes a little shaky. But it is working. As a professional user this is a major requirement for my work tool.
I would really like to see this implemented (for the nth+1 time) in VLC. It doesn't have to be perfect and work for all formats. But having it will rise the player miles up for professional usage.
I also don't think it will confuse the average user. Have it obscured in standard settings, not having any shortcut etc. But have it as an option for those who desire it.
Professionals who will use this feature are apt to use formats that are better suited for it so for them I don't think it will be a huge issue either.

No, I do not know how to code myself. That's why I'm nagging here on this forum. I could however set up a bounty for this somehow if desired by developers.

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Frame-stepping forwards and backwards

Postby Jean-Baptiste Kempf » 03 Apr 2012 00:52

I have a quick patch for backward stepping. It is very very very very ugly and will not be merged. But if needed contact me (if you are a programmer, of course).
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

rylleman
Blank Cone
Blank Cone
Posts: 11
Joined: 24 Feb 2009 09:07

Re: Frame-stepping forwards and backwards

Postby rylleman » 03 Apr 2012 09:00

I have a quick patch for backward stepping. It is very very very very ugly and will not be merged. But if needed contact me (if you are a programmer, of course).
Wow! :)
Although I'm no developer I would very much like to test this.
I can probably compile with the patch.

Rik1975
New Cone
New Cone
Posts: 5
Joined: 22 Jul 2010 17:01

Re: Frame-stepping forwards and backwards

Postby Rik1975 » 03 Apr 2012 09:47

Ah, that's great!

I would love to see the code. I'm no advanced programmer, but I do have some programmers down the hall who could have a look at it, and see if they can compile it.

I'll also try and contact the programmer we hired some time ago to make our frame-stepping player and ask him how he solved some of the problems... Perhaps there could be some cross pollination.

Khain
New Cone
New Cone
Posts: 3
Joined: 28 Oct 2013 11:26

Re: Frame-stepping forwards and backwards

Postby Khain » 12 Nov 2013 09:30

Hello,
I am very interested in this piece of code to set up the frame-steping backward.

If you can send it to me

Best regards.

JSLover
Cone that earned his stripes
Cone that earned his stripes
Posts: 116
Joined: 20 Mar 2012 00:08
VLC version: 2.1.5
Operating System: Win7
Location: r.secsrv.net/youtube.lua

Re: Frame-stepping forwards and backwards

Postby JSLover » 12 Nov 2013 22:19

It is THAT hard to implement.
...I see it like this...have a pref: Number of frames to keep in memory for frame skipping, default to say 10 or 20. As VLC plays the file forward it would keep each frame (in memory or in a cache on disk), it wouldn't matter what format the source video was in (if VLC can play/decode the video, it can remember that frame), take one frame at a time (as you are playing the file normally) & "remember" them (basically do the same as the "snapshot" function, just to memory or a specific disk location {a cache, separate from user-initiated snapshots}). Regardless of muxer/codec/whatever...or being a local file or not. I understand "seeking" backward is "THAT hard" (even tho the seek bar does go backward?)...but this would not be seeking, it's just "remembering (caching) what you already decoded".

I don't care if storing 20 frames of HD is bad on the memory/disk...it would be worth it for this feature. You don't know how many times I've tried to stop on a specific frame & my only choice is to have faster reflexes (trying to stop right before the frame) or stop early (way before the frame) & press E 9000 times to get to the frame I want. I would love to just Pause as soon as I see the frame I want (after it just happened) & "back up" a little. I would start with my prefs set to caching 10 frames back & 10 frames forward. If I get within 5 frames from the end of my back buffer (depending on my pref, above), VLC should anticipate that I might need more back frames: while I'm still looking at whatever frame I'm on, it should (in the background) seek back (if it can -- I understand "seeking" is not always possible) 5 or 10 secs & generate more "back frames"...it should not try to seek exactly "5 frames back", just seek back however far (5-10 secs), step/play the video forward (still in the background/in memory, not on screen) & see how many frames you got (comparing to the "oldest" one already in the back buffer). Add any frames found to the "back buffer". If you didn't get enough frames (according to the pref), seek farther & remember (only temporarily) that the default seek wasn't enough for this stream. If you can't seek, that's fine: the user can simply increase their buffer pref & restart the stream.
"why the f*** do i need a google+ account to comment on a video?" — jawed (2013) (about YouTube's new requirement of Google+)

youtube.lua — Play YouTube videos in VLC!
Updated: Thu, Jan 15, 2015 --- 1/15/15, 7:19:19pm EST

forum.videolan.org/viewtopic.php?f=29&t=111977&p=379147#p379147
Sigh, the above can't be a link: "You cannot use certain BBCodes: [url]."...so, I can't even link back to a post on this forum?
How about this: can long-term/trusted users be allowed links in sigs?

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Frame-stepping forwards and backwards

Postby Jean-Baptiste Kempf » 19 Nov 2013 00:11

Sorry, but you have a very naive idea on how frames are rendered in VLC. :)
Patches are welcome though...
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

JSLover
Cone that earned his stripes
Cone that earned his stripes
Posts: 116
Joined: 20 Mar 2012 00:08
VLC version: 2.1.5
Operating System: Win7
Location: r.secsrv.net/youtube.lua

Re: Frame-stepping forwards and backwards

Postby JSLover » 19 Nov 2013 09:19

Sorry, but you have a very naive idea on how frames are rendered in VLC. :)
...yes, I'm not a VLC dev, so I have no idea how frames are rendered. BUT I can imagine: you "somehow" decode the "frame" from all the file formats VLC supports, this part I don't care about...then you "draw" that frame to the screen, this part I do care about. As soon as you "draw" it to the screen, it might as well be a "screenshot"/"bitmap" (or other more compressed format, like png). That is what I'm saying: AFTER VLC's already decoded the frame, then save a copy to a buffer. Yes, I do know about "key frames", I do know that "not all frames are complete". But that's still simple: every keyframe, you start a new reference buffer frame: draw to screen, draw to a buffer (or draw only to the buffer & "copy" that to the screen {if copying is faster than "drawing" twice}). On the next frame, a non-keyframe, draw to screen (which properly covers the image already on screen), copy the last buffered frame to a new buffer & draw the partial frame on the copy of the buffer, so then the buffer would have a full "keyframe", for each frame. You can run the buffer (copy it to the screen) forwards or backwards (without seeking the original source), since they are essentially just images/screenshots now.
Patches are welcome though...
...I have nowhere to start. & I can't compile VLC. You said you had a patch that was "very very very very ugly", if you posted that online somewhere, people would have a "place to start". I'd like to see the method you chose that, even you, thought was very ugly.

Instead of telling me "you have a very naive idea on how frames are rendered in VLC"...I'd like you to tell me exactly why taking a "snapshot" of each rendered frame would NOT work. I think my idea is fine. Prove me wrong. No, I have no idea how VLC renders frames, but I'm talking about after it's on screen...or at least, after it's decoded & ready to be drawn. Once it's decoded & ready to be "drawn"...simply draw to the screen & also to a "frame buffer".

Other than not knowing how I should write it: I think my overall idea is sound. Think of it this way: if I played a video in VLC & stepped thru each frame, manually taking a snapshot of each frame...I'd have a directory full of full-frame snapshots (they would all be keyframes at this point). Then if I tell VLC to play that directory, it would appear to be a copy of the video (without sound, of course), in this case VLC could "frame backward", since each frame is a separate playlist item. So, in my "naive" way, that's all I'm suggesting: make VLC "do that" faster than a person could do manually. Make VLC either buffer each frame, as it's about to be drawn -- or simply draw to the screen & run the "snapshot" function, after each frame. Drawing the frames to the screen is already in VLC (duh). Taking a snapshot of the current frame is already in VLC. So, marry those 2.

Just to be clear, I am suggesting 2 different ways to solve this problem...
  • BEFORE drawing to the screen, save a copy of what's about to be drawn to a buffer. Making sure each buffer is a full keyframe.
  • AFTER drawing to the screen, run the built-in "snapshot" function (this option might be slower, but cannot not work)
"why the f*** do i need a google+ account to comment on a video?" — jawed (2013) (about YouTube's new requirement of Google+)

youtube.lua — Play YouTube videos in VLC!
Updated: Thu, Jan 15, 2015 --- 1/15/15, 7:19:19pm EST

forum.videolan.org/viewtopic.php?f=29&t=111977&p=379147#p379147
Sigh, the above can't be a link: "You cannot use certain BBCodes: [url]."...so, I can't even link back to a post on this forum?
How about this: can long-term/trusted users be allowed links in sigs?

Jean-Baptiste Kempf
Site Administrator
Site Administrator
Posts: 37523
Joined: 22 Jul 2005 15:29
VLC version: 4.0.0-git
Operating System: Linux, Windows, Mac
Location: Cone, France
Contact:

Re: Frame-stepping forwards and backwards

Postby Jean-Baptiste Kempf » 24 Nov 2013 02:35

I'd like you to tell me exactly why taking a "snapshot" of each rendered frame would NOT work. I think my idea is fine. Prove me wrong. No, I have no idea how VLC renders frames, but I'm talking about after it's on screen...or at least, after it's decoded & ready to be drawn. Once it's decoded & ready to be "drawn"...simply draw to the screen & also to a "frame buffer".
The frames are inside the GPU. So you would need to keep 20 frames (btw, why 20 and not 40 or 60) of 6MB each in an extra memory pool on the CPU side and do one or two extra copying of ALL frames in VLC? You realize the impact in performance, no?

So, the only way is to seek back quickly. And that's hard.
Jean-Baptiste Kempf
http://www.jbkempf.com/ - http://www.jbkempf.com/blog/category/Videolan
VLC media player developer, VideoLAN President and Sites administrator
If you want an answer to your question, just be specific and precise. Don't use Private Messages.

madsjuul
New Cone
New Cone
Posts: 1
Joined: 05 May 2014 12:45

Re: Frame-stepping forwards and backwards

Postby madsjuul » 05 May 2014 12:48

To be able to frame step Backwards would be very usefull for me.
I hope to see this feature implemented in VLC at some Point
Kind Regards
Mads Juul


Return to “VLC media player Feature Requests”

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 3 guests