Reverse frame by frame

[THX1138]

There is a shortcut key to advance one frame and the key is E. For some reason there is no reverse one frame function? Why?!

[EodiV]

Made a ticket 18699
Thanks for letting us know.

[Rémi Denis-Courmont]

The answer was just a few posts below... https://forum.videolan.org/viewtopic.php?f=7&t=79893

[Cantar4]

Jean-Baptiste Kempf » 07 Aug 2017 18:44 : It's harder in VLC than in QuickTime, because VLC supports way more formats than QT.

Knowing that h264 is by far the most used format for reviewing films, would it be possible
to offer this highly requested feature for analytic purposes (QuickT style) on mp4 files only ?
--
jpb

[Jean-Baptiste Kempf]

Jean-Baptiste Kempf » 07 Aug 2017 18:44 : It's harder in VLC than in QuickTime, because VLC supports way more formats than QT.

Knowing that h264 is by far the most used format for reviewing films, would it be possible
to offer this highly requested feature for analytic purposes (QuickT style) on mp4 files only ?
--
jpb

Issue is not h264 but mp4 vs mkv vs avi, etc...

And people will not understand that it does not work for everyone...

[Cantar4]

A short explanation from you will do !
If that is the format you select over mkv, avi, etc., students** (and profs) will understand
why they have to Handbrake convert their study film, whatever its original format, to mp4.
--jpb

** no cocktails for them, Soja milk only.
--

[Huntervlc]

Thanks for making this topic, I was about to make it.

You could just say "not allowed" if not possible with that format.

[Jean-Baptiste Kempf]

A short explanation from you will do !
If that is the format you select over mkv, avi, etc., students** (and profs) will understand
why they have to Handbrake convert their study film, whatever its original format, to mp4.

But that can't be inferred from the format... You need to read to be able to know if you can seek-back...

[Jean-Baptiste Kempf]

Thanks for making this topic, I was about to make it.

You could just say "not allowed" if not possible with that format.

Sure, it is not allowed with any formats for now.

Seriously, do you really believe we're not making this on purpose? It is a hard problem.

[Cantar4]

It is a hard problem.

for sure! but my test with h264 on the IINA player shows their reverse frame by frame method is very usable.
Note : you have to click the ‘Playback’ pane and press ‘alt’ to read the shortcuts,
e.g., ‘alt’ ‘cmd’ —> : next frame / immediate display.
.......‘alt’ ‘cmd’ <— : previous frame / immediate display (1sec delay on out-of-buffer frames if you explore too far back)

[Rémi Denis-Courmont]

my test with h264 on the IINA player shows their reverse frame by frame method is very usable.
note: you have to click the ‘Playback’ pane and press ‘alt’ to learn about the shortcuts,
e.g., ‘alt’ ‘cmd’ —> : next frame / immediate display.
.......‘alt’ ‘cmd’ <— : previous frame / immediate display (1sec delay on out-of-buffer frames if you explore too far back)

Ok so say H.264... How many different file formats did you try? What was the GOP length? Did you try with "infinite" GOP?

[Cantar4]

To bring low latency to video gamers, NVIDIA recommends to encode H.264 with an "infinite" Group of Pictures ;
but low latency is detrimental to image quality,
For film students and filmmakers there is no advantage to analyse a film coded under this GOP ,
for me the sweet spot (quality vs weight) seems to be a length of 96 to 240 frames.

PS: don't you think that VLC, born in Ecole Centrale Paris, should preferably accomplish
this June 2010 student wish ?

[Cantar4]

Ok so say H.264... How many different file formats did you try? What was the GOP length? Did you try with "infinite" GOP?

@Rémi
How do you know what's the GOP of an H.264 file ? Is there an app on the web or a tool within VLC to get access to the GOP length and I/B/P sequence too? Thanks.

[Rémi Denis-Courmont]

That was a rhetorical question. The GOP can be infinite, and reverse frame by frame cannot be implemented within reasonable time and memory boundaries in that case.

[brainburst]

Fixating on the codec is besides the point. I think that this feature could be easily implemented if they simply insert a video buffer after decoding and before display. Just for example insert a 30 frame video buffer to enable stepping back within the frame buffer. Most people do not want to step back that many frames. If they do just reengage play pause to fill the buffer again.

[brainburst]

Thanks for making this topic, I was about to make it.

You could just say "not allowed" if not possible with that format.

Sure, it is not allowed with any formats for now.

Seriously, do you really believe we're not making this on purpose? It is a hard problem.

No it's not a hard problem. You are simply trying the more difficult solution. Just enable option of a one or two second buffer in memory before presentation. All calls to step forward or backwards get called from the buffer. That way it doesn't matter what codec is involved.

[Cantar4]

That was a rhetorical question.

My search for an app or a tool to get access to the GOP length and I/B/P sequence was triggered by your rhetorical question.
It made me discover there is no means for us simple mortals to understand why some compressed films are watchable and some others not ; so, Remi, if you could help us find one...
.

[Jean-Baptiste Kempf]

Thanks for making this topic, I was about to make it.

You could just say "not allowed" if not possible with that format.

Sure, it is not allowed with any formats for now.

Seriously, do you really believe we're not making this on purpose? It is a hard problem.

No it's not a hard problem. You are simply trying the more difficult solution. Just enable option of a one or two second buffer in memory before presentation. All calls to step forward or backwards get called from the buffer. That way it doesn't matter what codec is involved.

And why 1 second and not 3? How do you decide the length? Also, 3 seconds of video can mean literality GB of RAM (yes, one video frame can be easily 30MB in the GPU memory).

And finally, if it is _that_ easy, then please share code to do that in VLC. We would love your patch.

[Cantar4]

Rémi Denis-Courmont : How many different file formats did you try? What was the GOP length?

@Rémi, by not answering my "how do I know what's the GOP length of an H.264 file ? "
you didn't help me answer your question...
... but I finally found a way : ffmpeg being installed in the CLI terminal, entering the following line, ffprobe -show_frames /(path to the media file) | grep pict_type
I got a list of the I/P/B type of all frames, hence an access to the variable length of the GOP due to images complexity and compression level.
.

[Rémi Denis-Courmont]

So what? You can answer the question for one video. The point is that you cannot answer the question for all videos. In general, the GOP may be arbitrarily long. As such, reverse frame by frame can be implemented, but with the minor limitatiosn of unbound memory usage and unbound processing time.

[Cantar4]

As such, reverse frame by frame can be implemented, but with the minor limitatiosn of unbound memory usage and unbound processing time.

"can be implemented" is good news.
I always convert the films to be analysed under H.264 (now hevc 265) with Handbrake.
At compression level 20, he GOPs are between 240 and 100 :

GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBB
BPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBB
BPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBPBBBPBBBPBBBPBBBPBBBPBBB
PBBBP 240 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBPBBBPBBBPBBBPBBB
PBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBP 108 CLOSED
GOP: IBBBPBBBPBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBB
PBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBB
PBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBPBBBPBBBPBBBPBBBP
BBBPP 240 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBB
BPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBB
BPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPP 194 CLOSED
GOP: IBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBB
BPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPBBBPP 142 CLOSED

I share the advice of the many other film-analysts, you could bound the buffer to one second only.
This would be more than enough to scrub back and forth an event or an edit, 5 to 10 picts would make us happy
--jp

For enquiring minds, here below the CLI tool a friend gave me to display the GOP lenghts:

Last login: Thu Feb 22 12:44:06 on ttys000
pc26:~ jp$ cd MyPython
pc26:MyPython jp$ python iframe-probe.py /Users/jp/8-35_Godard/C0008_Hndbrk.MP4
ffprobe version 3.4.2 Copyright (c) 2007-2018 the FFmpeg developers
built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4.2 --enable-shared --enabl
e-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=
clang --host-cflags= --host-ldflags= --disable-jack --enable-gpl --enable-libmp3
lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/jp/8-35_Godard/C0008_Hndbrk.MP4':
Metadata:
major_brand : mp42
minor_version : 512
compatible_brands: isomiso2mp41
creation_time : 2018-02-22T13:06:42.000000Z
encoder : HandBrake 20180203154437-155e573-master 2018020601
Duration: 00:00:38.59, start: -0.001333, bitrate: 3946 kb/s
Stream #0:0(und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 3745 kb/s, 24 fps, 24 tbr, 90k tbn, 24 tbc (default)
.

[daffdaemon]

Excuse my 2 cents; my being both new and NOT a programmer.
This conversation, here and in it's previous incarnation, has been going on for nearly 8 years!

I came to the forum to see about getting snapshots time-stamped in vid time vs sys time.
When I want a snapshot I hit pause when I see the frame I want (which always misses the frame, of course).
Hit ctrl-T & enter (which moves to the first frame of the second I'm at); then move forward to the frame I want (e,e,e,e).
Occasionally if the frame wanted is too close to the end of the second, I have to jump back to the previous second.
[BTW would be helpful here if video time display included decimal seconds (2 significant digits is plenty)]

Why doesn't a similar strategy work for this reverse frame problem?
Jump back, buffer a second of video, move forward to the frame wanted.
When reverse frame request moves out of range, buffer previous second.
If the video format is such that a 1 second buffer is too large, buffer less, 1/2 sec, 1/4 sec, whatever.

I know buffering has been previously suggested, I guess I just don't understand why it's not feasible.

I have been using VLC for years now, and I just want to add that it is a wonderful effort.

[Jean-Baptiste Kempf]

Why doesn't a similar strategy work for this reverse frame problem?

Because VLC is not precise enough for seeking like that. This will come, but requires MAJOR architecture changes in VLC. I would expect this to come in 4.0 or 5.0.

[Omnius]

I've been reading through all the forums about the difficulty of doing "frame back" in all scenarios and at the risk of suggesting something that someone else has already suggested, I'm going to anyway.

I get that the general solution of keeping frames handy any time you stop the video is very hard. It seems like it would be trivial to save a frame that was jumped to via the "frame ahead" function. So, once I hit pause, I have a valid frame, then when I hit "frame ahead" I get another frame, so now i could have two frames cached. Every time I hit "e" I get another frame cached. As long as I don't hit "play" or "jump to time" or anything else that requires resetting the stream, I could keep buffering frames so that I can then do a "frame back". At some point I would return to the point where I "stopped" (or started, if you'd rather) and there would be no more buffered frames.

For the person who is trying to watch exactly where that catch was made, or where the "wardrobe malfunction" happened (let's be real here) this would probably be good enough. As long as they jump to a time code before the "event" they are trying to scrub through and then scrub forward frame-by-frame through the event, then they would be able to back up frame by frame through that same event. There could be a limit to the number of frames cached (at some point memory becomes an issue), and it would probably require a special UI indication of the frame buffer as it's being used, to convey the limitation to the user, but the actual implementation of it sounds relatively easy as it's completely out of the original stream.

I know for most of the situations where I'm quickly going through my own videos this would be more than good enough (otherwise I have to launch Premiere, load up a video, etc. just to look at the 30-40 frames that I care about).

OK, let the flaming begin...

[Rémi Denis-Courmont]

I am pretty sure that people expect backward stepping further back than the last pause.

And even then, the amount of memory needed to keep all frames stepped through is unbound and very quickly gets enormous. One decoded HD frame is 4 MiB; one 4K HDR frame is maybe quadruple that.