4 ms·

To do this well you need to keep the old frames around...or go back to the previous keyframe and re-render. That might be hard if your design is playback-optimi

by mannyv 2y ago

To do this well you need to keep the old frames around...or go back to the previous keyframe and re-render. That might be hard if your design is playback-optimized.

Dylan16807 2y ago
Re-rendering shouldn't be hard, it's just a specialized version of seeking. VLC has seeking. The claim that you would have to decode all previous frames in the entire video is... completely baffling to see coming from the dev. He's arguing a stupid technicality that a video might not have keyframes. That's not a reason to omit the feature entirely.
- weaksauce 2y ago
  yeah that's a weird edge case that's not really worth considering. it's obviously something that can be done technically even if some edge cases are not performant.
- mannyv 2y ago
  You misunderstand. The point is you have to go backwards to find a keyframe, then render forwards from that. Going backwards might be hard because if you structured your code in certain ways you may not be able to go backwards efficiently. You can "seek", but how far back to you want to seek? A second? Two seconds? current-X frames? key frames may be in a standard cadence, but they may not be. So again, how far back do you seek to go back one frame? And keyframes may be abstracted away from the player itself, since really, the codecs are the ones that deal with that stuff. For example, I believe mjpeg doesn't do frame differencing (I'm probably wrong about that). The ideal implementation would save the last X frames then re-render once you go back like X/2 frames. But again, it depends.
  - Aeolun 2y ago
    Since I can drag the bar backwards in VLC and have it resume playback apparently this is already implemented? This would be a very narrow use of that.
  - Dylan16807 2y ago
    I don't misunderstand. The program already has seeking code, and if it needs to aim back slightly so that it can be more precise that's not a massive change to how it already works. Efficiency is not as important as having the feature at all. "Go back 5 seconds and then run forward to the right frame" is a sufficient algorithm, as long as it can track and combine multiple presses of the previous-frame key. Improvements can come later. Maybe buffering, maybe tracking keyframes, maybe other things. But this is a big case of letting the perfect be the enemy of the good. If it fails to find a keyframe, that sucks, but 99% of the time it'll work.
- throwaway22032 2y ago
  The strange thing is that the same argument is true for seeking in general. Going back from frame 500000 to frame 499999 is in the limiting case as complex as seeking from 1 to 499999, and in most cases far better. I think the forum thread would be better answered "you do it, I don't need this feature" which is basically the gist of it and is a completely fair answer.
  - a1o 2y ago
    It's not even about doing it, once added you have to maintain it, and then tomorrow a new format arises that makes this more a hassle, or some memory issue in an existing format is fixed in a way that changes it's memory profile and now VLC will crash with an out of memory. Seriously, I don't get these people that have infinite demands from open source developers and contribute zero.
    - armada651 2y ago
      You already have to maintain seeking code, stepping back one frame is just a specialized case for your seeking code.
      - mrob 2y ago
        You already have to maintain code for seeking by time. There's not necessarily any easy way to convert between time and frame count.
        _flux 2y ago
        VLC knows the timestamp of the current frame. From that information it can seek to a frame that is before the current frame, possibly by just subtracting the inverse of frame rate from the current frame's timestamp and if seeking to that time results in seeking to the same frame, try again a bit older. I'm relatively sure this can be implemented in terms of timestamp-based seeking. Quite possibly the metadata of the frames is already in the memory, further simplifying the process.
seanthemon 2y ago
Just a ridiculous idea, could a reversed version of the video be kept around so that if you hit frame backwards it's frame forwards operation on the reversed video so you can show that frame, if you continue playback the head goes to the closest second of the forward video
- armada651 2y ago
  This would require completely re-encoding the video or keeping all decoded frames around. This is what he referred to as the "unbounded memory" solutions.
  - seanthemon 2y ago
    That makes sense, thank you!
DonHopkins 2y ago
Again: There is no reason to start caching previous frames until AFTER the user has paused and pressed the "back frame" key. Only THEN does it need to rewind to the previous i-frame and re-render and cache frames. And there is no measurable cost to remembering the timestamp of the last i-frame, so you know where to rewind to.

by mannyv 2y ago

To do this well you need to keep the old frames around...or go back to the previous keyframe and re-render. That might be hard if your design is playback-optimized.

Dylan16807 2y ago

Re-rendering shouldn't be hard, it's just a specialized version of seeking. VLC has seeking. The claim that you would have to decode all previous frames in the entire video is... completely baffling to see coming from the dev. He's arguing a stupid technicality that a video might not have keyframes. That's not a reason to omit the feature entirely.

weaksauce 2y ago
yeah that's a weird edge case that's not really worth considering. it's obviously something that can be done technically even if some edge cases are not performant.
mannyv 2y ago
You misunderstand. The point is you have to go backwards to find a keyframe, then render forwards from that. Going backwards might be hard because if you structured your code in certain ways you may not be able to go backwards efficiently. You can "seek", but how far back to you want to seek? A second? Two seconds? current-X frames? key frames may be in a standard cadence, but they may not be. So again, how far back do you seek to go back one frame? And keyframes may be abstracted away from the player itself, since really, the codecs are the ones that deal with that stuff. For example, I believe mjpeg doesn't do frame differencing (I'm probably wrong about that). The ideal implementation would save the last X frames then re-render once you go back like X/2 frames. But again, it depends.
- Aeolun 2y ago
  Since I can drag the bar backwards in VLC and have it resume playback apparently this is already implemented? This would be a very narrow use of that.
- Dylan16807 2y ago
  I don't misunderstand. The program already has seeking code, and if it needs to aim back slightly so that it can be more precise that's not a massive change to how it already works. Efficiency is not as important as having the feature at all. "Go back 5 seconds and then run forward to the right frame" is a sufficient algorithm, as long as it can track and combine multiple presses of the previous-frame key. Improvements can come later. Maybe buffering, maybe tracking keyframes, maybe other things. But this is a big case of letting the perfect be the enemy of the good. If it fails to find a keyframe, that sucks, but 99% of the time it'll work.
throwaway22032 2y ago
The strange thing is that the same argument is true for seeking in general. Going back from frame 500000 to frame 499999 is in the limiting case as complex as seeking from 1 to 499999, and in most cases far better. I think the forum thread would be better answered "you do it, I don't need this feature" which is basically the gist of it and is a completely fair answer.
- a1o 2y ago
  It's not even about doing it, once added you have to maintain it, and then tomorrow a new format arises that makes this more a hassle, or some memory issue in an existing format is fixed in a way that changes it's memory profile and now VLC will crash with an out of memory. Seriously, I don't get these people that have infinite demands from open source developers and contribute zero.
  - armada651 2y ago
    You already have to maintain seeking code, stepping back one frame is just a specialized case for your seeking code.
    - mrob 2y ago
      You already have to maintain code for seeking by time. There's not necessarily any easy way to convert between time and frame count.
      - _flux 2y ago
        VLC knows the timestamp of the current frame. From that information it can seek to a frame that is before the current frame, possibly by just subtracting the inverse of frame rate from the current frame's timestamp and if seeking to that time results in seeking to the same frame, try again a bit older. I'm relatively sure this can be implemented in terms of timestamp-based seeking. Quite possibly the metadata of the frames is already in the memory, further simplifying the process.

seanthemon 2y ago

Just a ridiculous idea, could a reversed version of the video be kept around so that if you hit frame backwards it's frame forwards operation on the reversed video so you can show that frame, if you continue playback the head goes to the closest second of the forward video

armada651 2y ago
This would require completely re-encoding the video or keeping all decoded frames around. This is what he referred to as the "unbounded memory" solutions.
- seanthemon 2y ago
  That makes sense, thank you!

DonHopkins 2y ago

Again: There is no reason to start caching previous frames until AFTER the user has paused and pressed the "back frame" key. Only THEN does it need to rewind to the previous i-frame and re-render and cache frames. And there is no measurable cost to remembering the timestamp of the last i-frame, so you know where to rewind to.