4 ms·

How would one most easily construct a pipeline or script to either split apart an mp4 into images, process them all with this tool, and then recombine into a vi

How would one most easily construct a pipeline or script to either split apart an mp4 into images, process them all with this tool, and then recombine into a video, or else modify this tool to take video as input? Anyone have any ideas / suggestions for a path to try that might require the least installation and scripting? And can anyone tell whether it's deterministic or not as far as whether the same input generates the same output each time?

dawnerd 6y ago
Ffmpeg can: https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%20every%20X%20seconds%20of%20the%20video https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%... https://superuser.com/a/624573 https://superuser.com/a/624573 You’ll just need to figure out the middle bit, shouldn’t be too hard though.
thih9 6y ago
Regarding actual input and output, I assume that’s relatively easy with ffmpeg. As for results, the challenge might be keeping the illusion of movement from frame to frame. It’s likely small changes to the input image result in largely different positioning of the shapes. When watched as a movie, this wouldn’t result in a fluid motion.
- boxfire 6y ago
  With the way this optimizes it would produce crap, even if using the same seed frame to frame. One very interesting prospect is in taking the motion vectors out of the video and applying them to overlapping primitives in the shape data form as a first step to apply when fitting the shapes, that is to do this greedy hill climb in tandem with mpeg motion decoding. Maybe it wouldn't be too hard but it's not something I've got the motivation to try out and tweak right now
  - boxfire 6y ago
    And more on this, maybe seeding the next frame to favor the motion vectors mutation of the shape before this libraries random seed mutation
riidom 6y ago
A filter that works on a still image doesn't necessarily work well on motion graphics, because you need a certain stability from frame to frame.
suby 6y ago
I've started work down this path for a project I am working on. I downloaded some footage of satellites flying over the earth from NASA, used FFMPEG to extract each frame of the video, wrote a ruby script which runs over every image in a folder and calls primitive (https://github.com/fogleman/primitive https://github.com/fogleman/primitive) on each image. The final step is to recombine it into a single video, which I haven't done yet because I haven't yet implemented video playback in the program I'm writing. In the meantime I'm just loading each image on the fly and playing it as a sprite animation. This results in low FPS but it gets the gist across. https://www.youtube.com/watch?v=SBFGqxWYyvI&feature=youtu.be https://www.youtube.com/watch?v=SBFGqxWYyvI&feature=youtu.be I'm thinking it'd still look pretty choppy even if it was a 60 FPS video though. I'm not yet sure how I'm going to smooth it out. I can share the FFMPEG command I used / script I made with you if you want, but I'd have to find it first (this was maybe a year ago). I'm sure the script is garbage. I do not believe the go program that I am using is deterministic. You can run the same command on an image twice and it will produce a different result. It might be possible to make it deterministic, but it's not something I've wanted so I haven't looked into it.
- istorical 6y ago
  I actually know how to do the splitting and recombining part pretty well using Da Vinci Resolve (kind of like the open source version of Adobe Premiere / Final Cut Pro) but I was wondering what the easiest tool would be to process. Do you have to just call into the python script process.py primitive? Or do you call into that main.go? It's just this simple IO that I feel like would be the hard part. Your video looks really good! If you wanna take 2 min to see if you can find it, that might be super helpful for a VR video project I'm doing!

dawnerd 6y ago

Ffmpeg can: https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%20every%20X%20seconds%20of%20the%20video https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%... https://superuser.com/a/624573 https://superuser.com/a/624573 You’ll just need to figure out the middle bit, shouldn’t be too hard though.

thih9 6y ago

Regarding actual input and output, I assume that’s relatively easy with ffmpeg. As for results, the challenge might be keeping the illusion of movement from frame to frame. It’s likely small changes to the input image result in largely different positioning of the shapes. When watched as a movie, this wouldn’t result in a fluid motion.

boxfire 6y ago
With the way this optimizes it would produce crap, even if using the same seed frame to frame. One very interesting prospect is in taking the motion vectors out of the video and applying them to overlapping primitives in the shape data form as a first step to apply when fitting the shapes, that is to do this greedy hill climb in tandem with mpeg motion decoding. Maybe it wouldn't be too hard but it's not something I've got the motivation to try out and tweak right now
- boxfire 6y ago
  And more on this, maybe seeding the next frame to favor the motion vectors mutation of the shape before this libraries random seed mutation

riidom 6y ago

A filter that works on a still image doesn't necessarily work well on motion graphics, because you need a certain stability from frame to frame.

suby 6y ago

I've started work down this path for a project I am working on. I downloaded some footage of satellites flying over the earth from NASA, used FFMPEG to extract each frame of the video, wrote a ruby script which runs over every image in a folder and calls primitive (https://github.com/fogleman/primitive https://github.com/fogleman/primitive) on each image. The final step is to recombine it into a single video, which I haven't done yet because I haven't yet implemented video playback in the program I'm writing. In the meantime I'm just loading each image on the fly and playing it as a sprite animation. This results in low FPS but it gets the gist across. https://www.youtube.com/watch?v=SBFGqxWYyvI&feature=youtu.be https://www.youtube.com/watch?v=SBFGqxWYyvI&feature=youtu.be I'm thinking it'd still look pretty choppy even if it was a 60 FPS video though. I'm not yet sure how I'm going to smooth it out. I can share the FFMPEG command I used / script I made with you if you want, but I'd have to find it first (this was maybe a year ago). I'm sure the script is garbage. I do not believe the go program that I am using is deterministic. You can run the same command on an image twice and it will produce a different result. It might be possible to make it deterministic, but it's not something I've wanted so I haven't looked into it.

istorical 6y ago
I actually know how to do the splitting and recombining part pretty well using Da Vinci Resolve (kind of like the open source version of Adobe Premiere / Final Cut Pro) but I was wondering what the easiest tool would be to process. Do you have to just call into the python script process.py primitive? Or do you call into that main.go? It's just this simple IO that I feel like would be the hard part. Your video looks really good! If you wanna take 2 min to see if you can find it, that might be super helpful for a VR video project I'm doing!