4 ms·
I have a cli tool I maintain that finds visually similar images. As a fun experiment several years ago I extracted all the frames of Skyfall and all the frames
by donatj 2y ago
I have a cli tool I maintain that finds visually similar images.
As a fun experiment several years ago I extracted all the frames of Skyfall and all the frames of the first Harry Potter movie.
I then reconstructed Harry Potter frame by frame using the corresponding frame from Skyfall that was most visually similar.
The end result was far more indecipherable than I'd ever expected. The much darker color pallet of Harry Potter lead to the final result largely using frames from a single dark scene in Skyfall, with single frames often being used over and over. It was pretty disappointing given it took hours and hours to process.
Thinking about it now, there's probably a way to compensate for this. Some sort of overall pallette compensation.
- xnx 2y agoI'm interested in this. What's your similarity measure? I was going to write a frame comparison tool to make seamlessly looping video.
- deleted 2y ago[deleted]
- Retr0id 2y agoIf you had a much bigger corpus, and used some semantics-aware similarity metrics (think embeddings), you could maybe end up with something actually coherent
- dekhn 2y agoif you take this idea to the limit you'll basically generate a visual embedding equivalent to tvtropes.org
- photonboom 2y agoThat's an interesting idea. I wonder how well the film iris/barcodes could be used to figure out which movies make the best 'palette' to recreate a given scene. I spent some time earlier this year on creating mosaics of movie posters using other posters as tiles: https://joshmosier.com/posts/movie-posters/full-res.jpg https://joshmosier.com/posts/movie-posters/full-res.jpg (warning: 20mb file) Using this on each frame of a scene gave some good results with a fine enough grid even with no repeating tiles: https://youtu.be/GVHPi-FrDY4 https://youtu.be/GVHPi-FrDY4
- jazzyjackson 2y agoI imagine the best case outcome would look something like Jack Gallant's 2011 work on visual mind reading, where they trained a model on brain activity watching hundreds of hours of YouTube and then attempted to reconstruct a view of video not in the training set by correlating the new brain activity with frames from the input... Sorry I'm not explaining very clearly but there's a YouTube video of the result naturally :) https://youtu.be/nsjDnYxJ0bo https://youtu.be/nsjDnYxJ0bo Learned about this from Mary Lou Jepsens 2013 TED talk, what a throwback https://www.ted.com/talks/mary_lou_jepsen_could_future_devices_read_images_from_our_brains/ https://www.ted.com/talks/mary_lou_jepsen_could_future_devic...