8 ms·
Blade Runner re-encoded using neural networks
- Sam_Smith 10y agoAs mentioned, there is likely no legal precedent regarding this type of video creation resulting from AI. We will have to watch how the first cases are presented by the original owners of the content to see which methods they will use to hinder this sort of "recreation." -- Sam Smith Web Developer and Aspiring Chef~ Large file transfers made easy. www.innorix.com/en/DS
- ljf 10y agoJust guessing but the downvotes may be due to the 'sig' you've included coming across as spammy. You might notice that no one else here is doing that, and it likely isn't a good way to encourage interest in your product. How about submitting a post where you discuss some of the problems you've solved?
- dogma1138 10y agoCan anyone explain what does this encoder actually does and how is it different than any other encoder? The article almost seems to make it sound that this encoder somehow reconstructed the film...
- foota 10y agoI believe it's similar to Google's deep dreaming created images, but with videos.
- kastnerkyle 10y agoThis technique is quite different from deep dream. An autoencoder is a model that is trained to compress then reconstruct an image. After seeing and trying to reconstruct many images, it learns how to make a "compressed representation" from original images by going through this compress -> uncompress scheme, which typically results in blurry/lossy reconstructions of images which are very different than the normal artifacts we see in lossy compression schemes like jpg, etc. In deep dream, a network whose goal is to predict image categories is "run in reverse" and used as a generative model, but it was not intended to be generative from the outset. Many times autoencoders are intended to be generative, or at least learn relationships between images (sometimes called a latent space/latent variable), where classification tasks care more about just getting the classification correct - no explicit concern on learning relationships between categories/images in those groups.
- manmal 10y agoWould it be possible to make it look better or even the same by increasing digits per frame (eg 1000 instead of 200)?
- snickmy 10y agoYes, technically yes. Although I'd say this is not what this experiment aims to. In the future we may be able to colouring b/w movies and increase fps via this automatic software.
- kastnerkyle 10y agoThe future is now! [0,1] [0] http://richzhang.github.io/colorization/ http://richzhang.github.io/colorization/ [1] https://www.youtube.com/watch?v=cizgVZ8rjKA https://www.youtube.com/watch?v=cizgVZ8rjKA
- kastnerkyle 10y agoYes, although the particular technique used here (variational autoencoder, or VAE) doesn't benefit from increasing the code space due to a particular penalty (KL divergence against a fixed N(0, 1) Gaussian prior) during training. So even by making it go to 1000 digits in the code space, the model will probably still only choose to use 5 or 10 dimensions unless the KL penalty is relaxed, making it closer to a standard autoencoder. An autoencoder with a much larger code space could be an improvement, or some newer works such as Adversarially Learned Inference / Adversarial Feature Learning [0, 1] or real NVP [2, 3] could probably do a much better job at the task, at the cost of increased computation. Also something like inpainting parts of the frames with pixel RNN [4] would be interesting. [0] http://arxiv.org/abs/1606.00704 http://arxiv.org/abs/1606.00704 [1] http://arxiv.org/abs/1605.09782 http://arxiv.org/abs/1605.09782 [2] http://www-etud.iro.umontreal.ca/~dinhlaur/real_nvp_visual/ http://www-etud.iro.umontreal.ca/~dinhlaur/real_nvp_visual/ [3] http://arxiv.org/abs/1605.08803 http://arxiv.org/abs/1605.08803 [4] http://arxiv.org/abs/1601.06759 http://arxiv.org/abs/1601.06759
- 10y ago
- mcv 10y agoAs far as I can tell, it just sent it through a deep learning network. Cool, but not exactly the same as an AI reconstructing the movie. It's just a distorted version of the original, and Warner Bros was not entirely incorrect in requesting its removal (though it doesn't hurt them much to allow it, because it's so heavily distorted).
- IshKebab 10y agoAutoencoders are a special type of deep neural network where there is a small middle layer, and the labels are the same as the input. The idea is you train the network to reproduce the input, but it has to pass the information through the small middle layer which has relatively few neurons. Thus the network learns a good (hopefully) way to represent typical inputs with not much information. You have automatically generated an encoding (hence the name). You're right it's not really 'reconstructing' the film in the way they imply. But it isn't just a distorted version of the original either. The really important question is how big the middle layer is (and hence how much it has learned / how good the compression is). Unless this is just done for artistic purposes, which seems to be the case.
- Mithaldu 10y agoThis is possibly the next generation of video/audio/image codecs. What he did was create a specialized compression algorithm that works very well to compress the data that is each frame of Blade Runner, and decompress it (lossily, like mp3) back into a video stream. To put this into perspective: Blade Runner is 117 minutes long. At 25 frames per second, that is 175_500 frames. As he says, the input data he used was 256x144 with 3 colour channels, meaning each frame was 110_592 bytes. This results in an input amount of 18_509 MB, uncompressed. His neural network compresses each image down to 200 floats though, i.e. 800 bytes. So the whole movie as compressed by the NN is 132 MB. A friend of mine who works with neural networks estimated his NN to be roughly 90MB at 256x144, so the storage needed for the movie he created is about 222 MB. This means that if this technology can be made fast enough, and the reproduction high enough in fidelity, we could be looking at replacing compressors (Fraunhofer MP3, Lame, MP4, DIVX, JPEG) made to handle all types of input equally well; with compressors that can are so good for one specific set of data that compressor + data is smaller than anything the traditional compressors could create. And even if its not fast enough, it can still be a very efficient compressor, trading size for CPU. Plus, lastly, the NN itself can be compressed as well, and classic compressors can possibly be applied to the frame data.
- radarsat1 10y agoExcept that in my understand, the decoded version looks like shit. So, while optimistic that this technology might be quite good eventually, more heuristics are probably needed for the right way to extract an optimal encoding. e.g. see the screenshots here: http://www.eteknix.com/blade-runner-gets-trippy-auto-encoded-ai/ http://www.eteknix.com/blade-runner-gets-trippy-auto-encoded...
- zodiac 10y agoIt applies the unsupervised learning technique of https://en.wikipedia.org/wiki/Autoencoder https://en.wikipedia.org/wiki/Autoencoder As for why it's important and not just another encoder like mpeg4, see my response at https://news.ycombinator.com/item?id=11830437 https://news.ycombinator.com/item?id=11830437
- ole_roy 10y agoBut no audio, right? So, you'd think that alone would be a strong indicator that there was no intent (or capacity) to profit from the reconstructed version.
- timlyo 10y agoIt's saying down for scheduled maintenance for me, anyone got a mirror?
- kefka 10y agoSure do. If you're running IPFS locally use this link: http://127.0.0.1:8080/ipfs/QmPo9savMewcGCzUhzNQi96b6gSSoQ4k4DFEVkF51dCU83 http://127.0.0.1:8080/ipfs/QmPo9savMewcGCzUhzNQi96b6gSSoQ4k4... If not, use this one (public IPFS gateway) http://ipfs.io/ipfs/QmPo9savMewcGCzUhzNQi96b6gSSoQ4k4DFEVkF51dCU83 http://ipfs.io/ipfs/QmPo9savMewcGCzUhzNQi96b6gSSoQ4k4DFEVkF5... Be aware, there is no sound with this video. You'll have to add it from your own sources (nor did I want to add the sound from my dvd set). And frankly, IPFS is awesome :)
- eprime 10y agoPrevious discussion https://news.ycombinator.com/item?id=11766063 https://news.ycombinator.com/item?id=11766063 based on the original (1) article https://medium.com/@Terrybroad/autoencoding-blade-runner-88941213abbe https://medium.com/@Terrybroad/autoencoding-blade-runner-889... (1) cannot access this Vox article, it seems offline, so cannot compare.
- teh_klev 10y agoThis is the original article by Terrence Broad: https://medium.com/@Terrybroad/autoencoding-blade-runner-88941213abbe https://medium.com/@Terrybroad/autoencoding-blade-runner-889... and its appearance on HN the other week: https://news.ycombinator.com/item?id=11766063 https://news.ycombinator.com/item?id=11766063
- dang 10y agoThanks; forgot about that one.
- nurikolan 10y agoCorrect me if I'm wrong, but this is simply a compression+decompression of the original film. This so-called 're-representation' is just the result of the dimensionality reduction.
- mhax 10y agoThis reminds me very much of Parag Mital's work on audiovisual resynthesis: http://pkmital.com/home/ http://pkmital.com/home/ IIRC he has been beset with takedown notices for his 'smash up' videos which resynthesise the weeks #1 most viewed youtube video from the next 9 most viewed videos http://pkmital.com/home/youtube-smash-up/ http://pkmital.com/home/youtube-smash-up/
- annalewiss 10y agoThese complexities and nuances of sci-fi culture and artificial learning were quite possibly lost on whoever decided to file the takedown claim for Warner Bros.
- stefs 10y ago> whoever decided to file the takedown maybe that was a bot, which would add another layer to the story.
- intoverflow2 10y agoThe idea of this is way more interesting than the actual result. Unfortunately it just ends up looking like something run through content aware fill rather than anything interesting visually.
- svantana 10y agoThis article alludes to, but never actually brings up, something much more interesting than the project in question: How can copyright coexist with human-level automatic analysis and synthesis of works of art? For example, Spotify is flooded with "covers" of hit songs that are made to sound as similar to the original as possible. In my understanding, it doesn't matter how similar it sounds, as long as it's "remade", it's considered a new work of art. So what happens when the cover song process can be automated? It seems likely that courts and lawmakers will need to define a sphere of similarity around each work of art, within which all points are considered copies.
- buro9 10y agoMusic has that sorted already. There is copyright. There is publishing right. If you make a cover of a recording, it is the latter that you are involved with. You'd have to go to the publishers of a piece of music to license the right to publish your interpretation of the work. Obviously this post simplifies, but it's already answered.
- rangibaby 10y agoRe: publishing rights, you can make a cover version of a song using a "compulsory mechanical license", whether the owner likes it or not.
- ionwake 10y agoSo are remixes allowed, or do you need permission from the original publisher?
- jstelly 10y agoAssuming the remix uses the actual recording you need permission from the person who made that recording (the recording is covered by copyright).
- squeaky-clean 10y ago
- klakier 10y agoBullshit... this is still encoding. It does not matter that it is using neural network to encode frame. Guys, when you get over this neural network hype? I can as well create and encoder using some machine learning, to create a blurry and inferior version of the movie. But what's the point? I could as well used other prediction method with some kind of memory, much superior and nobody will enjoy it. Summing up: nothing impressive here.
- l3m0ndr0p 10y agoI agree. I was wondering why this is such a big deal. When a "neural networking" computer has a flash of inspiration and creates a screen play for a movie, and creates a virtual world based on that screen play. Then we'll have a something to talk about.
- Mithaldu 10y agoSee my explanation for why this matters here: https://news.ycombinator.com/item?id=11830140 https://news.ycombinator.com/item?id=11830140
- l3m0ndr0p 10y agoOH. Thank you. Now that makes a bit more sense & it's looks like an impressive feat for data compression.
- tacos 10y agoAn explanation written by someone who actually did it a decade ago: https://www.quora.com/What-is-the-potential-of-neural-networks-in-data-compression https://www.quora.com/What-is-the-potential-of-neural-networ...
- zodiac 10y ago> But what's the point? Autoencoders don't really have much practical use now, but that's not the point of them. The point is we really want to figure out how to do unsupervised learning well and autoencoders are one of the few ways of doing it. We want to do unsupervised learning well because most learning that humans do is unsupervised. The idea behind autoencoding is that by forcing the network to try to learn efficient ways to compress the data, it could learn important features of the data. The fact that the pictures are so blurry means that this doesn't work very well, but that's why it's a research problem. Autoencoders don't work well, but some unsupervised techniques that extend on them do, and we get impressive results like https://arxiv.org/pdf/1511.06434.pdf https://arxiv.org/pdf/1511.06434.pdf (see page 5) where the network learns to generate natural-looking bedroom images.
- nickpsecurity 10y agoThe worst part about this encoder is it thinks it's dealing with static pictures in plenty of cases. The version on the right simply has no movement while movie continues to play on left. Also, faces don't move when one person speaks. That's a no go even if the picture quality didn't look like crap.
- Houshalter 10y agoIt depends what your goal is. This is quite impressive for compressing each image down to 200 bytes. If you were actually using this for compression, this could be useful.You could store the "diff" between the reconstruction and the actual pixels, which would take many fewer bits than storing the whole image. Video compression already does something like this, storing a frame, and then the following frames just being diffs to the pixels of the previous frame. The goal of this project seems to be more artistic, so the poor quality might even be desirable.
- nickpsecurity 10y agoIt's not impressive to get an image small if you delete a bunch of critical stuff out of it plus warp what remains. Ill agree on the art project part, though.
- Houshalter 10y agoWhat do you expect from compressing an image to 200 bytes? It's impressive that it has as much detail as it does.
- nickpsecurity 10y agoI expect that decompressing the image produces one that looks close enough to the original to meaningfully call it compression. This I call mostly image deletion.
- jegoodwin3 10y agoDiscussions would be more focused if participants were aware of the AFC test -- Abstraction-Filtration-Comparison https://en.wikipedia.org/wiki/Abstraction-Filtration-Comparison_test https://en.wikipedia.org/wiki/Abstraction-Filtration-Compari... Will that test be applied here? Does it need to be revised in light of deep learning?