6 ms·
Can anyone explain what does this encoder actually does and how is it different than any other encoder? The article almost seems to make it sound that this enco
by dogma1138 10y ago
Can anyone explain what does this encoder actually does and how is it different than any other encoder?
The article almost seems to make it sound that this encoder somehow reconstructed the film...
- foota 10y agoI believe it's similar to Google's deep dreaming created images, but with videos.
- kastnerkyle 10y agoThis technique is quite different from deep dream. An autoencoder is a model that is trained to compress then reconstruct an image. After seeing and trying to reconstruct many images, it learns how to make a "compressed representation" from original images by going through this compress -> uncompress scheme, which typically results in blurry/lossy reconstructions of images which are very different than the normal artifacts we see in lossy compression schemes like jpg, etc. In deep dream, a network whose goal is to predict image categories is "run in reverse" and used as a generative model, but it was not intended to be generative from the outset. Many times autoencoders are intended to be generative, or at least learn relationships between images (sometimes called a latent space/latent variable), where classification tasks care more about just getting the classification correct - no explicit concern on learning relationships between categories/images in those groups.
- manmal 10y agoWould it be possible to make it look better or even the same by increasing digits per frame (eg 1000 instead of 200)?
- snickmy 10y agoYes, technically yes. Although I'd say this is not what this experiment aims to. In the future we may be able to colouring b/w movies and increase fps via this automatic software.
- kastnerkyle 10y agoThe future is now! [0,1] [0] http://richzhang.github.io/colorization/ http://richzhang.github.io/colorization/ [1] https://www.youtube.com/watch?v=cizgVZ8rjKA https://www.youtube.com/watch?v=cizgVZ8rjKA
- kastnerkyle 10y agoYes, although the particular technique used here (variational autoencoder, or VAE) doesn't benefit from increasing the code space due to a particular penalty (KL divergence against a fixed N(0, 1) Gaussian prior) during training. So even by making it go to 1000 digits in the code space, the model will probably still only choose to use 5 or 10 dimensions unless the KL penalty is relaxed, making it closer to a standard autoencoder. An autoencoder with a much larger code space could be an improvement, or some newer works such as Adversarially Learned Inference / Adversarial Feature Learning [0, 1] or real NVP [2, 3] could probably do a much better job at the task, at the cost of increased computation. Also something like inpainting parts of the frames with pixel RNN [4] would be interesting. [0] http://arxiv.org/abs/1606.00704 http://arxiv.org/abs/1606.00704 [1] http://arxiv.org/abs/1605.09782 http://arxiv.org/abs/1605.09782 [2] http://www-etud.iro.umontreal.ca/~dinhlaur/real_nvp_visual/ http://www-etud.iro.umontreal.ca/~dinhlaur/real_nvp_visual/ [3] http://arxiv.org/abs/1605.08803 http://arxiv.org/abs/1605.08803 [4] http://arxiv.org/abs/1601.06759 http://arxiv.org/abs/1601.06759
- vintermann 10y agoGiven that Adversarially Learned Inference came out yesterday, the best approach may be to just wait a couple of months and see what the state of the art is then.
- kastnerkyle 10y agoSure - that is always an option especially in deep learning right now. But this current crop of models (counting in DCGAN and LAPGAN/Eyescream) has really made a leap in my eyes from before "oh cool generative model" to "are these thumbnails real?". They are really generating a lot of cohesive global structure, which is pretty awesome!
- mcv 10y agoAs far as I can tell, it just sent it through a deep learning network. Cool, but not exactly the same as an AI reconstructing the movie. It's just a distorted version of the original, and Warner Bros was not entirely incorrect in requesting its removal (though it doesn't hurt them much to allow it, because it's so heavily distorted).
- IshKebab 10y agoAutoencoders are a special type of deep neural network where there is a small middle layer, and the labels are the same as the input. The idea is you train the network to reproduce the input, but it has to pass the information through the small middle layer which has relatively few neurons. Thus the network learns a good (hopefully) way to represent typical inputs with not much information. You have automatically generated an encoding (hence the name). You're right it's not really 'reconstructing' the film in the way they imply. But it isn't just a distorted version of the original either. The really important question is how big the middle layer is (and hence how much it has learned / how good the compression is). Unless this is just done for artistic purposes, which seems to be the case.
- Mithaldu 10y agoThis is possibly the next generation of video/audio/image codecs. What he did was create a specialized compression algorithm that works very well to compress the data that is each frame of Blade Runner, and decompress it (lossily, like mp3) back into a video stream. To put this into perspective: Blade Runner is 117 minutes long. At 25 frames per second, that is 175_500 frames. As he says, the input data he used was 256x144 with 3 colour channels, meaning each frame was 110_592 bytes. This results in an input amount of 18_509 MB, uncompressed. His neural network compresses each image down to 200 floats though, i.e. 800 bytes. So the whole movie as compressed by the NN is 132 MB. A friend of mine who works with neural networks estimated his NN to be roughly 90MB at 256x144, so the storage needed for the movie he created is about 222 MB. This means that if this technology can be made fast enough, and the reproduction high enough in fidelity, we could be looking at replacing compressors (Fraunhofer MP3, Lame, MP4, DIVX, JPEG) made to handle all types of input equally well; with compressors that can are so good for one specific set of data that compressor + data is smaller than anything the traditional compressors could create. And even if its not fast enough, it can still be a very efficient compressor, trading size for CPU. Plus, lastly, the NN itself can be compressed as well, and classic compressors can possibly be applied to the frame data.
- radarsat1 10y agoExcept that in my understand, the decoded version looks like shit. So, while optimistic that this technology might be quite good eventually, more heuristics are probably needed for the right way to extract an optimal encoding. e.g. see the screenshots here: http://www.eteknix.com/blade-runner-gets-trippy-auto-encoded-ai/ http://www.eteknix.com/blade-runner-gets-trippy-auto-encoded...
- zodiac 10y agoIt applies the unsupervised learning technique of https://en.wikipedia.org/wiki/Autoencoder https://en.wikipedia.org/wiki/Autoencoder As for why it's important and not just another encoder like mpeg4, see my response at https://news.ycombinator.com/item?id=11830437 https://news.ycombinator.com/item?id=11830437