5 ms·
I've been waiting for something to implement this concept for so long, and I'm so happy to finally get a chance to explore how it works in practice! It's reall
by crazygringo 6y ago
I've been waiting for something to implement this concept for so long, and I'm so happy to finally get a chance to explore how it works in practice!
It's really fascinating to zoom comparing the compressed version with the original -- the "general idea" is all there and the quality is roughly the same, but strands of hair move to different positions, skin pores are in a completely different pattern, etc. -- it's all regenerated to look "right", but at the lowest level the texture isn't "real".
Which is so fascinatingly different from JPEG or other lossy schemes -- the compression artifacts are obviously exactly that, you'd never confuse them for real detail. While here, you could never guess what is real detail vs. what is reconstructed detail.
My hunch is that this won't actually take off at first for images, because images on the internet don't use up that much of our bandwidth anyways, and the performance and model storage requirements here are huge.
BUT -- if the same idea can be applied to 1080p video with a temporal dimension, so that even if reconstructed textures are "false" they persist across frames, then suddenly the economics here would start to make sense: dedicated chips for encoding and decoding, with a built-in 10 GB model in ROM, trained on a decent proportion of every movie and TV show ever made... imagine if 2-hour movies were 200 MB instead of 2 GB?
(And once we have chips for video, they could be trivially re-used for still images, much like Apple reuses h.265 for HEIC.)
- dividuum 6y agoRegarding your last paragraph: I wouldn't be surprised if existing "upscale" chips already work similar to what you suggested: They convert lower res content to (for example) 4K. See https://www.sony.com/electronics/4k-resolution-4k-upscaling-tvs https://www.sony.com/electronics/4k-resolution-4k-upscaling-... for an example.
- swiley 6y agoI think it will be extremely popular. Most people are concerned about how computers look. Wether or not the behavior is correct usually only bothers people who spend time working on computers. I’m not sure if it’s because the first group is just used to life being unpredictable and don’t realize things could be better, or if they really don’t care.
- ksec 6y ago> because images on the internet don't use up that much of our bandwidth anyways, On the Internet, Yes. But for web pages, Image is still by far the largest payload. It could be anywhere between 60 to 90% of total content size. Thinking in the context of latency, where you could benefits with images appearing much quicker or instantaneous ( embedding inside the HTML ) because of its reduction. Especially on first screen.
- crazygringo 6y agoProgressive JPEG's go a long way towards that though, no?
- ksec 6y agoNot JPEG in its current form. ( May be JPEG XL ) The user experience is horrible. And during the last HN discussion on Medium using progressive JPEG it turns out a lot of people have similar thoughts.
- IshKebab 6y ago> Image is still by far the largest payload. A good point that has been made before is that images are often by far the biggest part of a web page, but most sites still don't even do basic optimisations like resizing them to the viewing size, using WebP, using pngcrush, etc. That said, this would be cool for making really well optimised sites.
- phs318u 6y agoGiven the seemingly random nature of the compression "artefacts", versus the more predictable compression artefacts with other algos, I wonder if this would limit use of such images in scenarios where the resulting images had to be relied on in court as evidence of something?
- crazygringo 6y agoAbsolutely. Exactly for this reason, I would hope not to see this ever become the default format for cell phone cameras unless there were somehow explicit guarantees around what is or is not faithful. Same with security cameras, etc. But, anywhere where the purpose of output is social/entertainment, it seems this would be freely adopted. Profile pictures, marketing images, videos and movies. If could very much mean, however, that a photo on someone's Facebook profile might not be admissable as evidence, if used to identify an exact match with someone, rather than just "someone who looks similar".
- sitkack 6y agohttp://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
- phs318u 6y agoWow! The implications are huge. At least the Xerox bug had the benefit of being reproducible. An AI algo might _occasionally_ spit out an incorrect digit, and other times not. Really, this highlights the need for explicability in AI/ML.
- viraptor 6y agoI just can't wait for things that people could spot. Trained networks can "imagine" some elements existing in the same way that they recreate the different texture. If this gets applied based on existing movies, there will definitely be a time when someone in the background raises their hand to ear to brush their hair, but suddenly a mobile phone materialises from a shadow on their hand. It will be just a few pixels that you'd miss... But it will be there to spot, like distant power lines on period dramas.
- rrrrrrrrrrrryan 6y agoGiven that our networks can generally handle 2GB movies now, can you imagine streaming a 4K movie, but instead of using 20GB it only takes 2GB?
- dehrmann 6y agoIt's going to be funny when the neural network is clever enough to hallucinate sequels.
- gfody 6y agothere's a similar paper shared here a couple years ago: https://data.vision.ee.ethz.ch/aeirikur/extremecompression/ https://data.vision.ee.ethz.ch/aeirikur/extremecompression/ ..when compared to low bitrate dct-based compressors it looks impressive, but then compared to the original source the compression artifacts stuff like incorrect license plate numbers, building architecture, species of trees, etc.
- bamfito 6y agoYes! Nvidia shows what is possible with dlss2 as well. I'm very curious about what we will be able to do with voice as well. Perhaps you will sync voice profiles from your familiy/friends for better compression or better reconstruction. We will also see models which will have higher details through incorporating all frames and enhancing existing material. We see this already anyway in gaming as well.