6 ms·

Compressing Images with Stable Diffusion

97 points by jstanley 4y ago

jstanley 4y ago
I've seen other Stable Diffusion examples where it starts with a low quality image and a prompt, and makes a higher-quality version. I'd be really interested to see this work extended with something that transmits (say) a 64x64 pixel copy of the original image and uses it as source material for Stable Diffusion to work on. Depending on the parameters it might be a legitimately viable image compression algorithm, at least for some types of images.
- Morgawr 4y ago
  Hilarious post aside, I've been using Stable Diffusion to generate "ideas" in large volume by creating a lot of images from a single (very generic) prompt with very low accuracy/depth levels, then when I find something that might be interesting, I fire up img2img and iterate over that with better accuracy, resolution, and depth. The results are pretty amazing.
  - stavros 4y ago
    That's a great idea! I've been curious, isn't img2img basically just more SD passes? Ie you generate something with 5 passes, then img2img just runs the other 95 over that to "complete" the image. Basically, as far as I understand it (which is not much), txt2img generates a very rough blob from a prompt, then img2img does additional passes over that to refine it to something that has much higher quality. Is this incorrect?
    - Morgawr 4y ago
      I'm actually not sure how it works specifically because I don't know much about the technology behind it. All I know is that if I change the prompt (compared to the original prompt given by the txt2img script) and finetune the parameters I can get some *very* differing results that only keep a certain "style" or vibe. For example I generated [0] as my base image, I passed it into a dithering filter to make it look more "retro" and got [1] (was going for retro PC98 visual novel style backgrounds), and then [3] are all new results generated from it (I added stuff like "village" or "trees" etc in the new prompt and also made the resolution widescreen). You can tweak parameters like the prompt, the depth (ddim), the "strength" of the original image, and the "scale" (not sure what it is but it does change things around). Then you can play with that and even iterate on more images. [0] https://upload.i.ng/file/goaltendersgrievousSumter/seed_643610_00002.png https://upload.i.ng/file/goaltendersgrievousSumter/seed_6436... [1] https://upload.i.ng/file/overstatementrearwardsplighting/dithered-image.png https://upload.i.ng/file/overstatementrearwardsplighting/dit... [2] https://twitter.com/xMorgawr/status/1564271156462440448 https://twitter.com/xMorgawr/status/1564271156462440448
cantaloupe 4y ago
This technique needs to be adopted everywhere immediately. With infinite gains over jpeg, how could we delay? The commentary gave me a good chuckle, but I was actually pretty impressed with some of the results. I wonder if using stock images is playing to the strength of the model, because they may have actually been part of training.
- stavros 4y ago
  I think that's actually the case, yeah. I think it mostly "remembered" these images, as it had been trained on them.
bitshiftfaced 4y ago
You can always reproduce the exact image by using the same prompt, seed value, and config info (assuming you've created the image with SD to start with).
- dr_dshiv 4y ago
  Is there a unique image identifier system that might facilitate the preservation of this metadata?
- WheelsAtLarge 4y ago
  You would think but there seems to be some kind of randomization added to the image creation. You can use the same prompt over and over again and get different images.
  - bitshiftfaced 4y ago
    When I use the same seed and same prompt, I get the same image.
    - astrange 4y ago
      Compression algorithms using floating point have always been flaky though. Need a fixed-point image model.
    - WheelsAtLarge 4y ago
      You are correct. My mistake.
    - nextaccountic 4y ago
      Same seed, same prompt and same hardware perhaps, if the GPU concurrency doesn't introduce nondeterminism. Otherwise you might have the same problem of this: https://github.com/EmbarkStudios/texture-synthesis#notes https://github.com/EmbarkStudios/texture-synthesis#notes > When using multiple threads for generation, the output image is not guaranteed to be deterministic with the same inputs. To have 100% determinism, you must use a thread count of one, which can by done via > CLI - texture-synthesis --threads 1 > API - SessionBuilder::max_thread_count(1)
voiper1 4y ago
I'm _pretty sure_ this is tongue-in-cheek: the final "uncompressed" image bears only a superficial resemblance to the original.
- WheelsAtLarge 4y ago
  You're right but I can see a future where an image can be analyzed by an AI and get the minimum amount of data to recreate it.
- kfarr 4y ago
  > There are some minor kinks that need to be worked out, such as the fact that each image takes around a day to generate on mobile, but this is more than acceptable in certain domains. Website visitors, for example, are well-accustomed to such loading times, and would barely notice any difference.
avocado2 4y ago
- dr_dshiv 4y ago
  Image to Prompt is discussed in this article. I’m scared to find out what adjectives it would use to prompt my bio picture.. https://replicate.com/methexis-inc/img2prompt https://replicate.com/methexis-inc/img2prompt
sklivvz1971 4y ago
You still have to transfer the generated image from stable diffusion to your browser... How do you compress that? :-D
- stavros 4y ago
  You don't, you generate it in the browser.
deleted 4y ago
[deleted]
esjeon 4y ago
This is NOT compression. Almost all information gets lost. This is simply about replacing images with captions + image generator.
- bawolff 4y ago
- Nition 4y ago
  Probably shouldn't have to say it but, that's the joke.
a_shovel 4y ago
Okay, I've been really negative about AI image generation, but this is a really cool use for it.
dr_dshiv 4y ago
“quality is better than the original, and it is astonishing for a compressor to achieve 100%+ quality.” Wow. Sounds like we’ve got a new standard image format on the the way. With a continuous “truthy” scale rating.
- dylan604 4y ago
  believe half of what you see and nothing of what you hear...
bawolff 4y ago
To nitpick a joke, jpeg cant have infinite size, since the format only supports at most 2^16 × 2^16 pixels.
- lifthrasiir 4y ago
  Well you can put anything into APPn segments, at most 65,536 bytes each, unlimited times.
obblekk 4y ago
1. This is obviously a (funny) joke (pretty great article stav!) 2. Any image generated with stable diffusion, could be distributed as prompt + seed leading to 99% compression 3. Image -> downsample –> super-resolution model -> render -- this must already exist as a compression model? The only limitation is that super-res models probably can't run on phones yet... but some day soon they'll be good enough. 4. All deep learning is fundamentally a compression system (reduce a lot of input data to a set of finite features and weights). But it might also be a very good lossy compression system because it can identify what's worth keeping high res and what's worth aggressively downsampling. 5. Interestingly, we seem to be compute limited more than bandwidth limited nowadays so it might take a while for an economically useful DL-compression product to emerge.
- thrdbndndn 4y ago
  > this must already exist as a compression model? Definitely exist in video compression, not so sure about still image (but most of still image compression advancements lately are just side effect of video compression).
- agumonkey 4y ago
  Reminds me of the old IBM joke: you can compress everything down to a bit, if you include the data in the decompresser.
- Nition 4y ago
  I wonder how close to an arbitrary image a truly optimal Stable Diffusion prompt could actually get (the ones on the blog being just a good guess). Like say you magically could try infinite prompt variations to find the closest match. Maybe some of the text would look like encoded gibberish. I imagine the "optimal" prompt would be very long, maybe - probably - larger than the original image.
- rasz 4y ago
  bonus points for Getty watermark in one of the "results"
ackbar03 4y ago
I'm not sure compression with deep learning makes sense though? Currently data storage and transportation costs are much cheaper than computation. Until that's reversed, I'm not sure this is really a viable use case? And just intuitively, since both costs are proportionate to the size of the raw data, I don't see that happening with deep learning models.
deleted 4y ago
[deleted]
thrdbndndn 4y ago
What's going on with that deformed bowl in "three bowls of food on a white table" one? Also, "a couple of women"? Is it part of the joke or the right one is really a woman?
- stavros 4y ago
  No, that's just what img2prompt thought.
userbinator 4y ago
Pretty much the old trick of moving the bulk of the complexity into the compressor/decompressor, including the content itself, and taking that into the AI realm.
yaddaor 4y ago
The JPEG-XR link probably was meant to point to https://en.m.wikipedia.org/wiki/JPEG_XL https://en.m.wikipedia.org/wiki/JPEG_XL instead.
- stavros 4y ago
  Oops, will fix, thanks!
seydor 4y ago
This is better than middle-out
mmahemoff 4y ago
Obvious application is situations extremely constrained by bandwidth. A space explorer's family could transmit lovingly promptcrafted pictures to a station on a distant planet. (Videos should similarly lossy-compress down to a plot synopsis and the key cast/crew. Throw in the script too for more accuracy but biggger file.)
- extheat 4y ago
  Pictures alone are pretty low bandwidth with good compression. While the concept sounds cool, I don't see any immediate application for it. For long distance communication (/interplanetary) it's a problem of latency, not bandwidth. However, transmitting moderate/large length high-res video clips from prompts could be interesting, probably magnitude more efficient than existing compression algorithms.
lnyan 4y ago
Use stable diffusion for image compression Store stable diffusion model in the π file system (https://github.com/philipl/pifs https://github.com/philipl/pifs) "Almost zero overhead in terms of data storage"
ilaksh 4y ago
I believe that video codecs will probably work along these lines at some point. Maybe have a base AI layer and then a more traditional codec layer on top to refine it. I think there are already AI-based codecs for reducing bandwidth of video calls with faces.
ilaksh 4y ago
We may have AI-powered "reality compression" in the next five years when mixed reality goggles/glasses become more comfortable and popular as mainstream alternatives for actually physically visiting places and people. Which will basically create realistic 3d streams of dynamic environments and people (so they can be viewed stereoscopically and from different perspectives). The enhanced AI-based compression should make it more practical to stream in the moving 3d data with fewer delays and latency and more realism than something like a traditional game engine. Unless no one wants to go to real places anymore because imaginary ones are nicer. But think about virtual reality with realistic faces and eye contact. Or a movie where you can view a scene from any perspective you want at any time. Even one of the protagonists. When you are trying to realistically transmit and recreate many entire dynamic 3d scenes, that is where AI can help a lot with scanning and compression/decompression.
- specialist 4y ago
  Ya. If our universe is a simulation, perhaps our perceived reality is generated using something like stable diffusion. Ages ago, for some raytracing stuff, a buddy used l-systems to generate forests, gardens, etc. Each plant was generated from a seed. A very compact reproducible, representation. https://en.wikipedia.org/wiki/L-system https://en.wikipedia.org/wiki/L-system https://en.wikipedia.org/wiki/The_Algorithmic_Beauty_of_Plants https://en.wikipedia.org/wiki/The_Algorithmic_Beauty_of_Plan...
  - ilaksh 4y ago
    The compactness of L-systems makes me think it might be a good idea to combine these more traditional types of procedural generation with AI-based generation to take advantage of the efficiency. Especially if we were trying to quickly load in something like a virtual world (rather than needing to replicate a real one). The L-systems rules are probably much more compact than the AI vectors. And there might even be some parallel versions. What we know about our brains (whether it's a simulated brain or not) tells us that effectively our perception is a simulation of our own. Because for example as you look around, your eyes only get high resolution data in the center, and that can only represent a fraction of the relevant immediate environment. And none of the senses or motor activations are quite real time, there is always some type of lag. The brain takes the data samples and uses them to fine tune a generative prediction system that constructs the dynamic and holistic 3d scene and your position in it using the computational power of your own network of neurons. To me, dreams provide a certain level of proof that we can do this type of simulation, and something similar is happening while we are awake, but with environmental data sampling and other cognitive processes feeding in.
fragmede 4y ago
Jokes aside, that the 250 terabytes of the LAION 5B dataset are remotely able to be, in any form, represented by the weights contained within the sub 5-gigabyte file that the represents StableDiffusion 1.4 model download, is nothing short of amazing.
ale42 4y ago
... games that come on multiple floppy disks! :-D
wernsey 4y ago
This technology might actually make The Stick of Jan Sloot [0], which I learned about on here recently, possible. The idea behind the stick was that you'd have a decoder next to your TV that would contain all the movies ever made encoded in a special format, and if you wanted to watch a particular one, you just needed a specific key that told the decoder how to decompress it. It didn't really work, as you'd imagine. [0] https://spronck.net/sloot.html https://spronck.net/sloot.html
- Nition 4y ago
  I think much like the stick, or a Minecraft seed, or other examples of procedural generation, you can get a near infinite amount of results via different seeds from a program with a relatively small total size. But you cannot ever get an exact result you want, unless it's already one of the outputs of the algorithm. Much like the key described in your article, it may be possible to get a very accurate result from an AI via a very long and specific prompt. But that long prompt is then where the "magic" compression goes away.
rapjr9 4y ago
I wonder if iterated function systems used for fractal image compression could be combined with AI to make a similar image compression method that actually works: https://ieeexplore.ieee.org/document/918789 https://ieeexplore.ieee.org/document/918789 https://link.springer.com/chapter/10.1007/978-0-387-69216-6_11 https://link.springer.com/chapter/10.1007/978-0-387-69216-6_...
RiversHaveWings 4y ago
Diffusion models can be used to lossily compress images for real, see for example Lossy Compression with Gaussian Diffusion (https://arxiv.org/abs/2206.08889 https://arxiv.org/abs/2206.08889). I would expect this to work with text-conditional diffusion models too and give you better compression when supplying a correct caption.
deleted 4y ago
[deleted]
m3kw9 4y ago
So reading the comments, this answers my question if this was a joke.

97 points by jstanley 4y ago

I've seen other Stable Diffusion examples where it starts with a low quality image and a prompt, and makes a higher-quality version. I'd be really interested to see this work extended with something that transmits (say) a 64x64 pixel copy of the original image and uses it as source material for Stable Diffusion to work on. Depending on the parameters it might be a legitimately viable image compression algorithm, at least for some types of images.

Morgawr 4y ago
Hilarious post aside, I've been using Stable Diffusion to generate "ideas" in large volume by creating a lot of images from a single (very generic) prompt with very low accuracy/depth levels, then when I find something that might be interesting, I fire up img2img and iterate over that with better accuracy, resolution, and depth. The results are pretty amazing.
- stavros 4y ago
  That's a great idea! I've been curious, isn't img2img basically just more SD passes? Ie you generate something with 5 passes, then img2img just runs the other 95 over that to "complete" the image. Basically, as far as I understand it (which is not much), txt2img generates a very rough blob from a prompt, then img2img does additional passes over that to refine it to something that has much higher quality. Is this incorrect?
  - Morgawr 4y ago
    I'm actually not sure how it works specifically because I don't know much about the technology behind it. All I know is that if I change the prompt (compared to the original prompt given by the txt2img script) and finetune the parameters I can get some *very* differing results that only keep a certain "style" or vibe. For example I generated [0] as my base image, I passed it into a dithering filter to make it look more "retro" and got [1] (was going for retro PC98 visual novel style backgrounds), and then [3] are all new results generated from it (I added stuff like "village" or "trees" etc in the new prompt and also made the resolution widescreen). You can tweak parameters like the prompt, the depth (ddim), the "strength" of the original image, and the "scale" (not sure what it is but it does change things around). Then you can play with that and even iterate on more images. [0] https://upload.i.ng/file/goaltendersgrievousSumter/seed_643610_00002.png https://upload.i.ng/file/goaltendersgrievousSumter/seed_6436... [1] https://upload.i.ng/file/overstatementrearwardsplighting/dithered-image.png https://upload.i.ng/file/overstatementrearwardsplighting/dit... [2] https://twitter.com/xMorgawr/status/1564271156462440448 https://twitter.com/xMorgawr/status/1564271156462440448

cantaloupe 4y ago

This technique needs to be adopted everywhere immediately. With infinite gains over jpeg, how could we delay? The commentary gave me a good chuckle, but I was actually pretty impressed with some of the results. I wonder if using stock images is playing to the strength of the model, because they may have actually been part of training.

stavros 4y ago
I think that's actually the case, yeah. I think it mostly "remembered" these images, as it had been trained on them.

bitshiftfaced 4y ago

You can always reproduce the exact image by using the same prompt, seed value, and config info (assuming you've created the image with SD to start with).

dr_dshiv 4y ago
Is there a unique image identifier system that might facilitate the preservation of this metadata?
WheelsAtLarge 4y ago
You would think but there seems to be some kind of randomization added to the image creation. You can use the same prompt over and over again and get different images.
- bitshiftfaced 4y ago
  When I use the same seed and same prompt, I get the same image.
  - astrange 4y ago
    Compression algorithms using floating point have always been flaky though. Need a fixed-point image model.
  - WheelsAtLarge 4y ago
    You are correct. My mistake.
  - nextaccountic 4y ago
    Same seed, same prompt and same hardware perhaps, if the GPU concurrency doesn't introduce nondeterminism. Otherwise you might have the same problem of this: https://github.com/EmbarkStudios/texture-synthesis#notes https://github.com/EmbarkStudios/texture-synthesis#notes > When using multiple threads for generation, the output image is not guaranteed to be deterministic with the same inputs. To have 100% determinism, you must use a thread count of one, which can by done via > CLI - texture-synthesis --threads 1 > API - SessionBuilder::max_thread_count(1)

voiper1 4y ago

I'm _pretty sure_ this is tongue-in-cheek: the final "uncompressed" image bears only a superficial resemblance to the original.

WheelsAtLarge 4y ago
You're right but I can see a future where an image can be analyzed by an AI and get the minimum amount of data to recreate it.
kfarr 4y ago
> There are some minor kinks that need to be worked out, such as the fact that each image takes around a day to generate on mobile, but this is more than acceptable in certain domains. Website visitors, for example, are well-accustomed to such loading times, and would barely notice any difference.

avocado2 4y ago

dr_dshiv 4y ago
Image to Prompt is discussed in this article. I’m scared to find out what adjectives it would use to prompt my bio picture.. https://replicate.com/methexis-inc/img2prompt https://replicate.com/methexis-inc/img2prompt

sklivvz1971 4y ago

You still have to transfer the generated image from stable diffusion to your browser... How do you compress that? :-D

stavros 4y ago
You don't, you generate it in the browser.

deleted 4y ago

[deleted]

esjeon 4y ago

This is NOT compression. Almost all information gets lost. This is simply about replacing images with captions + image generator.

bawolff 4y ago
Nition 4y ago
Probably shouldn't have to say it but, that's the joke.

a_shovel 4y ago

Okay, I've been really negative about AI image generation, but this is a really cool use for it.

dr_dshiv 4y ago

“quality is better than the original, and it is astonishing for a compressor to achieve 100%+ quality.” Wow. Sounds like we’ve got a new standard image format on the the way. With a continuous “truthy” scale rating.

dylan604 4y ago
believe half of what you see and nothing of what you hear...

bawolff 4y ago

To nitpick a joke, jpeg cant have infinite size, since the format only supports at most 2^16 × 2^16 pixels.

lifthrasiir 4y ago
Well you can put anything into APPn segments, at most 65,536 bytes each, unlimited times.

obblekk 4y ago

1. This is obviously a (funny) joke (pretty great article stav!) 2. Any image generated with stable diffusion, could be distributed as prompt + seed leading to 99% compression 3. Image -> downsample –> super-resolution model -> render -- this must already exist as a compression model? The only limitation is that super-res models probably can't run on phones yet... but some day soon they'll be good enough. 4. All deep learning is fundamentally a compression system (reduce a lot of input data to a set of finite features and weights). But it might also be a very good lossy compression system because it can identify what's worth keeping high res and what's worth aggressively downsampling. 5. Interestingly, we seem to be compute limited more than bandwidth limited nowadays so it might take a while for an economically useful DL-compression product to emerge.

thrdbndndn 4y ago
> this must already exist as a compression model? Definitely exist in video compression, not so sure about still image (but most of still image compression advancements lately are just side effect of video compression).
agumonkey 4y ago
Reminds me of the old IBM joke: you can compress everything down to a bit, if you include the data in the decompresser.
Nition 4y ago
I wonder how close to an arbitrary image a truly optimal Stable Diffusion prompt could actually get (the ones on the blog being just a good guess). Like say you magically could try infinite prompt variations to find the closest match. Maybe some of the text would look like encoded gibberish. I imagine the "optimal" prompt would be very long, maybe - probably - larger than the original image.
rasz 4y ago
bonus points for Getty watermark in one of the "results"

ackbar03 4y ago

I'm not sure compression with deep learning makes sense though? Currently data storage and transportation costs are much cheaper than computation. Until that's reversed, I'm not sure this is really a viable use case? And just intuitively, since both costs are proportionate to the size of the raw data, I don't see that happening with deep learning models.

deleted 4y ago

[deleted]

thrdbndndn 4y ago

What's going on with that deformed bowl in "three bowls of food on a white table" one? Also, "a couple of women"? Is it part of the joke or the right one is really a woman?

stavros 4y ago
No, that's just what img2prompt thought.

userbinator 4y ago

Pretty much the old trick of moving the bulk of the complexity into the compressor/decompressor, including the content itself, and taking that into the AI realm.

yaddaor 4y ago

The JPEG-XR link probably was meant to point to https://en.m.wikipedia.org/wiki/JPEG_XL https://en.m.wikipedia.org/wiki/JPEG_XL instead.

stavros 4y ago
Oops, will fix, thanks!

seydor 4y ago

This is better than middle-out

mmahemoff 4y ago

Obvious application is situations extremely constrained by bandwidth. A space explorer's family could transmit lovingly promptcrafted pictures to a station on a distant planet. (Videos should similarly lossy-compress down to a plot synopsis and the key cast/crew. Throw in the script too for more accuracy but biggger file.)

extheat 4y ago
Pictures alone are pretty low bandwidth with good compression. While the concept sounds cool, I don't see any immediate application for it. For long distance communication (/interplanetary) it's a problem of latency, not bandwidth. However, transmitting moderate/large length high-res video clips from prompts could be interesting, probably magnitude more efficient than existing compression algorithms.

lnyan 4y ago

Use stable diffusion for image compression Store stable diffusion model in the π file system (https://github.com/philipl/pifs https://github.com/philipl/pifs) "Almost zero overhead in terms of data storage"

ilaksh 4y ago

I believe that video codecs will probably work along these lines at some point. Maybe have a base AI layer and then a more traditional codec layer on top to refine it. I think there are already AI-based codecs for reducing bandwidth of video calls with faces.

ilaksh 4y ago

We may have AI-powered "reality compression" in the next five years when mixed reality goggles/glasses become more comfortable and popular as mainstream alternatives for actually physically visiting places and people. Which will basically create realistic 3d streams of dynamic environments and people (so they can be viewed stereoscopically and from different perspectives). The enhanced AI-based compression should make it more practical to stream in the moving 3d data with fewer delays and latency and more realism than something like a traditional game engine. Unless no one wants to go to real places anymore because imaginary ones are nicer. But think about virtual reality with realistic faces and eye contact. Or a movie where you can view a scene from any perspective you want at any time. Even one of the protagonists. When you are trying to realistically transmit and recreate many entire dynamic 3d scenes, that is where AI can help a lot with scanning and compression/decompression.

specialist 4y ago
Ya. If our universe is a simulation, perhaps our perceived reality is generated using something like stable diffusion. Ages ago, for some raytracing stuff, a buddy used l-systems to generate forests, gardens, etc. Each plant was generated from a seed. A very compact reproducible, representation. https://en.wikipedia.org/wiki/L-system https://en.wikipedia.org/wiki/L-system https://en.wikipedia.org/wiki/The_Algorithmic_Beauty_of_Plants https://en.wikipedia.org/wiki/The_Algorithmic_Beauty_of_Plan...
- ilaksh 4y ago
  The compactness of L-systems makes me think it might be a good idea to combine these more traditional types of procedural generation with AI-based generation to take advantage of the efficiency. Especially if we were trying to quickly load in something like a virtual world (rather than needing to replicate a real one). The L-systems rules are probably much more compact than the AI vectors. And there might even be some parallel versions. What we know about our brains (whether it's a simulated brain or not) tells us that effectively our perception is a simulation of our own. Because for example as you look around, your eyes only get high resolution data in the center, and that can only represent a fraction of the relevant immediate environment. And none of the senses or motor activations are quite real time, there is always some type of lag. The brain takes the data samples and uses them to fine tune a generative prediction system that constructs the dynamic and holistic 3d scene and your position in it using the computational power of your own network of neurons. To me, dreams provide a certain level of proof that we can do this type of simulation, and something similar is happening while we are awake, but with environmental data sampling and other cognitive processes feeding in.

fragmede 4y ago

Jokes aside, that the 250 terabytes of the LAION 5B dataset are remotely able to be, in any form, represented by the weights contained within the sub 5-gigabyte file that the represents StableDiffusion 1.4 model download, is nothing short of amazing.

ale42 4y ago

... games that come on multiple floppy disks! :-D

wernsey 4y ago

This technology might actually make The Stick of Jan Sloot [0], which I learned about on here recently, possible. The idea behind the stick was that you'd have a decoder next to your TV that would contain all the movies ever made encoded in a special format, and if you wanted to watch a particular one, you just needed a specific key that told the decoder how to decompress it. It didn't really work, as you'd imagine. [0] https://spronck.net/sloot.html https://spronck.net/sloot.html

Nition 4y ago
I think much like the stick, or a Minecraft seed, or other examples of procedural generation, you can get a near infinite amount of results via different seeds from a program with a relatively small total size. But you cannot ever get an exact result you want, unless it's already one of the outputs of the algorithm. Much like the key described in your article, it may be possible to get a very accurate result from an AI via a very long and specific prompt. But that long prompt is then where the "magic" compression goes away.

rapjr9 4y ago

I wonder if iterated function systems used for fractal image compression could be combined with AI to make a similar image compression method that actually works: https://ieeexplore.ieee.org/document/918789 https://ieeexplore.ieee.org/document/918789 https://link.springer.com/chapter/10.1007/978-0-387-69216-6_11 https://link.springer.com/chapter/10.1007/978-0-387-69216-6_...

RiversHaveWings 4y ago

Diffusion models can be used to lossily compress images for real, see for example Lossy Compression with Gaussian Diffusion (https://arxiv.org/abs/2206.08889 https://arxiv.org/abs/2206.08889). I would expect this to work with text-conditional diffusion models too and give you better compression when supplying a correct caption.

deleted 4y ago

[deleted]

m3kw9 4y ago

So reading the comments, this answers my question if this was a joke.