3 ms·
From the title, I expected this to be basically pairing stable diffusion with an image captioning algorithm by 'compressing' the image to a simple human readabl
by _yb2s 4y ago
From the title, I expected this to be basically pairing stable diffusion with an image captioning algorithm by 'compressing' the image to a simple human readable description, and then regenerating a comparable image from the text. I imagine that would work and be possible, essentially an autoencoder with a 'latent space' of single short human readable sentences.
The way this actually works is pretty impressive. I wonder if it could be made lossless or less lossy in a similar manner to FLAC and/or video compression algorithms... basically first do the compression, and then add on a correction that converts the result partially or completely into the true image. Essentially, e.g. encoding real images of the most egregiously modified regions of the photo and putting them back over the result.
- vanderZwan 4y agoIt definitely can be made lossless, all you need to do is a compress/decompress roundtrip, and also save the resulting difference with the ground truth in a lossless image like PNG, QOI or lossless JXL. The final size would be the lossy compression + difference image. This is of course the least sophisticated approach, but who knows, it might compare pretty well with "plain" lossless formats.