3 ms·

It says the final output before pixel space is 64x64x4? How can that be enough information?

by ilaksh 4y ago

l33tman 4y ago
The autoencoder that maps between that and the 512x512x3 RGB space was trained together with the model, so it is specialized in upscaling the 64x64x4 info to pixel space for this particular purpose. It's "just" a factor of 48 (de)compression..
- astrange 4y ago
  And remember we only care about 8 bits of the output (actually less due to JPEG compression) but each latent value is a 32-bit float.
  - ilaksh 4y ago
    Oh so each dimension is measured in 64 bit numbers? So the total bytes is (64*8) * (64*8) * (4 *8 )?*
    - astrange 4y ago
      I think it’s considered an implementation detail so it’s just not mentioned. A lot of models still work if you round the values to FP16, which personally I find surprising.
Imnimo 4y ago
The way I think of it, we have a 512x512x3 target, so that's 48x the information. I don't think it's unreasonable to say that far less than 1/48th of the space of 512x512x3 outputs are natural images (meaning an image that might actually exist, rather than meaningless pixels). So if we think about that 64x64x4 tensor as telling us what in the smaller space of natural images we should draw, it seems like plenty of information. Especially since we have the information stored weights of the output network also.
- zestyping 4y ago
  The amount of information in a 64x64x4 array would depend on the precision of the numbers in it, right? For example, a 512x512 image in 24-bit colour could be completely encoded in a 64x64x4 array if each of the 64 x 64 x 4 = 16,384 values had 384 bits of precision. So, I wonder — what's the minimum number of bits of precision in the 64x64x4 array that would be sufficient for this to work?
  - a1369209993 4y ago
    According to a anecdotal test on one of the images found elsewhere in this thread, JPEG comression at 80% quality can chop a factor of 16 off the size of a 24bpp .bmp file. 384 / 16 = 24bits per [64x64x4-] array value. Integer range of 32bit float is 2^24. So "literally just a jpeg packed into 16K floats" is a option.

It says the final output before pixel space is 64x64x4? How can that be enough information?

by ilaksh 4y ago

It says the final output before pixel space is 64x64x4? How can that be enough information?

The autoencoder that maps between that and the 512x512x3 RGB space was trained together with the model, so it is specialized in upscaling the 64x64x4 info to pixel space for this particular purpose. It's "just" a factor of 48 (de)compression..

astrange 4y ago
And remember we only care about 8 bits of the output (actually less due to JPEG compression) but each latent value is a 32-bit float.
- ilaksh 4y ago
  Oh so each dimension is measured in 64 bit numbers? So the total bytes is (64*8) * (64*8) * (4 *8 )?*
  - astrange 4y ago
    I think it’s considered an implementation detail so it’s just not mentioned. A lot of models still work if you round the values to FP16, which personally I find surprising.

Imnimo 4y ago

The way I think of it, we have a 512x512x3 target, so that's 48x the information. I don't think it's unreasonable to say that far less than 1/48th of the space of 512x512x3 outputs are natural images (meaning an image that might actually exist, rather than meaningless pixels). So if we think about that 64x64x4 tensor as telling us what in the smaller space of natural images we should draw, it seems like plenty of information. Especially since we have the information stored weights of the output network also.

zestyping 4y ago
The amount of information in a 64x64x4 array would depend on the precision of the numbers in it, right? For example, a 512x512 image in 24-bit colour could be completely encoded in a 64x64x4 array if each of the 64 x 64 x 4 = 16,384 values had 384 bits of precision. So, I wonder — what's the minimum number of bits of precision in the 64x64x4 array that would be sufficient for this to work?
- a1369209993 4y ago
  According to a anecdotal test on one of the images found elsewhere in this thread, JPEG comression at 80% quality can chop a factor of 16 off the size of a 24bpp .bmp file. 384 / 16 = 24bits per [64x64x4-] array value. Integer range of 32bit float is 2^24. So "literally just a jpeg packed into 16K floats" is a option.