4 ms·

Small text is indeed one of the fail cases, and more research needs to be done here (see also my other comment [1]). We mention this issue in the supplementary

Small text is indeed one of the fail cases, and more research needs to be done here (see also my other comment [1]). We mention this issue in the supplementary materials [2], and you can check out an example here at [3] [1] https://news.ycombinator.com/item?id=23654161 https://news.ycombinator.com/item?id=23654161 [2] https://storage.googleapis.com/hific/data/appendixb/appendixb.pdf https://storage.googleapis.com/hific/data/appendixb/appendix... [3] https://storage.googleapis.com/hific/userstudy/visualize.html?perPage=1&page=5 https://storage.googleapis.com/hific/userstudy/visualize.htm...

mmastrac 6y ago
This might be a good opportunity to lead to research on a "good faith watermark" for GAN-compressed images that may include hallucinated details.
- gtoderici 6y ago
  One of the things we discussed to address this is to have the ability to: a) turn off detail hallucination completely given the same bitstream; and b) store the median/maximum absolute error across the image (b) should allow the user to determine whether the image is suitable for their use-case.
  - AbrahamParangi 6y ago
    You could use an OCR encoder and cat that to the feature vector. Great work by the way!
    - atorodius 6y ago
      Agreed! Alternatively, some semantic segmentation network could be used and a masked MSE loss. In this paper, we focussed on showing the crazy potential of GANs for compression - let's see what future work brings.
  - the8472 6y ago
    Is it possible to put the decoder into a feedback loop and search among multiple possible encodings that minimize the residual errors? Similar to trellis optimization in video codecs. http://akuvian.org/src/x264/trellis.txt http://akuvian.org/src/x264/trellis.txt
    - atorodius 6y ago
      It would be possible - but by minimizing residual errors you end up in a similar regime as when minimizing MSE again, likely making reconstructions blurry!

mmastrac 6y ago

This might be a good opportunity to lead to research on a "good faith watermark" for GAN-compressed images that may include hallucinated details.

gtoderici 6y ago
One of the things we discussed to address this is to have the ability to: a) turn off detail hallucination completely given the same bitstream; and b) store the median/maximum absolute error across the image (b) should allow the user to determine whether the image is suitable for their use-case.
- AbrahamParangi 6y ago
  You could use an OCR encoder and cat that to the feature vector. Great work by the way!
  - atorodius 6y ago
    Agreed! Alternatively, some semantic segmentation network could be used and a masked MSE loss. In this paper, we focussed on showing the crazy potential of GANs for compression - let's see what future work brings.
- the8472 6y ago
  Is it possible to put the decoder into a feedback loop and search among multiple possible encodings that minimize the residual errors? Similar to trellis optimization in video codecs. http://akuvian.org/src/x264/trellis.txt http://akuvian.org/src/x264/trellis.txt
  - atorodius 6y ago
    It would be possible - but by minimizing residual errors you end up in a similar regime as when minimizing MSE again, likely making reconstructions blurry!