6 ms·

Something interesting about the San Francisco test image is that if you start to look into the details, its clear that some real changes have been made to the c

Something interesting about the San Francisco test image is that if you start to look into the details, its clear that some real changes have been made to the city. Rather than losing texture or grain or clarity, the information lost in this is information about the particular layout of a neighborhood of streets, which has now been replaced as if some one were drawing the scene from memory. A very different kind of loss that with out the original might be imperceptible because the information that was lost isn't replaced with random or systematic noise, but rather new, structured information..

thaumasiotes 4y ago
There was a scandal when it was discovered that Xerox machines were doing this; in that case, the example showed "photocopies" replacing numbers in documents with other numbers.
- smitec 4y ago
  There is a talk about that issue [1]. During my PhD this issue came up amongst those in the group looking into compressed sensing in MRI. Many reconstruction methods (AI being a modern variant) work well because a best guess is visually plausible. These kinds of methods fall apart when visually plausible and "true" are different in a meaningful way. The simplest examples here being the numbers in scanned documents, or in the MRI case, areas of the brain where "normal brain tissue" was on average more plausible than "tumor". [1]: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
  - nl 4y ago
    It's worth noting that these problems are things to be aware of, not the complete showstoppers some people seem to think that they are.
    - thaumasiotes 4y ago
      I'm having a hard time seeing where the random substitution of all numbers isn't supposed to be a complete showstopper.
      - nl 4y ago
        Well for example you train the VAE to reduce the compression on characters.
        kgwgk 4y ago
        The right amount of compression in a photocopy machine is zero. Compression that gives you a blurred image is a trade-off. But what does it mean to “be aware of” compression that may give you a crisp image of some made up document?
        rjmunro 4y ago
        If you are making an image of a cityscape to illustrate an article it probably doesn't matter what the city looks like. But if the article is about the architecture of the specific city, it probably does, so you need to 'be aware' that the image you are showing people isn't correct, and reduce the compression.
        kgwgk 4y ago
        This subthread was about changing numbers in scanned documents and vanishing tumors in medical images.
        nl 4y ago
        > The right amount of compression in a photocopy machine is zero. This isn't an obvious statement to me. If you've had the misfortune of scanning documents to PDF and getting the 100MB per page files automatically emailed to you then you might see the benefit in all that white space being compressed somehow. > But what does it mean to “be aware of” compression that may give you a crisp image of some made up document? This isn't something I said. A good compression system for documents will not change characters in any circumstances.
tlrobinson 4y ago
One thing that worries me about generative AI is the degradation of “truth” over time. AI will be the cheapest way to generated content, by far. It will sometimes get facts subtly wrong, and eventually that AI generated content will be used to train future models. Rinse and repeat.
- Xcelerate 4y ago
  So you’ve described humans.
  - _nalply 4y ago
    Currently computers can reliably do maths. Later AI will unreliably do maths. Exactly like humans.
    - pishpash 4y ago
      So it will get stupider... maybe the singularity isn't bad like too smart but bad like dealing with too many stupid people.
      - gpderetta 4y ago
        Both stupider and less deterministic, but also and smarter and more flexible. Like humans.
      - ciphol 4y ago
        The nice thing about math is that often it's much harder to find a proof than to verify that proof. So math AI is allowed to make lots of dumb mistakes, we just want it to make the occasional real finding too.
        MauranKilom 4y ago
        Unless we also ask AI to do the proof verification...
jhrmnn 4y ago
It's interesting that this is closer to how human memory operates—we're quite good in unconsciously fabricating false yet strong memories.
- laundermaf 4y ago
  True, but I'd like to continue using products that produce close-to-real images. Phones nowadays already process images at lot. The moment they start replacing pixels it'll all be fake. And… Some manufacturer apparently already did it on their ultra zoom phones when taking photos of the moon.
  - NavinF 4y ago
    Meh. Cameras have been "replacing pixels" for as long as I've been alive. Consider that a 4K camera only has 2k*4k pixels whereas a 4K screen has 2k*4k*3 subpixels. 2/3 of the image is just dreamed up by the ISP (image signal processor) when it debayers the raw image. I'm not aware of any consumer hardware that has open source ISP firmware or claims to optimize for accuracy over beauty.
    - montroser 4y ago
      Okay, but a camera doing this is unlikely to dream up plausible features that didn't actually exist in the scene.
      - NavinF 4y ago
        Of course it is! Try feeding static into a modern ISP. It will find patterns that don't exist.
aaaaaaaaaaab 4y ago
The good old JBIG2 debacle. “When used in lossy mode, JBIG2 compression can potentially alter text in a way that's not discernible as corruption. This is in contrast to some other algorithms, which simply degrade into a blur, making the compression artifacts obvious.[14] Since JBIG2 tries to match up similar-looking symbols, the numbers "6" and "8" may get replaced, for example. In 2013, various substitutions (including replacing "6" with "8") were reported to happen on many Xerox Workcentre photocopier and printer machines, where numbers printed on scanned (but not OCR-ed) documents could have potentially been altered. This has been demonstrated on construction blueprints and some tables of numbers; the potential impact of such substitution errors in documents such as medical prescriptions was briefly mentioned.” https://en.m.wikipedia.org/wiki/JBIG2 https://en.m.wikipedia.org/wiki/JBIG2
kybernetikos 4y ago
Yeah, if it were actually adopted as a way to do compression, it seems likely to lead to even worse problems than JBIG2 did https://news.ycombinator.com/item?id=6156238 https://news.ycombinator.com/item?id=6156238 Invisibly changing the content rather than the image quality seems like a really concerning failure mode for image compression! I wonder if it'd be possible to use SD as part of a lossless system - use SD as something that tells us the liklihood of various pixel values given the rest of the image and combine that liklihood with a huffman encoding. Either way, fantastic hack, but we really should avoid using anything lossy built on AI for image compression.
- pishpash 4y ago
  Give it "enough" bits and it won't be a problem. How many is enough is the question.
- eloisius 4y ago
  Imagine a world where bandwidth constraints meant transmitting a hidden compressed representation that gets expanded locally by smart TVs that have pretrained weights baked into the OS. Everyone sees a slightly different reconstitution of the same input video. Firmware updates that push new weights to your TV result in stochastic changes to a movie you've watched before.
  - kybernetikos 4y ago
    "The weather forecast was correct as broadcast, sir, it's just your smart TV thought it was more likely that the weather in your region would be warm on that day, so it adjusted the symbol and temperature accordingly"
  - jacobr1 4y ago
    You could still use some kind of adaptive huffman coding. Current compression schemes have some kind of dictionary embedded in the file to map between the common strings and the compressed representation. Google tried proposing SDCH a few years using a common dictionary for wep pages. There isn't any reason why we can't be a bit more deterministic and share a much larger latent representation of "human visual comprehension" or whatever to do the same. It doesn't need to be stochastic once generated.
ZiiS 4y ago
It opens up an interesting question that is it suggesting "improvements" that could be done in the real world.
- RosanaAnaDana 4y ago
  Are you suggesting a lossy but 'correct' version? IE, the algorithm ignores and loses the 'irrelevant' information, but holds the important stuff?
phkahler 4y ago
This needs to be compared with automated tests. A lack of visual artifacts doesnt mean an accurate representation of the image in this case.
freediver 4y ago
Arguably this is still fine with the definition of lossy compression. The compressed image still roughly shows the idea of the original image.
taberiand 4y ago
I would've thought anyone relying on lossy-compressed images of any sort already needs to be aware of the potential effects, or otherwise isn't really concerned by the effect on the image (and I'd guess that the vast majority of use cases actually don't care if parts of the image are essentially "imaginary")

thaumasiotes 4y ago

There was a scandal when it was discovered that Xerox machines were doing this; in that case, the example showed "photocopies" replacing numbers in documents with other numbers.

smitec 4y ago
There is a talk about that issue [1]. During my PhD this issue came up amongst those in the group looking into compressed sensing in MRI. Many reconstruction methods (AI being a modern variant) work well because a best guess is visually plausible. These kinds of methods fall apart when visually plausible and "true" are different in a meaningful way. The simplest examples here being the numbers in scanned documents, or in the MRI case, areas of the brain where "normal brain tissue" was on average more plausible than "tumor". [1]: http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...
- nl 4y ago
  It's worth noting that these problems are things to be aware of, not the complete showstoppers some people seem to think that they are.
  - thaumasiotes 4y ago
    I'm having a hard time seeing where the random substitution of all numbers isn't supposed to be a complete showstopper.
    - nl 4y ago
      Well for example you train the VAE to reduce the compression on characters.
      - kgwgk 4y ago
        The right amount of compression in a photocopy machine is zero. Compression that gives you a blurred image is a trade-off. But what does it mean to “be aware of” compression that may give you a crisp image of some made up document?
        rjmunro 4y ago
        If you are making an image of a cityscape to illustrate an article it probably doesn't matter what the city looks like. But if the article is about the architecture of the specific city, it probably does, so you need to 'be aware' that the image you are showing people isn't correct, and reduce the compression.
        kgwgk 4y ago
        This subthread was about changing numbers in scanned documents and vanishing tumors in medical images.
        nl 4y ago
        > The right amount of compression in a photocopy machine is zero. This isn't an obvious statement to me. If you've had the misfortune of scanning documents to PDF and getting the 100MB per page files automatically emailed to you then you might see the benefit in all that white space being compressed somehow. > But what does it mean to “be aware of” compression that may give you a crisp image of some made up document? This isn't something I said. A good compression system for documents will not change characters in any circumstances.

tlrobinson 4y ago

One thing that worries me about generative AI is the degradation of “truth” over time. AI will be the cheapest way to generated content, by far. It will sometimes get facts subtly wrong, and eventually that AI generated content will be used to train future models. Rinse and repeat.

Xcelerate 4y ago
So you’ve described humans.
- _nalply 4y ago
  Currently computers can reliably do maths. Later AI will unreliably do maths. Exactly like humans.
  - pishpash 4y ago
    So it will get stupider... maybe the singularity isn't bad like too smart but bad like dealing with too many stupid people.
    - gpderetta 4y ago
      Both stupider and less deterministic, but also and smarter and more flexible. Like humans.
    - ciphol 4y ago
      The nice thing about math is that often it's much harder to find a proof than to verify that proof. So math AI is allowed to make lots of dumb mistakes, we just want it to make the occasional real finding too.
      - MauranKilom 4y ago
        Unless we also ask AI to do the proof verification...

jhrmnn 4y ago

It's interesting that this is closer to how human memory operates—we're quite good in unconsciously fabricating false yet strong memories.

laundermaf 4y ago
True, but I'd like to continue using products that produce close-to-real images. Phones nowadays already process images at lot. The moment they start replacing pixels it'll all be fake. And… Some manufacturer apparently already did it on their ultra zoom phones when taking photos of the moon.
- NavinF 4y ago
  Meh. Cameras have been "replacing pixels" for as long as I've been alive. Consider that a 4K camera only has 2k*4k pixels whereas a 4K screen has 2k*4k*3 subpixels. 2/3 of the image is just dreamed up by the ISP (image signal processor) when it debayers the raw image. I'm not aware of any consumer hardware that has open source ISP firmware or claims to optimize for accuracy over beauty.
  - montroser 4y ago
    Okay, but a camera doing this is unlikely to dream up plausible features that didn't actually exist in the scene.
    - NavinF 4y ago
      Of course it is! Try feeding static into a modern ISP. It will find patterns that don't exist.

aaaaaaaaaaab 4y ago

The good old JBIG2 debacle. “When used in lossy mode, JBIG2 compression can potentially alter text in a way that's not discernible as corruption. This is in contrast to some other algorithms, which simply degrade into a blur, making the compression artifacts obvious.[14] Since JBIG2 tries to match up similar-looking symbols, the numbers "6" and "8" may get replaced, for example. In 2013, various substitutions (including replacing "6" with "8") were reported to happen on many Xerox Workcentre photocopier and printer machines, where numbers printed on scanned (but not OCR-ed) documents could have potentially been altered. This has been demonstrated on construction blueprints and some tables of numbers; the potential impact of such substitution errors in documents such as medical prescriptions was briefly mentioned.” https://en.m.wikipedia.org/wiki/JBIG2 https://en.m.wikipedia.org/wiki/JBIG2

kybernetikos 4y ago

Yeah, if it were actually adopted as a way to do compression, it seems likely to lead to even worse problems than JBIG2 did https://news.ycombinator.com/item?id=6156238 https://news.ycombinator.com/item?id=6156238 Invisibly changing the content rather than the image quality seems like a really concerning failure mode for image compression! I wonder if it'd be possible to use SD as part of a lossless system - use SD as something that tells us the liklihood of various pixel values given the rest of the image and combine that liklihood with a huffman encoding. Either way, fantastic hack, but we really should avoid using anything lossy built on AI for image compression.

pishpash 4y ago
Give it "enough" bits and it won't be a problem. How many is enough is the question.
eloisius 4y ago
Imagine a world where bandwidth constraints meant transmitting a hidden compressed representation that gets expanded locally by smart TVs that have pretrained weights baked into the OS. Everyone sees a slightly different reconstitution of the same input video. Firmware updates that push new weights to your TV result in stochastic changes to a movie you've watched before.
- kybernetikos 4y ago
  "The weather forecast was correct as broadcast, sir, it's just your smart TV thought it was more likely that the weather in your region would be warm on that day, so it adjusted the symbol and temperature accordingly"
- jacobr1 4y ago
  You could still use some kind of adaptive huffman coding. Current compression schemes have some kind of dictionary embedded in the file to map between the common strings and the compressed representation. Google tried proposing SDCH a few years using a common dictionary for wep pages. There isn't any reason why we can't be a bit more deterministic and share a much larger latent representation of "human visual comprehension" or whatever to do the same. It doesn't need to be stochastic once generated.

ZiiS 4y ago

It opens up an interesting question that is it suggesting "improvements" that could be done in the real world.

RosanaAnaDana 4y ago
Are you suggesting a lossy but 'correct' version? IE, the algorithm ignores and loses the 'irrelevant' information, but holds the important stuff?

phkahler 4y ago

This needs to be compared with automated tests. A lack of visual artifacts doesnt mean an accurate representation of the image in this case.

freediver 4y ago

Arguably this is still fine with the definition of lossy compression. The compressed image still roughly shows the idea of the original image.

taberiand 4y ago

I would've thought anyone relying on lossy-compressed images of any sort already needs to be aware of the potential effects, or otherwise isn't really concerned by the effect on the image (and I'd guess that the vast majority of use cases actually don't care if parts of the image are essentially "imaginary")