4 ms·

It isn't real information though. This is effectively a Chinese whispers. The only way AI can create information is by doing something in the real world.

ofrzeta 2y ago
Maybe information needs to be understood relationally as in "information for a subject x". So if we have an image with a license plate that is unreadable and there's an algorithm that makes it readable to x, there is an information gain for x, although the information might have been in the image all along.
- bippihippi1 2y ago
  eliminating the noise makes the useful information clearer, but the information describing the noise is lost
- vunderba 2y ago
  Sure, but what if the upscaling algorithm misinterpreted a P as an F? Without manual supervision/tagging, there's an inherent risk that this information will have an adverse effect on future models.
- ablob 2y ago
  If the license plate was not readable, then the additional information is false data. You do not know more about the image than you knew before by definition. Replacing pixels with plausible data does not mean a gain of information. If anything, I'd argue that a loss of information occurs: The fact that x was hardly readable/unreadable before is lost, and any decision later on can not factor this in as "x" is now clearly defined and not fuzzy anymore. Would you accept a system that "enhances" images to find the license plate numbers of cars and fine their owners? If the plate number is unreadable the only acceptable option is to not use it. Inserting a plausible number and rolling with it even means that instead of a range of suspects, only one culprit can be supposed. Would you like to find yourself in court for crimes/offenses you never comitted because some black box decided it was a great idea to pretend it knew it was you? Edit: I think I misunderstood the premise. Nonetheless my comment shall stay.
  - immibis 2y ago
    For an example of this, see "Xerox scanners and photocopiers randomly alter numbers in scanned documents" https://news.ycombinator.com/item?id=6156238 https://news.ycombinator.com/item?id=6156238
jppittma 2y ago
It’s information taken from many other photos and embedded into a single one of interest no?
dragonwriter 2y ago
> The only way AI can create information is by doing something in the real world. Everything done is done in the real world, but the only way an AI can gather (not create) information about some particular thing is to interact with that thing. Without interacting with anything external to itself, all information it can gather is the information already gathered to create it.
- tomrod 2y ago
  Is there a formalization of this idea? Would love to read more.
- antihero 2y ago
  That's a better way of putting it, yes.
Lerc 2y ago
It is real information, it is just information that is not targeted at anything in particular. Random passwords are, well, random. That they are random and information is what makes them useful as passwords. As said by others, There is nothing terribly insightful about making something estimate the output of another by a non-perfect reproduction mechanism and noticing the output is different. Absent any particular guidance the difference will not be targeted. That is tautologically obvious. The difference is still information though, and with guidance you can target the difference to perform some goal. This is essentially what the gpt-o1 training was doing. Training on data generated by itself, but only when the generated data produced the correct answer.

ofrzeta 2y ago

Maybe information needs to be understood relationally as in "information for a subject x". So if we have an image with a license plate that is unreadable and there's an algorithm that makes it readable to x, there is an information gain for x, although the information might have been in the image all along.

bippihippi1 2y ago
eliminating the noise makes the useful information clearer, but the information describing the noise is lost
vunderba 2y ago
Sure, but what if the upscaling algorithm misinterpreted a P as an F? Without manual supervision/tagging, there's an inherent risk that this information will have an adverse effect on future models.
ablob 2y ago
If the license plate was not readable, then the additional information is false data. You do not know more about the image than you knew before by definition. Replacing pixels with plausible data does not mean a gain of information. If anything, I'd argue that a loss of information occurs: The fact that x was hardly readable/unreadable before is lost, and any decision later on can not factor this in as "x" is now clearly defined and not fuzzy anymore. Would you accept a system that "enhances" images to find the license plate numbers of cars and fine their owners? If the plate number is unreadable the only acceptable option is to not use it. Inserting a plausible number and rolling with it even means that instead of a range of suspects, only one culprit can be supposed. Would you like to find yourself in court for crimes/offenses you never comitted because some black box decided it was a great idea to pretend it knew it was you? Edit: I think I misunderstood the premise. Nonetheless my comment shall stay.
- immibis 2y ago
  For an example of this, see "Xerox scanners and photocopiers randomly alter numbers in scanned documents" https://news.ycombinator.com/item?id=6156238 https://news.ycombinator.com/item?id=6156238

jppittma 2y ago

It’s information taken from many other photos and embedded into a single one of interest no?

dragonwriter 2y ago

> The only way AI can create information is by doing something in the real world. Everything done is done in the real world, but the only way an AI can gather (not create) information about some particular thing is to interact with that thing. Without interacting with anything external to itself, all information it can gather is the information already gathered to create it.

tomrod 2y ago
Is there a formalization of this idea? Would love to read more.
antihero 2y ago
That's a better way of putting it, yes.

Lerc 2y ago

It is real information, it is just information that is not targeted at anything in particular. Random passwords are, well, random. That they are random and information is what makes them useful as passwords. As said by others, There is nothing terribly insightful about making something estimate the output of another by a non-perfect reproduction mechanism and noticing the output is different. Absent any particular guidance the difference will not be targeted. That is tautologically obvious. The difference is still information though, and with guidance you can target the difference to perform some goal. This is essentially what the gpt-o1 training was doing. Training on data generated by itself, but only when the generated data produced the correct answer.