5 ms·

My worry is when this gets used for something forensic. I imagine this will tend to reproduce things in the dataset, e.g. up-scaling blurry text may look like

My worry is when this gets used for something forensic. I imagine this will tend to reproduce things in the dataset, e.g. up-scaling blurry text may look like fonts that it has memorized more than the original. Or upscaling a feather will provide details like the feather of more common birds. Or upscaling blured out numbers will pick some numbers at random [1]. We need to make sure people don't rely on these details, e.g. in courts, HR reviews, when reddit sleuths try and investigate an incident, when someone looks for cheating partners etc. [1] https://www.theregister.com/2013/08/06/xerox_copier_flaw_means_dodgy_numbers_and_dangerous_designs/ https://www.theregister.com/2013/08/06/xerox_copier_flaw_mea...

xadhominemx 6y ago
I am not worried about it this at all. It’s not likely to randomly incriminate a real person.
- fsflover 6y ago
  Yes, it is: https://news.ycombinator.com/item?id=26504144 https://news.ycombinator.com/item?id=26504144
  - xadhominemx 6y ago
    Ok that’s pretty far removed from a prosecutor relying on that image though
- sennight 6y ago
  I remember that time a guy's life got turned upside down because his fingerprints "matched" those found on a bomb. Despite the fact that he had no motive, disposition, or access - they were determined to convict, because to do otherwise would be to admit that fingerprint analysis doesn't enjoy the scientific foundation that DNA evidence does. So yeah, real people have been harmed by bad matching algorithms.
- speeder 6y ago
  I am from Brazil. On the first day of trials of deep-learning based facial recognition here, a random person was arrested because the algorithm confused that person with another one. Even more stupid, is that the person with "outstanding warrant" was actually ALREADY in prison. So yes, AI managed to arrest the same person, twice, one time the real person, one time a random look-alike.
  - xadhominemx 6y ago
    Seems like a different situation - the facial recognition algo was wrong. Not the same as an AI resolving a face that resembles someone and then prosecuting that person on the basis of the image.
    - koluna 6y ago
      You realize that the idea is the same, right? AI made an incorrect determination and people ran with it.
      - xadhominemx 6y ago
        You do understand the difference between arrest and prosecution? People are arrested on the basis of a weak mistaken identity all the time, no AI needed
- vbezhenar 6y ago
  Imagine low-resolution face photo. Police found 3 men whose faces are similar to that photo. Now they're super-resolutioning that photo and suddenly first person looks very similar to it, while second and third persons do not. First person is in danger because of algorithm which was trained on some specific photos.
  - xadhominemx 6y ago
    One can imagine a lot of future scenarios. In fact that describes is an entire genre of story writing known as “science fiction”
    - sennight 6y ago
      It isn't "science fiction" though, we have a mountain of examples that demonstrate how probabilistic AI fails when the training dataset doesn't perfectly align with the eventual application. It isn't restricted to AI either, there have been numerous studies showing that cross-race identification to be garbage data. <Insert other race> all look the same: https://onlinelibrary.wiley.com/doi/abs/10.1002/acp.898 https://onlinelibrary.wiley.com/doi/abs/10.1002/acp.898 This bus is an ostrich: https://arxiv.org/abs/1312.6199 https://arxiv.org/abs/1312.6199 Racist autofocus: https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... Google thinks black people look like gorillas: https://www.wired.com/story/when-it-comes-to-gorillas-google-photos-remains-blind https://www.wired.com/story/when-it-comes-to-gorillas-google... And here is an argument to look forward to: should the training dataset be build to represent the general population, or the specific subgroup that the software would be most often encountering? Because that FBI UCR...
- michaelpb 6y ago
  Out of curiosity, do you think forensic science methods in general are likely to randomly incriminate people?
natch 6y ago
You’re worrying about the wrong thing. In formal settings, the problem will be taken care of. The bigger problem is informal settings. Propaganda, for one.
- BiteCode_dev 6y ago
  The way DNA testing is misused, I wouldn't be so sure about formal settings.
  - natch 6y ago
    Good point! You have me reconsidering what I said.
- pfortuny 6y ago
  Although you have already replied to the DNA comment, one must take care with bureaucracies always. They will drag on with some preconception and the citizen always loses.
- cooper12 6y ago
  > In formal settings, the problem will be taken care of Forensic evidence has been and still is systematically abused: > * a 2002 FBI re-examination of microscopic hair comparisons the agency’s scientists had performed in criminal cases, in which DNA testing revealed that 11 percent of hair samples found to match microscopically actually came from different individuals; > * a 2004 National Research Council report, commissioned by the FBI, on bullet-lead evidence, which found that there was insufficient research and data to support drawing a definitive connection between two bullets based on compositional similarity of the lead they contain; > * a 2005 report of an international committee established by the FBI to review the use of latent fingerprint evidence in the case of a terrorist bombing in Spain, in which the committee found that “confirmation bias”—the inclination to confirm a suspicion based on other grounds—contributed to a misidentification and improper detention; and > * studies reported in 2009 and 2010 on bitemark evidence, which found that current procedures for comparing bitemarks are unable to reliably exclude or include a suspect as a potential biter. > Beyond these kinds of shortfalls with respect to “reliable methods” in forensic feature-comparison disciplines, reviews have found that expert witnesses have often overstated the probative value of their evidence, going far beyond what the relevant science can justify. (https://web.archive.org/web/20170120002449/https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_forensic_science_report_final.pdf https://web.archive.org/web/20170120002449/https://www.white... page 16) Even more: * Tire and shoe prints: https://www.apmreports.org/story/2016/09/27/questionable-science-tire-track-shoe-print https://www.apmreports.org/story/2016/09/27/questionable-sci... * Lie detector tests: https://en.wikipedia.org/wiki/Polygraph#Effectiveness https://en.wikipedia.org/wiki/Polygraph#Effectiveness * Burn patterns: https://www.pbs.org/wgbh/frontline/article/forensic-tools-whats-reliable-and-whats-not-so-scientific/ https://www.pbs.org/wgbh/frontline/article/forensic-tools-wh...
BiteCode_dev 6y ago
Yeah, the "uncrop" joke from red dwarf doesn't seem like a joke anymore (https://www.dailymotion.com/video/x2qlmuy https://www.dailymotion.com/video/x2qlmuy). Except if it's used in security camera, it's going to be a disaster. Those things have super low resolution, and software will be cheaper than upgrading. And models have huge bias. Tons of fun.
andi999 6y ago
Gosling will probably go to jail for a lot of crimes...
michaelpb 6y ago
Yeah, so much of forensic "science" is notoriously flawed, that this seems like a likely addition the usual junk-science pantheon of "gunshot analysis, footprint analysis, hair comparison and bite mark comparison" - https://innocenceproject.org/forensic-science-problems-and-solutions/ https://innocenceproject.org/forensic-science-problems-and-s... Personally, I think one reason we still do this has a lot to do with detective shows being so ridiculously popular that people think that it's some sort of scientific process, when it's not. As a thought experiment, we'd probably have flat-earther-ism be the dominant belief if was like 15/20 broadcast TV shows are dedicated to glorifying flat-earther-ism. This means the most dangerous thing about "Super Resolution" is the public has already been "primed" with cop shows having the "enhance" feature.

xadhominemx 6y ago

I am not worried about it this at all. It’s not likely to randomly incriminate a real person.

fsflover 6y ago
Yes, it is: https://news.ycombinator.com/item?id=26504144 https://news.ycombinator.com/item?id=26504144
- xadhominemx 6y ago
  Ok that’s pretty far removed from a prosecutor relying on that image though
sennight 6y ago
I remember that time a guy's life got turned upside down because his fingerprints "matched" those found on a bomb. Despite the fact that he had no motive, disposition, or access - they were determined to convict, because to do otherwise would be to admit that fingerprint analysis doesn't enjoy the scientific foundation that DNA evidence does. So yeah, real people have been harmed by bad matching algorithms.
speeder 6y ago
I am from Brazil. On the first day of trials of deep-learning based facial recognition here, a random person was arrested because the algorithm confused that person with another one. Even more stupid, is that the person with "outstanding warrant" was actually ALREADY in prison. So yes, AI managed to arrest the same person, twice, one time the real person, one time a random look-alike.
- xadhominemx 6y ago
  Seems like a different situation - the facial recognition algo was wrong. Not the same as an AI resolving a face that resembles someone and then prosecuting that person on the basis of the image.
  - koluna 6y ago
    You realize that the idea is the same, right? AI made an incorrect determination and people ran with it.
    - xadhominemx 6y ago
      You do understand the difference between arrest and prosecution? People are arrested on the basis of a weak mistaken identity all the time, no AI needed
vbezhenar 6y ago
Imagine low-resolution face photo. Police found 3 men whose faces are similar to that photo. Now they're super-resolutioning that photo and suddenly first person looks very similar to it, while second and third persons do not. First person is in danger because of algorithm which was trained on some specific photos.
- xadhominemx 6y ago
  One can imagine a lot of future scenarios. In fact that describes is an entire genre of story writing known as “science fiction”
  - sennight 6y ago
    It isn't "science fiction" though, we have a mountain of examples that demonstrate how probabilistic AI fails when the training dataset doesn't perfectly align with the eventual application. It isn't restricted to AI either, there have been numerous studies showing that cross-race identification to be garbage data. <Insert other race> all look the same: https://onlinelibrary.wiley.com/doi/abs/10.1002/acp.898 https://onlinelibrary.wiley.com/doi/abs/10.1002/acp.898 This bus is an ostrich: https://arxiv.org/abs/1312.6199 https://arxiv.org/abs/1312.6199 Racist autofocus: https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... Google thinks black people look like gorillas: https://www.wired.com/story/when-it-comes-to-gorillas-google-photos-remains-blind https://www.wired.com/story/when-it-comes-to-gorillas-google... And here is an argument to look forward to: should the training dataset be build to represent the general population, or the specific subgroup that the software would be most often encountering? Because that FBI UCR...
michaelpb 6y ago
Out of curiosity, do you think forensic science methods in general are likely to randomly incriminate people?

natch 6y ago

You’re worrying about the wrong thing. In formal settings, the problem will be taken care of. The bigger problem is informal settings. Propaganda, for one.

BiteCode_dev 6y ago
The way DNA testing is misused, I wouldn't be so sure about formal settings.
- natch 6y ago
  Good point! You have me reconsidering what I said.
pfortuny 6y ago
Although you have already replied to the DNA comment, one must take care with bureaucracies always. They will drag on with some preconception and the citizen always loses.
cooper12 6y ago
> In formal settings, the problem will be taken care of Forensic evidence has been and still is systematically abused: > * a 2002 FBI re-examination of microscopic hair comparisons the agency’s scientists had performed in criminal cases, in which DNA testing revealed that 11 percent of hair samples found to match microscopically actually came from different individuals; > * a 2004 National Research Council report, commissioned by the FBI, on bullet-lead evidence, which found that there was insufficient research and data to support drawing a definitive connection between two bullets based on compositional similarity of the lead they contain; > * a 2005 report of an international committee established by the FBI to review the use of latent fingerprint evidence in the case of a terrorist bombing in Spain, in which the committee found that “confirmation bias”—the inclination to confirm a suspicion based on other grounds—contributed to a misidentification and improper detention; and > * studies reported in 2009 and 2010 on bitemark evidence, which found that current procedures for comparing bitemarks are unable to reliably exclude or include a suspect as a potential biter. > Beyond these kinds of shortfalls with respect to “reliable methods” in forensic feature-comparison disciplines, reviews have found that expert witnesses have often overstated the probative value of their evidence, going far beyond what the relevant science can justify. (https://web.archive.org/web/20170120002449/https://www.whitehouse.gov/sites/default/files/microsites/ostp/PCAST/pcast_forensic_science_report_final.pdf https://web.archive.org/web/20170120002449/https://www.white... page 16) Even more: * Tire and shoe prints: https://www.apmreports.org/story/2016/09/27/questionable-science-tire-track-shoe-print https://www.apmreports.org/story/2016/09/27/questionable-sci... * Lie detector tests: https://en.wikipedia.org/wiki/Polygraph#Effectiveness https://en.wikipedia.org/wiki/Polygraph#Effectiveness * Burn patterns: https://www.pbs.org/wgbh/frontline/article/forensic-tools-whats-reliable-and-whats-not-so-scientific/ https://www.pbs.org/wgbh/frontline/article/forensic-tools-wh...

BiteCode_dev 6y ago

Yeah, the "uncrop" joke from red dwarf doesn't seem like a joke anymore (https://www.dailymotion.com/video/x2qlmuy https://www.dailymotion.com/video/x2qlmuy). Except if it's used in security camera, it's going to be a disaster. Those things have super low resolution, and software will be cheaper than upgrading. And models have huge bias. Tons of fun.

andi999 6y ago

Gosling will probably go to jail for a lot of crimes...

michaelpb 6y ago

Yeah, so much of forensic "science" is notoriously flawed, that this seems like a likely addition the usual junk-science pantheon of "gunshot analysis, footprint analysis, hair comparison and bite mark comparison" - https://innocenceproject.org/forensic-science-problems-and-solutions/ https://innocenceproject.org/forensic-science-problems-and-s... Personally, I think one reason we still do this has a lot to do with detective shows being so ridiculously popular that people think that it's some sort of scientific process, when it's not. As a thought experiment, we'd probably have flat-earther-ism be the dominant belief if was like 15/20 broadcast TV shows are dedicated to glorifying flat-earther-ism. This means the most dangerous thing about "Super Resolution" is the public has already been "primed" with cop shows having the "enhance" feature.