4 ms·

This dataset contains over 5.8 billion images, so 3226 of them being child pornography is a contamination rate of 0.55ppm. It's likely you will find this materi

by 127361 3y ago

This dataset contains over 5.8 billion images, so 3226 of them being child pornography is a contamination rate of 0.55ppm. It's likely you will find this material anywhere there are large collections of images.

deleted 3y ago
[deleted]
sneak 3y ago
It’s 340TB, for reference. AIUI it was scraped from the web.
deleted 3y ago
[deleted]
ToucanLoucan 3y ago
That seems less like an argument against removing/preventing the inclusion of CSAM and more like an admitting on behalf of those building these image sets that they're incredibly irresponsibly sourcing content for the aforementioned image sets. I'm not saying this is easy to do, in fact I'd wager it's incredibly hard: but it seems increasingly obvious that there was no curation, no standards, no approvals process, nothing but a ramshackle mad dash to include as much visual content as possible to train these models on, with zero consideration of whether the people who created it were alright with their content being used that way, if the content was legal to use either by licensing or by being incredibly illegal just by it's own existence. We get more and more stories of the ethical lapses of the massive companies/organizations behind this tech and the resounding chorus from my fellow engineers is seemingly just "well just remove it and let's keep going," or shrugging shoulders "that'll happen, we're trying to do research here." And fair enough but like, when you're doing research, you don't just run out and grab every chemical you can get your hands on and pour them all in a bucket and see what happens. How is this stuff not being found? What good are these datasets if they include this type of material? How can you trust the results you get from your training anymore once you know what went in?
- 127361 3y ago
  Well I would consider a 0.55ppm contamination rate very low indeed? Mistakes happen in the creation of such datasets, it's human nature? Just as we can't police a society to the extent it's completely free of crime, otherwise we will not function as a society and/or we will find ourselves living in a totalitarian state?
  - jprete 3y ago
    Including CSAM in your image set isn't excusable by "mistakes happen". Also, the totalitarian analogy is total nonsense.
    - whelp_24 3y ago
      I think this kinda shows that you don't really understand the problem. If you had a human sitdown and review each image, there still would be images that get through. Even if you told him to err heavily on the false positive side.
    - _jal 3y ago
      Just to be clear, you are saying that no collection of images can ever contain any CSAM, ever [1]. If some activity might run the risk of including it, that activity must be stopped. Is that correct? If so, why are you not calling for the internet to be shut down? The collection under discussion is just a subset of the internet. You'll also want to either ban or enforce review of all camera output, including closed-circuit cameras. Someone is going to have to watch all those watchers pretty closely... As for totalitarianism, well, how are you going to make any of this happen? As we just saw you'll need client-side visibility into everyone's visual data stores. Not to mention old photo albums world-wide contain images that would set off ImageNet. There are famous paintings and statues that would, too. [1] Do you exempt NCMEC? If so, how do you justify that?
    - failbuffer 3y ago
      You abstain from coffee, chocolate, and fish too I trust? Otherwise you're undoubtedly supporting trace amounts of slavery and child labor. I bet the device you're reading this with was also made with such (and if it doesn't then it's still supporting a genocidal regime). Get real. Scraped data is going to have problems. The expectation should be that reasonable measures are taken to filter proactively and that reactive measures are taken to remove illicit content found after that.
misnome 3y ago
Somewhat reasonable position - except images were found that matched known and catalogued CSAM images. One would have thought that checking against that would be bare minimum for cleaning a large dataset?
- dublinben 3y ago
  Access to PhotoDNA and other 'perceptual hashing' APIs is not open or freely granted. It's not unreasonable that a team compiling a database would be unable to perform this level of cleaning.
  - indrora 3y ago
    PhotoDNA is explicitly free to LE, Nonprofits, and UGC-backed companies. LAION would have easily been able to get access to it for free. In fact, at that scale, they would likely be able to get assistance. EDIT: Stanford themselves acknowledges in the original paper that bulk PhotoDNA access was given.
  - misnome 3y ago
    Did they even try?
anigbrowl 3y ago
Frequentist analysis is exactly the wrong approach here, for the same reason that one can't defend a crime by pointing to a much greater number of legal acts.
- 127361 3y ago
  No, we accept road traffic accidents as fact and do not shut down entire road intersections due to deaths that occur. The same thing applies to child pornography, and 0.5ppm is certainly a tolerable amount in an extremely large dataset. There are something like ~42000 traffic fatalities in the United States every year, we do not adopt draconian measures in dealing with it, even though loss of life and severe life-altering injuries occur. And such traffic accidents are absolutely devastating for the victims and their families. Unless the dataset is particularly severely contaminated, the known illegal images should be removed and the rest of it should be allowed to stay up.
  - 7jjjjjjj 3y ago
    >No, we accept road traffic accidents as fact Have you considered that maybe you shouldn't?
    - arjun_krishna1 3y ago
      This is how nuclear was killed.
    - 127361 3y ago
      And deaths resulting from cancer and a whole list of other ailments. The act of living itself carries risk.
sebastiangula 3y ago
0.55ppm is low but you can still be prosecuted if that contamination is found on your drive.
- 127361 3y ago
  The laws regarding child pornography, which were likely drafted by moral entrepreneurs[1] who created a moral panic out of it, are completely insane. Such persecution belongs in the 17th Century, not the modern age. 1. https://en.wikipedia.org/wiki/Moral_entrepreneur https://en.wikipedia.org/wiki/Moral_entrepreneur

by 127361 3y ago

deleted 3y ago

[deleted]

sneak 3y ago

It’s 340TB, for reference. AIUI it was scraped from the web.

deleted 3y ago

[deleted]

ToucanLoucan 3y ago

That seems less like an argument against removing/preventing the inclusion of CSAM and more like an admitting on behalf of those building these image sets that they're incredibly irresponsibly sourcing content for the aforementioned image sets. I'm not saying this is easy to do, in fact I'd wager it's incredibly hard: but it seems increasingly obvious that there was no curation, no standards, no approvals process, nothing but a ramshackle mad dash to include as much visual content as possible to train these models on, with zero consideration of whether the people who created it were alright with their content being used that way, if the content was legal to use either by licensing or by being incredibly illegal just by it's own existence. We get more and more stories of the ethical lapses of the massive companies/organizations behind this tech and the resounding chorus from my fellow engineers is seemingly just "well just remove it and let's keep going," or shrugging shoulders "that'll happen, we're trying to do research here." And fair enough but like, when you're doing research, you don't just run out and grab every chemical you can get your hands on and pour them all in a bucket and see what happens. How is this stuff not being found? What good are these datasets if they include this type of material? How can you trust the results you get from your training anymore once you know what went in?

127361 3y ago
Well I would consider a 0.55ppm contamination rate very low indeed? Mistakes happen in the creation of such datasets, it's human nature? Just as we can't police a society to the extent it's completely free of crime, otherwise we will not function as a society and/or we will find ourselves living in a totalitarian state?
- jprete 3y ago
  Including CSAM in your image set isn't excusable by "mistakes happen". Also, the totalitarian analogy is total nonsense.
  - whelp_24 3y ago
    I think this kinda shows that you don't really understand the problem. If you had a human sitdown and review each image, there still would be images that get through. Even if you told him to err heavily on the false positive side.
  - _jal 3y ago
    Just to be clear, you are saying that no collection of images can ever contain any CSAM, ever [1]. If some activity might run the risk of including it, that activity must be stopped. Is that correct? If so, why are you not calling for the internet to be shut down? The collection under discussion is just a subset of the internet. You'll also want to either ban or enforce review of all camera output, including closed-circuit cameras. Someone is going to have to watch all those watchers pretty closely... As for totalitarianism, well, how are you going to make any of this happen? As we just saw you'll need client-side visibility into everyone's visual data stores. Not to mention old photo albums world-wide contain images that would set off ImageNet. There are famous paintings and statues that would, too. [1] Do you exempt NCMEC? If so, how do you justify that?
  - failbuffer 3y ago
    You abstain from coffee, chocolate, and fish too I trust? Otherwise you're undoubtedly supporting trace amounts of slavery and child labor. I bet the device you're reading this with was also made with such (and if it doesn't then it's still supporting a genocidal regime). Get real. Scraped data is going to have problems. The expectation should be that reasonable measures are taken to filter proactively and that reactive measures are taken to remove illicit content found after that.

misnome 3y ago

Somewhat reasonable position - except images were found that matched known and catalogued CSAM images. One would have thought that checking against that would be bare minimum for cleaning a large dataset?

dublinben 3y ago
Access to PhotoDNA and other 'perceptual hashing' APIs is not open or freely granted. It's not unreasonable that a team compiling a database would be unable to perform this level of cleaning.
- indrora 3y ago
  PhotoDNA is explicitly free to LE, Nonprofits, and UGC-backed companies. LAION would have easily been able to get access to it for free. In fact, at that scale, they would likely be able to get assistance. EDIT: Stanford themselves acknowledges in the original paper that bulk PhotoDNA access was given.
- misnome 3y ago
  Did they even try?

anigbrowl 3y ago

Frequentist analysis is exactly the wrong approach here, for the same reason that one can't defend a crime by pointing to a much greater number of legal acts.

127361 3y ago
No, we accept road traffic accidents as fact and do not shut down entire road intersections due to deaths that occur. The same thing applies to child pornography, and 0.5ppm is certainly a tolerable amount in an extremely large dataset. There are something like ~42000 traffic fatalities in the United States every year, we do not adopt draconian measures in dealing with it, even though loss of life and severe life-altering injuries occur. And such traffic accidents are absolutely devastating for the victims and their families. Unless the dataset is particularly severely contaminated, the known illegal images should be removed and the rest of it should be allowed to stay up.
- 7jjjjjjj 3y ago
  >No, we accept road traffic accidents as fact Have you considered that maybe you shouldn't?
  - arjun_krishna1 3y ago
    This is how nuclear was killed.
  - 127361 3y ago
    And deaths resulting from cancer and a whole list of other ailments. The act of living itself carries risk.

sebastiangula 3y ago

0.55ppm is low but you can still be prosecuted if that contamination is found on your drive.

127361 3y ago
The laws regarding child pornography, which were likely drafted by moral entrepreneurs[1] who created a moral panic out of it, are completely insane. Such persecution belongs in the 17th Century, not the modern age. 1. https://en.wikipedia.org/wiki/Moral_entrepreneur https://en.wikipedia.org/wiki/Moral_entrepreneur