4 ms·

Exploitive, illegal photos of children found in the data that trains some AI

philk10 3y ago
dupe? https://news.ycombinator.com/item?id=38708099 https://news.ycombinator.com/item?id=38708099
GartzenDeHaes 3y ago
"Contractors in Kenya say they were traumatized by effort to screen out descriptions of violence and sexual abuse during run-up to OpenAI’s hit chatbot" https://www.wsj.com/articles/chatgpt-openai-content-abusive-sexually-explicit-harassment-kenya-workers-on-human-workers-cf191483 https://www.wsj.com/articles/chatgpt-openai-content-abusive-...
deleted 3y ago
[deleted]
ta1243 3y ago
While an AI accessing real abusive images is increasing demand for those images and thus arguably has real victim. But it does make me think, if someone were to ask an AI that is trained on normal pictures of children and normal pictures of consensual adult pornography and ask it to create a fake child [0] in a pornographic situation, where is the victim? I know some jurisdictions have laws to the point that a drawing of two stick characters having sex with the words "Lisa" and "Bart" on them would be illegal, so presumably an AI generated image like above would also be illegal, but I don't believe that's a uniform law across the world. [0] https://this-person-does-not-exist.com/en https://this-person-does-not-exist.com/en
- stvltvs 3y ago
  Indirectly, if AI-generated CSAM materials flood the internet, it would provide cover for real CSAM which would get lost in the crowd making it more difficult for law enforcement to identify and prosecute offenders.
tomalbrc 3y ago
https://archive.ph/uNQCv https://archive.ph/uNQCv
robterrell 3y ago
Gift link for anyone who can't view the article: https://wapo.st/41sWoLj https://wapo.st/41sWoLj
blondie9x 3y ago
We need to dissect training data more. Too much push to enhance models without ensuring the input data is free from issues.
ensignavenger 3y ago
"The researchers scanned for the abusive images by looking for their “hashes” — corresponding bits of code that identify them" I can't find the actual study, but I am wondering if anyone actually verified the photos are CSAM, and not a collision or even just misidentified images? There is substantial evidence that the technique used is highly likely to produce false positives when ran on a large dataset like this.
- ensignavenger 3y ago
  Found the study- https://stacks.stanford.edu/file/druid:kh752sm9123/ml_training_data_csam_report-2023-12-20.pdf https://stacks.stanford.edu/file/druid:kh752sm9123/ml_traini... The study says 825 were confirmed using one method, and 183 with another- not clear to me if those are unique or duplicates of each other. Also, "confirmed" only means "is or is likely" CSAM, and they don't provide any numbers as to how many were classified as "likely". It also isn't clear how they define "likely". EDIT: Previously I said the article was overstating the number of CSAM images found, but I since re-read it and the article may be citing the correct total from the two methods)
- Buttons840 3y ago
  I thought quality hashing algorithms have no known collisions? Like, it's theoretically possible, it just has never happened, anywhere, not even once. Why would a collision happen here?
  - ensignavenger 3y ago
    Cryptographic hashes aren't supposed to have collisions... the types of hashing that PhotoDNA and other similar systems use are not cryptographic. In fact, some say they aren't even real hashes. The reason for this is because they want images that have been altered to still match, whereas with a cryptographic has, even a minor change will change the hash. So it is more a a "similarity" check than a proper hash.
  - yencabulator 3y ago
    You're confusing hashing as in SHA-256 with perceptual hashing, which is more like a blurry thumbnail of an image converted into numbers. Any minor alteration to an image will change it's cryptographic hash completely, making those pretty poor tools for finding images that might want to be hidden. Perceptual hashing should match e.g. resized versions of the image, but with them two "blurry thumbnail hashes" can match accidentally.
XorNot 3y ago
This is your regular reminder that the LAION dataset does not itself contain any image data at all: it is an index of image URLs on the the public internet which has been classified as having decent-quality tagging (image alt-text) and a few other bits of bulk, automated analysis done. The accessibility of CSAM content links within the dataset means those images are already on the internet being made publicly accessible by another website, which was found during the LAION image crawl. (interestingly this does rather imply that if LAION was able to compare images to the relevant CSAM hashes during the crawl then they'd simply be able to exclude them automatically - although I imagine that database is always growing)
sebastiangula 3y ago
If you are attempting to index the whole internet then you are doomed to eventually download something illegal.

dupe? https://news.ycombinator.com/item?id=38708099 https://news.ycombinator.com/item?id=38708099

"Contractors in Kenya say they were traumatized by effort to screen out descriptions of violence and sexual abuse during run-up to OpenAI’s hit chatbot" https://www.wsj.com/articles/chatgpt-openai-content-abusive-sexually-explicit-harassment-kenya-workers-on-human-workers-cf191483 https://www.wsj.com/articles/chatgpt-openai-content-abusive-...

deleted 3y ago

[deleted]

ta1243 3y ago

While an AI accessing real abusive images is increasing demand for those images and thus arguably has real victim. But it does make me think, if someone were to ask an AI that is trained on normal pictures of children and normal pictures of consensual adult pornography and ask it to create a fake child [0] in a pornographic situation, where is the victim? I know some jurisdictions have laws to the point that a drawing of two stick characters having sex with the words "Lisa" and "Bart" on them would be illegal, so presumably an AI generated image like above would also be illegal, but I don't believe that's a uniform law across the world. [0] https://this-person-does-not-exist.com/en https://this-person-does-not-exist.com/en

stvltvs 3y ago
Indirectly, if AI-generated CSAM materials flood the internet, it would provide cover for real CSAM which would get lost in the crowd making it more difficult for law enforcement to identify and prosecute offenders.

tomalbrc 3y ago

https://archive.ph/uNQCv https://archive.ph/uNQCv

robterrell 3y ago

Gift link for anyone who can't view the article: https://wapo.st/41sWoLj https://wapo.st/41sWoLj

blondie9x 3y ago

We need to dissect training data more. Too much push to enhance models without ensuring the input data is free from issues.

ensignavenger 3y ago

"The researchers scanned for the abusive images by looking for their “hashes” — corresponding bits of code that identify them" I can't find the actual study, but I am wondering if anyone actually verified the photos are CSAM, and not a collision or even just misidentified images? There is substantial evidence that the technique used is highly likely to produce false positives when ran on a large dataset like this.

ensignavenger 3y ago
Found the study- https://stacks.stanford.edu/file/druid:kh752sm9123/ml_training_data_csam_report-2023-12-20.pdf https://stacks.stanford.edu/file/druid:kh752sm9123/ml_traini... The study says 825 were confirmed using one method, and 183 with another- not clear to me if those are unique or duplicates of each other. Also, "confirmed" only means "is or is likely" CSAM, and they don't provide any numbers as to how many were classified as "likely". It also isn't clear how they define "likely". EDIT: Previously I said the article was overstating the number of CSAM images found, but I since re-read it and the article may be citing the correct total from the two methods)
Buttons840 3y ago
I thought quality hashing algorithms have no known collisions? Like, it's theoretically possible, it just has never happened, anywhere, not even once. Why would a collision happen here?
- ensignavenger 3y ago
  Cryptographic hashes aren't supposed to have collisions... the types of hashing that PhotoDNA and other similar systems use are not cryptographic. In fact, some say they aren't even real hashes. The reason for this is because they want images that have been altered to still match, whereas with a cryptographic has, even a minor change will change the hash. So it is more a a "similarity" check than a proper hash.
- yencabulator 3y ago
  You're confusing hashing as in SHA-256 with perceptual hashing, which is more like a blurry thumbnail of an image converted into numbers. Any minor alteration to an image will change it's cryptographic hash completely, making those pretty poor tools for finding images that might want to be hidden. Perceptual hashing should match e.g. resized versions of the image, but with them two "blurry thumbnail hashes" can match accidentally.

XorNot 3y ago

This is your regular reminder that the LAION dataset does not itself contain any image data at all: it is an index of image URLs on the the public internet which has been classified as having decent-quality tagging (image alt-text) and a few other bits of bulk, automated analysis done. The accessibility of CSAM content links within the dataset means those images are already on the internet being made publicly accessible by another website, which was found during the LAION image crawl. (interestingly this does rather imply that if LAION was able to compare images to the relevant CSAM hashes during the crawl then they'd simply be able to exclude them automatically - although I imagine that database is always growing)

sebastiangula 3y ago

If you are attempting to index the whole internet then you are doomed to eventually download something illegal.