4 ms·

That seems less like an argument against removing/preventing the inclusion of CSAM and more like an admitting on behalf of those building these image sets that

That seems less like an argument against removing/preventing the inclusion of CSAM and more like an admitting on behalf of those building these image sets that they're incredibly irresponsibly sourcing content for the aforementioned image sets. I'm not saying this is easy to do, in fact I'd wager it's incredibly hard: but it seems increasingly obvious that there was no curation, no standards, no approvals process, nothing but a ramshackle mad dash to include as much visual content as possible to train these models on, with zero consideration of whether the people who created it were alright with their content being used that way, if the content was legal to use either by licensing or by being incredibly illegal just by it's own existence. We get more and more stories of the ethical lapses of the massive companies/organizations behind this tech and the resounding chorus from my fellow engineers is seemingly just "well just remove it and let's keep going," or shrugging shoulders "that'll happen, we're trying to do research here." And fair enough but like, when you're doing research, you don't just run out and grab every chemical you can get your hands on and pour them all in a bucket and see what happens. How is this stuff not being found? What good are these datasets if they include this type of material? How can you trust the results you get from your training anymore once you know what went in?

127361 3y ago
Well I would consider a 0.55ppm contamination rate very low indeed? Mistakes happen in the creation of such datasets, it's human nature? Just as we can't police a society to the extent it's completely free of crime, otherwise we will not function as a society and/or we will find ourselves living in a totalitarian state?
- jprete 3y ago
  Including CSAM in your image set isn't excusable by "mistakes happen". Also, the totalitarian analogy is total nonsense.
  - whelp_24 3y ago
    I think this kinda shows that you don't really understand the problem. If you had a human sitdown and review each image, there still would be images that get through. Even if you told him to err heavily on the false positive side.
  - _jal 3y ago
    Just to be clear, you are saying that no collection of images can ever contain any CSAM, ever [1]. If some activity might run the risk of including it, that activity must be stopped. Is that correct? If so, why are you not calling for the internet to be shut down? The collection under discussion is just a subset of the internet. You'll also want to either ban or enforce review of all camera output, including closed-circuit cameras. Someone is going to have to watch all those watchers pretty closely... As for totalitarianism, well, how are you going to make any of this happen? As we just saw you'll need client-side visibility into everyone's visual data stores. Not to mention old photo albums world-wide contain images that would set off ImageNet. There are famous paintings and statues that would, too. [1] Do you exempt NCMEC? If so, how do you justify that?
  - failbuffer 3y ago
    You abstain from coffee, chocolate, and fish too I trust? Otherwise you're undoubtedly supporting trace amounts of slavery and child labor. I bet the device you're reading this with was also made with such (and if it doesn't then it's still supporting a genocidal regime). Get real. Scraped data is going to have problems. The expectation should be that reasonable measures are taken to filter proactively and that reactive measures are taken to remove illicit content found after that.
- ToucanLoucan 3y ago
  [flagged]
  - artninja1988 3y ago
    Laion 5b isn't a model, it's a dataset. Model devs can do their own cleaning before training a model on it. In fact Laion state that this is explicitly encouraged.
Vt71fcAqt7 3y ago
They did curate the dataset – that's what the LAION-Aesthetics dataset is, which is what StableDifffusion used. I'm pretty sure they never recommended using LAION-5B directly as it has low quality images and would take more time to train and run. It's just a collection of links.
orbital-decay 3y ago
LAION is just a large scale scrape of the web, it's a bunch of images with the alt text. So you can say the same about the web itself. > How can you trust the results you get from your training anymore once you know what went in? Before taking a moral high ground, please take your time to learn how it works. LAION is well known for its notoriously poor quality simply due to being humongous. There's all sorts of unwanted crap in here. Everyone training on it knows that and does filter it, because it's practically unusable otherwise.
- ToucanLoucan 3y ago
  "Look everyone working in this field knows you can't just use the datasets as provided, you have to filter out the bad quality images, the dead animals, and the child pornography." blank stare
  - orbital-decay 3y ago
    Yes? If you crawl the web or use a result of that, you have to filter out the crap. That's... how it works?
  - orf 3y ago
    Is really the idea that after taking an automated sampling of things posted online, some of those things will be things people post online, such alien one to you?

127361 3y ago

Well I would consider a 0.55ppm contamination rate very low indeed? Mistakes happen in the creation of such datasets, it's human nature? Just as we can't police a society to the extent it's completely free of crime, otherwise we will not function as a society and/or we will find ourselves living in a totalitarian state?

jprete 3y ago
Including CSAM in your image set isn't excusable by "mistakes happen". Also, the totalitarian analogy is total nonsense.
- whelp_24 3y ago
  I think this kinda shows that you don't really understand the problem. If you had a human sitdown and review each image, there still would be images that get through. Even if you told him to err heavily on the false positive side.
- _jal 3y ago
  Just to be clear, you are saying that no collection of images can ever contain any CSAM, ever [1]. If some activity might run the risk of including it, that activity must be stopped. Is that correct? If so, why are you not calling for the internet to be shut down? The collection under discussion is just a subset of the internet. You'll also want to either ban or enforce review of all camera output, including closed-circuit cameras. Someone is going to have to watch all those watchers pretty closely... As for totalitarianism, well, how are you going to make any of this happen? As we just saw you'll need client-side visibility into everyone's visual data stores. Not to mention old photo albums world-wide contain images that would set off ImageNet. There are famous paintings and statues that would, too. [1] Do you exempt NCMEC? If so, how do you justify that?
- failbuffer 3y ago
  You abstain from coffee, chocolate, and fish too I trust? Otherwise you're undoubtedly supporting trace amounts of slavery and child labor. I bet the device you're reading this with was also made with such (and if it doesn't then it's still supporting a genocidal regime). Get real. Scraped data is going to have problems. The expectation should be that reasonable measures are taken to filter proactively and that reactive measures are taken to remove illicit content found after that.
ToucanLoucan 3y ago
[flagged]
- artninja1988 3y ago
  Laion 5b isn't a model, it's a dataset. Model devs can do their own cleaning before training a model on it. In fact Laion state that this is explicitly encouraged.

Vt71fcAqt7 3y ago

They did curate the dataset – that's what the LAION-Aesthetics dataset is, which is what StableDifffusion used. I'm pretty sure they never recommended using LAION-5B directly as it has low quality images and would take more time to train and run. It's just a collection of links.

orbital-decay 3y ago

LAION is just a large scale scrape of the web, it's a bunch of images with the alt text. So you can say the same about the web itself. > How can you trust the results you get from your training anymore once you know what went in? Before taking a moral high ground, please take your time to learn how it works. LAION is well known for its notoriously poor quality simply due to being humongous. There's all sorts of unwanted crap in here. Everyone training on it knows that and does filter it, because it's practically unusable otherwise.

ToucanLoucan 3y ago
"Look everyone working in this field knows you can't just use the datasets as provided, you have to filter out the bad quality images, the dead animals, and the child pornography." blank stare
- orbital-decay 3y ago
  Yes? If you crawl the web or use a result of that, you have to filter out the crap. That's... how it works?
- orf 3y ago
  Is really the idea that after taking an automated sampling of things posted online, some of those things will be things people post online, such alien one to you?