4 ms·

> I believe they're using some automated filtering (see e.g. https://www.wired.com/story/tumblr-porn-ai-adult-content/ https://www.wired.com/story/tumblr-porn-a

> I believe they're using some automated filtering (see e.g. https://www.wired.com/story/tumblr-porn-ai-adult-content/ https://www.wired.com/story/tumblr-porn-ai-adult-content/). I assume it's easier to train an AI to recognise pornography generally, rather than a specific kind -- and now that I think about it, the process of training an AI to recognise child porn sounds extremely unpleasant and legally dubious. Quite the opposite - it's much easier to allow pornography and ban only child pornography than it is to create an automated system to detect pornography. PhotoDNA makes it easy to find matches, which can then be used to uncover the networks of people posting child pornography, which are then added back to the database. It doesn't require any computer vision at all. By contrast, banning all pornography does require computer vision of some sort, and that's much more difficult, as evidenced by how terrible the new Tumblr NSFW content detector is.

retsibsi 8y ago
Maybe I'm misunderstanding, but PhotoDNA seems to be a tool for identifying reposts and edits of already known illegal images -- those that have previously been found and had their hashes entered into the database. So if Tumblr had a problem with original content (which could include underage users posting explicit pictures of themselves, as well as the worse things we think of when we hear the phrase 'child porn'), I don't think that would help. edit: I found this quote from an article[1] published last November: "In its updated statement, Tumblr said that while every image uploaded to the platform is “scanned against an industry database of child sexual abuse material” to filter out explicit images, a “routine audit” discovered content that was absent from the database, allowing it to slip through the filter." So it looks like they might have already been using PhotoDNA. [1] https://www.theverge.com/2018/11/20/18104366/tumblr-ios-app-child-pornography-removed-from-app-store https://www.theverge.com/2018/11/20/18104366/tumblr-ios-app-...
- chimeracoder 8y ago
  > So it looks like they might have already been using PhotoDNA. Of course they were working with NCMEC (and therefore using PhotoDNA, which is owned by NCMEC); it would have been legal suicide for them not to. They were doing this for years, before they were even acquired by Yahoo. > Maybe I'm misunderstanding, but PhotoDNA seems to be a tool for identifying reposts and edits of already known illegal images -- those that have previously been found and had their hashes entered into the database. So if Tumblr had a problem with original content (which could include underage users posting explicit pictures of themselves, as well as the worse things we think of when we hear the phrase 'child porn'), I don't think that would help. How many people do you think have Tumblr accounts where they post only new, never-before-seen pornography of underage children, which has never been posted on any other blog before, and never once post a single photo that has been previously identified as child pornography[0]? Of those, how many do you think are able to ensure that none of their followers ever repost/reblog that photo on any other blog which also happens to contain at least one other photo that's been previously identified as child pornography? Of those, how many do you think are able to ensure that none of their followers have been previously identified as highly-connected nodes in the underground networks dedicated to sharing child pornography, and therefore pretty much exclusively post child pornography or follow people who they believe are likely to post child pornography? Of those, how many do you think are able to ensure that nobody who ever sees one of those photos ever decides to download it and upload it as an attachement to an unsent email in their Gmail drafts folder, or put it in a private Dropbox folder (shared with nobody), or sends it on any of the many "cloud" services which also actively monitor for child pornography and do the same sorts of graph analysis to identify people who are using their services to store or share child pornography? Again, once you understand how these underground networks work, and once you realize that this is mostly a problem of social graph analysis and not image recognition/classification, you realize that it's very easy to solve. [0] Remember, not just new to Tumblr, but new to everyone who is working with NCMEC/ICMEC (which means every large and not-so-large company in the entire world that hosts user-provided content).
  - retsibsi 8y ago
    You are obviously better informed than me, but if the problem is so easy to solve, and they were already using the relevant tool, why was it still a problem? Are you arguing that they failed through incompetence, or that they were lying/exaggerating as an excuse to ban all pornography for other reasons?
    - chimeracoder 8y ago
      > Are you arguing that they failed through incompetence, or that they were lying/exaggerating as an excuse to ban all pornography for other reasons? Yes, SESTA was the real reason that they banned pornography. SESTA imposed way too much liability on them to be able to support it. It had nothing to do with child porn. They are still liable for child porn. They still have to work with NCMEC and go through the same process for identifying, reporting, and removing child porn. None of that has changed.
snvzz 8y ago
So, if I understand correctly, this means that by banning all pornography in the name of preventing child porn, they'll actually be harming efforts on this uncovering of child pornography networks effort. This is unfortunate.
- chimeracoder 8y ago
  > So, if I understand correctly, this means that by banning all pornography in the name of preventing child porn, they'll actually be harming efforts on this uncovering of child pornography networks effort. This is unfortunate. Unfortunate, and also wholly predictable. This was exactly why advocacy groups of former victims of sex trafficking opposed SESTA in the first place, as did even the DOJ: they both recognized that it would ultimately make it harder to identify and rescue victims of sex trafficking.

retsibsi 8y ago

Maybe I'm misunderstanding, but PhotoDNA seems to be a tool for identifying reposts and edits of already known illegal images -- those that have previously been found and had their hashes entered into the database. So if Tumblr had a problem with original content (which could include underage users posting explicit pictures of themselves, as well as the worse things we think of when we hear the phrase 'child porn'), I don't think that would help. edit: I found this quote from an article[1] published last November: "In its updated statement, Tumblr said that while every image uploaded to the platform is “scanned against an industry database of child sexual abuse material” to filter out explicit images, a “routine audit” discovered content that was absent from the database, allowing it to slip through the filter." So it looks like they might have already been using PhotoDNA. [1] https://www.theverge.com/2018/11/20/18104366/tumblr-ios-app-child-pornography-removed-from-app-store https://www.theverge.com/2018/11/20/18104366/tumblr-ios-app-...

chimeracoder 8y ago
> So it looks like they might have already been using PhotoDNA. Of course they were working with NCMEC (and therefore using PhotoDNA, which is owned by NCMEC); it would have been legal suicide for them not to. They were doing this for years, before they were even acquired by Yahoo. > Maybe I'm misunderstanding, but PhotoDNA seems to be a tool for identifying reposts and edits of already known illegal images -- those that have previously been found and had their hashes entered into the database. So if Tumblr had a problem with original content (which could include underage users posting explicit pictures of themselves, as well as the worse things we think of when we hear the phrase 'child porn'), I don't think that would help. How many people do you think have Tumblr accounts where they post only new, never-before-seen pornography of underage children, which has never been posted on any other blog before, and never once post a single photo that has been previously identified as child pornography[0]? Of those, how many do you think are able to ensure that none of their followers ever repost/reblog that photo on any other blog which also happens to contain at least one other photo that's been previously identified as child pornography? Of those, how many do you think are able to ensure that none of their followers have been previously identified as highly-connected nodes in the underground networks dedicated to sharing child pornography, and therefore pretty much exclusively post child pornography or follow people who they believe are likely to post child pornography? Of those, how many do you think are able to ensure that nobody who ever sees one of those photos ever decides to download it and upload it as an attachement to an unsent email in their Gmail drafts folder, or put it in a private Dropbox folder (shared with nobody), or sends it on any of the many "cloud" services which also actively monitor for child pornography and do the same sorts of graph analysis to identify people who are using their services to store or share child pornography? Again, once you understand how these underground networks work, and once you realize that this is mostly a problem of social graph analysis and not image recognition/classification, you realize that it's very easy to solve. [0] Remember, not just new to Tumblr, but new to everyone who is working with NCMEC/ICMEC (which means every large and not-so-large company in the entire world that hosts user-provided content).
- retsibsi 8y ago
  You are obviously better informed than me, but if the problem is so easy to solve, and they were already using the relevant tool, why was it still a problem? Are you arguing that they failed through incompetence, or that they were lying/exaggerating as an excuse to ban all pornography for other reasons?
  - chimeracoder 8y ago
    > Are you arguing that they failed through incompetence, or that they were lying/exaggerating as an excuse to ban all pornography for other reasons? Yes, SESTA was the real reason that they banned pornography. SESTA imposed way too much liability on them to be able to support it. It had nothing to do with child porn. They are still liable for child porn. They still have to work with NCMEC and go through the same process for identifying, reporting, and removing child porn. None of that has changed.

snvzz 8y ago

So, if I understand correctly, this means that by banning all pornography in the name of preventing child porn, they'll actually be harming efforts on this uncovering of child pornography networks effort. This is unfortunate.

chimeracoder 8y ago
> So, if I understand correctly, this means that by banning all pornography in the name of preventing child porn, they'll actually be harming efforts on this uncovering of child pornography networks effort. This is unfortunate. Unfortunate, and also wholly predictable. This was exactly why advocacy groups of former victims of sex trafficking opposed SESTA in the first place, as did even the DOJ: they both recognized that it would ultimately make it harder to identify and rescue victims of sex trafficking.