5 ms·

I am surprised their system for catching bad content is only user reports/flags. While revenge porn and stuff is bad, there are a number of known child related

I am surprised their system for catching bad content is only user reports/flags. While revenge porn and stuff is bad, there are a number of known child related incidents. So if a private group of scumbags, are posting stuff to a private group, noone will know unless someone within reports. Who needs the darkweb when u can have a private Facebook group? Edit: as per replies below, they do use image recognition aswell for I guess known content lists

DanBC 9y ago
I suspect they sign up to the IWF lists for filtering.
- rhizome 9y ago
  International Weightlifting Federation?
  - bdrool 9y ago
    https://www.iwf.org.uk/become-a-member/services-for-members/image-hash-list https://www.iwf.org.uk/become-a-member/services-for-members/... https://www.iwf.org.uk/become-a-member/services-for-members/url-list/url-list-policy https://www.iwf.org.uk/become-a-member/services-for-members/...
arkitaip 9y ago
They also use image recognition to detect nudity and previously classified child porn.
- lightedman 9y ago
  No, they use Magic Number among a bunch of various tricks. You can't really call it image recognition, more like 'data-hash comparison.' To train true image recognition would require a LARGE subset of stored illegal pornography.
  - tomp 9y ago
    Why? Just use two separate datasets, one of images of adults and children, to learn age, the other of porn and non-porn, to learn what's "nudity" or "sex".
    - freeone3000 9y ago
      Knowledge transfer training is really hard. If you can do this (with different categories, for instance, determining what is nudity, then determining black people, to automatically classify porn with black actors automatically), it would be quite impressive. Results have nearly 5% error on CIFAR-10, which is full of clearly distinct categories - planes versus boats.
      - cr0sh 9y ago
        > it would be quite impressive Not only impressive, but likely very profitable. > Results have nearly 5% error on CIFAR-10 Any idea what the error rate is for human classification on CIFAR-10 images?
      - tomp 9y ago
        Can you explain (just a broad overview) why that wouldn't work? I mean, it's not like black porn actors are categorically different from black people in general... in particular, if you have an algorithm that can locate the face, you could just pass the face of the porn actor to the other algorithm to determine age/race/whatever...
  - salmonfamine 9y ago
    What do you mean by "Magic Number" in this context?
    - rarepostinlurkr 9y ago
      It's in the file format https://en.m.wikipedia.org/wiki/List_of_file_signatures https://en.m.wikipedia.org/wiki/List_of_file_signatures Also check out "libmagic"
TheAceOfHearts 9y ago
Defining "bad content" is difficult. I don't think we're nearly advanced enough avoid human intervention. Using user reports works well. Here's an example: if you go on YouTube you can find harmless videos of parents bathing their babies, and I doubt anyone would call that child pornography. Same goes for something like a birth video, even though it shows a vagina. How do you detect the differences? I think it'd be an improvement if Facebook were more open with their policies and allowed nudity, even if it required flagging content as mature. To get around this people end up placing tiny black bars in any of the "offensive" parts. But how does that seriously change anything? I'd say it's pretty reasonable to request users flag content. Detecting it programmatically is currently impossible, so what other choices do you have if you want to allow users to upload content? Should only multi-billion dollar companies be able to have user-uploaded content? How would you handle this issue if you were building a small media sharing site?

DanBC 9y ago

I suspect they sign up to the IWF lists for filtering.

rhizome 9y ago
International Weightlifting Federation?
- bdrool 9y ago
  https://www.iwf.org.uk/become-a-member/services-for-members/image-hash-list https://www.iwf.org.uk/become-a-member/services-for-members/... https://www.iwf.org.uk/become-a-member/services-for-members/url-list/url-list-policy https://www.iwf.org.uk/become-a-member/services-for-members/...

arkitaip 9y ago

They also use image recognition to detect nudity and previously classified child porn.

lightedman 9y ago
No, they use Magic Number among a bunch of various tricks. You can't really call it image recognition, more like 'data-hash comparison.' To train true image recognition would require a LARGE subset of stored illegal pornography.
- tomp 9y ago
  Why? Just use two separate datasets, one of images of adults and children, to learn age, the other of porn and non-porn, to learn what's "nudity" or "sex".
  - freeone3000 9y ago
    Knowledge transfer training is really hard. If you can do this (with different categories, for instance, determining what is nudity, then determining black people, to automatically classify porn with black actors automatically), it would be quite impressive. Results have nearly 5% error on CIFAR-10, which is full of clearly distinct categories - planes versus boats.
    - cr0sh 9y ago
      > it would be quite impressive Not only impressive, but likely very profitable. > Results have nearly 5% error on CIFAR-10 Any idea what the error rate is for human classification on CIFAR-10 images?
    - tomp 9y ago
      Can you explain (just a broad overview) why that wouldn't work? I mean, it's not like black porn actors are categorically different from black people in general... in particular, if you have an algorithm that can locate the face, you could just pass the face of the porn actor to the other algorithm to determine age/race/whatever...
- salmonfamine 9y ago
  What do you mean by "Magic Number" in this context?
  - rarepostinlurkr 9y ago
    It's in the file format https://en.m.wikipedia.org/wiki/List_of_file_signatures https://en.m.wikipedia.org/wiki/List_of_file_signatures Also check out "libmagic"

TheAceOfHearts 9y ago

Defining "bad content" is difficult. I don't think we're nearly advanced enough avoid human intervention. Using user reports works well. Here's an example: if you go on YouTube you can find harmless videos of parents bathing their babies, and I doubt anyone would call that child pornography. Same goes for something like a birth video, even though it shows a vagina. How do you detect the differences? I think it'd be an improvement if Facebook were more open with their policies and allowed nudity, even if it required flagging content as mature. To get around this people end up placing tiny black bars in any of the "offensive" parts. But how does that seriously change anything? I'd say it's pretty reasonable to request users flag content. Detecting it programmatically is currently impossible, so what other choices do you have if you want to allow users to upload content? Should only multi-billion dollar companies be able to have user-uploaded content? How would you handle this issue if you were building a small media sharing site?