3 ms·

Are there any projects working to automate the identification of offensive material with Machine Vision or Machine Learning? Replacing these peoples' jobs with

Are there any projects working to automate the identification of offensive material with Machine Vision or Machine Learning? Replacing these peoples' jobs with computers might actually be a humane act.

kartan 10y ago
As the article states, "Although algorithms and artificial intelligence have helped streamline the process of moderation, most technology companies that host user-generated content employ moderators like Soto to screen video, text, and images, to see if they violate company guidelines." So the algorithms are used to create the list of dubious material. But in the end you need a human being to look at the picture/video and call the police as needed. The day that AI can do that for us, they will do any other job also.
shakna 10y ago
I really wish there was. I did forensics for a couple years, working with the police force to access and/or recover incriminating evidence. I really, really wish that a machine could replace a person in that circumstance. And to a certain extent, it is possible. One of the really unfortunate parts of the process, is to know a video is incriminating, someone at some point has to see that video. It was part of my job to analyse and catalogue the evidence I recovered, such as saying who was in a video, and what happened to who inside it. ML will hopefully take over this at some point, but it is difficult for a computer, at this stage, to tell if say, penetration, is involved. It's moving in that direction, and that is absolutely fantastic. No person should be required to witness this crap. Hopefully, we can offload it to something that can, one day, be better than us at it.
- deleted 10y ago
  [deleted]
- jasonkostempski 10y ago
  I'm sorry anyone has to see stuff like that, but there is no way in hell machines can be allowed to start "confirming" such things without several people having to review it.
brynedwards 10y ago
There's this by Yahoo: https://github.com/yahoo/open_nsfw https://github.com/yahoo/open_nsfw
- drdaeman 10y ago
  > For this reason, the model we describe below focuses only on one type of NSFW content: pornographic images. It tries to detect a subset of pornography. Others may have different opinions, but personally I wouldn't relate porn to a category of "worst of humanity".
  - brynedwards 10y ago
    Good point, I guess detecting something as e.g. showing child abuse would be much more difficult.
DannyBee 10y ago
Yes, there are a lot. For example, NCMEC maintains databases of child porn hashes using photodna and other things. Machine learning and other algorithms are used to identify images that end up in this database (and others). Sadly, teaching machines what exploitation looks like is hard and it's not as simple as "identify things that have explicit sexual activity" and then "filter those things by people who appear to be 6 years old". People still often have to make a final determination because categorization is ... hard (and honestly, nobody wants to be the guy handling mislabeling, etc)
- village-idiot 10y ago
  The cost of false negatives in this area is so painfully high too.
monista 10y ago
Most obvious problem is: define offensive. What other people take as offense doesn't look offensive to me at all, and vice versa. There are other problems too. The only way to fight offenses is to restrict phrasing to Common Universal Neutral Terms. Of course, message consisting only of Common Universal Neutral Term Sequences cannot possibly contain an offense!

kartan 10y ago

As the article states, "Although algorithms and artificial intelligence have helped streamline the process of moderation, most technology companies that host user-generated content employ moderators like Soto to screen video, text, and images, to see if they violate company guidelines." So the algorithms are used to create the list of dubious material. But in the end you need a human being to look at the picture/video and call the police as needed. The day that AI can do that for us, they will do any other job also.

shakna 10y ago

I really wish there was. I did forensics for a couple years, working with the police force to access and/or recover incriminating evidence. I really, really wish that a machine could replace a person in that circumstance. And to a certain extent, it is possible. One of the really unfortunate parts of the process, is to know a video is incriminating, someone at some point has to see that video. It was part of my job to analyse and catalogue the evidence I recovered, such as saying who was in a video, and what happened to who inside it. ML will hopefully take over this at some point, but it is difficult for a computer, at this stage, to tell if say, penetration, is involved. It's moving in that direction, and that is absolutely fantastic. No person should be required to witness this crap. Hopefully, we can offload it to something that can, one day, be better than us at it.

deleted 10y ago
[deleted]
jasonkostempski 10y ago
I'm sorry anyone has to see stuff like that, but there is no way in hell machines can be allowed to start "confirming" such things without several people having to review it.

brynedwards 10y ago

There's this by Yahoo: https://github.com/yahoo/open_nsfw https://github.com/yahoo/open_nsfw

drdaeman 10y ago
> For this reason, the model we describe below focuses only on one type of NSFW content: pornographic images. It tries to detect a subset of pornography. Others may have different opinions, but personally I wouldn't relate porn to a category of "worst of humanity".
- brynedwards 10y ago
  Good point, I guess detecting something as e.g. showing child abuse would be much more difficult.

DannyBee 10y ago

Yes, there are a lot. For example, NCMEC maintains databases of child porn hashes using photodna and other things. Machine learning and other algorithms are used to identify images that end up in this database (and others). Sadly, teaching machines what exploitation looks like is hard and it's not as simple as "identify things that have explicit sexual activity" and then "filter those things by people who appear to be 6 years old". People still often have to make a final determination because categorization is ... hard (and honestly, nobody wants to be the guy handling mislabeling, etc)

village-idiot 10y ago
The cost of false negatives in this area is so painfully high too.

monista 10y ago

Most obvious problem is: define offensive. What other people take as offense doesn't look offensive to me at all, and vice versa. There are other problems too. The only way to fight offenses is to restrict phrasing to Common Universal Neutral Terms. Of course, message consisting only of Common Universal Neutral Term Sequences cannot possibly contain an offense!