5 ms·

It's would be trivially easy to scan these streams of messages through some kind of ML classifier and identify cases of extortion before they result in suicide.

It's would be trivially easy to scan these streams of messages through some kind of ML classifier and identify cases of extortion before they result in suicide. Once the platforms identify the perps, they can block them from the platform. I hope this lawsuit makes the costs of platform inaction so high that they build out detection systems.

ctoth 3y ago
I can't believe you are actively suggesting this. Even in this horrific case. Okay, so you train your classifier on extortion. Then you train it to detect sexual content. Then you train it to detect potential piracy. Then you train it to detect wrongthink. Then you don't have a free society anymore.
- ok_dad 3y ago
  We don't have a free society when predators can easily prey on victims like this with no repercussions. That's not freedom for the victims, that's just another form of prison. I personally don't know what the answer is, but it isn't "do nothing to protect freedom".
- kenjackson 3y ago
  Don't we already do this with email spam? And movie/TV ratings? You could slippery slope any classifier for any purpose into this -- this is why hopefully we have brains to determine good and less good uses of technology.
- nullstyle 3y ago
  Social media is not required to have a free society. I don't think the progression you predict is how things would play out. IMO, At some point, people will simply talk in person (well, off-network) to be sociable. Casual piracy, e.g. link sharing in DMs, would be a tipping point if I had to guess.
- gnfargbl 3y ago
  Or: you stop after you train your classifier on extortion, because society is capable of having and maintaining standards and because not every slope has to be slippery? There are many other cases in which we allow limited intrusions into individual freedom in order to protect society. For instance, here in the UK, a police officer can enter my house without a warrant if they believe that it is necessary to facilitate an arrest for an indictable offence -- and that has been the case since at least 1984. If your way of thinking was accurate, police should now be routinely entering my house and beating confessions out of me.
  - xboxnolifes 3y ago
    Things don't need to be routine to be a problem. Just selectively enforced.
mecsred 3y ago
Why stop with messages? We have so much more information on people these days than chat history. Why not feed all of it into some kind of opaque classifier that can give us "grades" of human being. Then we could shadowban people below a certain grade from social media commons to keep scum like this out. Doesn't that sound like a fantastic society???
thomastjeffery 3y ago
It is impossible to exclusively classify bad behavior. Truth and lie are written the same. ML is useful because it can continue functioning with ambiguous data. Even so, it can't remove it or resolve it.
nindalf 3y ago
This already exists. There are classifiers for Scams, Sextortion and other crimes. Like with all interventions, recall isn't perfect.

ctoth 3y ago

I can't believe you are actively suggesting this. Even in this horrific case. Okay, so you train your classifier on extortion. Then you train it to detect sexual content. Then you train it to detect potential piracy. Then you train it to detect wrongthink. Then you don't have a free society anymore.

ok_dad 3y ago
We don't have a free society when predators can easily prey on victims like this with no repercussions. That's not freedom for the victims, that's just another form of prison. I personally don't know what the answer is, but it isn't "do nothing to protect freedom".
kenjackson 3y ago
Don't we already do this with email spam? And movie/TV ratings? You could slippery slope any classifier for any purpose into this -- this is why hopefully we have brains to determine good and less good uses of technology.
nullstyle 3y ago
Social media is not required to have a free society. I don't think the progression you predict is how things would play out. IMO, At some point, people will simply talk in person (well, off-network) to be sociable. Casual piracy, e.g. link sharing in DMs, would be a tipping point if I had to guess.
gnfargbl 3y ago
Or: you stop after you train your classifier on extortion, because society is capable of having and maintaining standards and because not every slope has to be slippery? There are many other cases in which we allow limited intrusions into individual freedom in order to protect society. For instance, here in the UK, a police officer can enter my house without a warrant if they believe that it is necessary to facilitate an arrest for an indictable offence -- and that has been the case since at least 1984. If your way of thinking was accurate, police should now be routinely entering my house and beating confessions out of me.
- xboxnolifes 3y ago
  Things don't need to be routine to be a problem. Just selectively enforced.

mecsred 3y ago

Why stop with messages? We have so much more information on people these days than chat history. Why not feed all of it into some kind of opaque classifier that can give us "grades" of human being. Then we could shadowban people below a certain grade from social media commons to keep scum like this out. Doesn't that sound like a fantastic society???

thomastjeffery 3y ago

It is impossible to exclusively classify bad behavior. Truth and lie are written the same. ML is useful because it can continue functioning with ambiguous data. Even so, it can't remove it or resolve it.

nindalf 3y ago

This already exists. There are classifiers for Scams, Sextortion and other crimes. Like with all interventions, recall isn't perfect.