3 ms·

There's a lot of pessimism in the comments. I believe this is one of the best uses for an AI that we've come up with. First, let's go into the details: 1. They

by Shank 9y ago

There's a lot of pessimism in the comments. I believe this is one of the best uses for an AI that we've come up with. First, let's go into the details: 1. They only monitor and elevate to moderators content from posts and Facebook live. 2. Moderators can then take action, and reports from this system are prioritized over other non-pressing issues. The actual life saving chances of this system are huge. Just from a pure safety area, it can help detect and alert someone in an authority area that something bad is happening when nobody else might. The bystander effect is a real problem, and I'd imagine that it might be even greater on social media. In many cases, posts that have suicidal tendencies are screaming for help from their friends. They're posting because they have no options left. In these cases, it's a safety net that can stop loss of life. Moderators make the final say, however, so it's not just AI diagnosis. It's AI warning that gets elevated to a human before anyone else. That's more eyes on the problem that can potentially help. Are there other implications for AI? Yep. Advertising? Yep. Other types of tracking? Certainly. But if we're going to use AI in those fields anyway, we may as well extend it to trying to save lives.

freedomben 9y ago
In general I agree with you. However I do worry about unintended consequences. For example, if someone knows that Facebook AI is looking for this, and they don't want their attempt thwarted, they may actually hide signs that might otherwise have been present and been opportunities. I have had friends commit suicide and it is absolutely devastating. I completely understand the desire to "do something." But one of my friends actually Googled, "signs of suicide" and then made sure not to give any of those signs, because he did not want to be stopped. Everybody is so different, so it's difficult to analyze the trade offs. But it is something I think about.
dwaltrip 9y ago
Facebook doesn't have the moral legitimacy to take on this task. No for-profit tech company should be diagnosing or reporting mental illness. Regardless, aren't there laws against such things? Wouldn't HIPAA be relevant here? I do understand the desire to address these troubling issues. I would argue these are symptoms of underlying issues that are better addressed at a deeper level. I'm thinking of the fractured and isolated nature of living in many modern cities, increased loneliness, huge overload of low quality, addictive information, inability to unplug from our devices and be mindful of our current experience, false sense of connectedness found in social media, etc.
anigbrowl 9y ago
I help to run a suicide/depression support group on FB and I do not at all trust FB as a corporation to do a good job on this. Having had friends who've been involuntarily hospitalized and all that goes with that I'm extremely skeptical that this will turn out well. 'But it might save lives!' is true, but it might also wreck lives by institutionalizing people that thought they were venting their feelings to a trusted circle of friends in a private context. (Don't start with 'nothing is private on FB' please.)
- anthrowaway 9y ago
  For those who aren't familiar, this is how it can play out: 1. Friend^H^H^H^H^H Facebook notices you said something that sounds suicidal to them. 2. They call the police. 3. The police show up and take you to an emergency psychiatric facility (you don't have as much choice in the matter as you think you do, and you can't anticipate the consequences). 4. The facility keeps you in their custody until your friends or relatives can convince them to release you. They are financially incentivized to keep you there. You have limited contact with the outside world. You are effectively a prisoner. 5. Once you get out, a. You get to explain why you didn't show up for N days of work b. You get a bill in the mail for thousands of dollars c. The courts will not help you. Based on a true story. Automating this is not acceptable.
  - rootsudo 9y ago
    Agree 100%. Happened to me too when I went to the ER for a panic attack. I reported the doctor and he told me to later go fuck myself. What a world. I got baker acted for going to the ER for a panic attack because of my anxiety.
AlexandrB 9y ago
> But if we're going to use AI in those fields anyway, we may as well extend it to trying to save lives. Absolutely, but the incentives of the organization that does so need to be aligned with the people it's trying to help. Facebook's incentives are aligned with those of their advertisers not their users. Even if this system is being conceived by well-meaning people with the best intentions the institutional pressure will be to monetize any information gathered and any conclusions arrived at from that information. It won't happen right away, but it's inevitable. For a contemporary example look at Google's transformation from "Don't be evil" to "we're going to gather tracking data on you wherever and whenever we can".
ThrowawayP 9y ago
Bear in mind that what will occur is that armed police will be dispatched to take the unfortunate individual into custody and transport them to a hospital to be involuntarily locked up in the mental health ward for psychiatric evaluation and it can be some number of days before they get a judicial hearing to attempt to secure their own release. (See https://en.wikipedia.org/wiki/Involuntary_commitment_internationally#United_States https://en.wikipedia.org/wiki/Involuntary_commitment_interna...) By all reports, it can be a highly traumatic and humiliating process. All this, based on some algorithm of unknown quality flagging posts for evaluation and reporting by moderators with no psychiatric or other medical training. And somehow this is supposed to be a good thing?

by Shank 9y ago

freedomben 9y ago

In general I agree with you. However I do worry about unintended consequences. For example, if someone knows that Facebook AI is looking for this, and they don't want their attempt thwarted, they may actually hide signs that might otherwise have been present and been opportunities. I have had friends commit suicide and it is absolutely devastating. I completely understand the desire to "do something." But one of my friends actually Googled, "signs of suicide" and then made sure not to give any of those signs, because he did not want to be stopped. Everybody is so different, so it's difficult to analyze the trade offs. But it is something I think about.

dwaltrip 9y ago

Facebook doesn't have the moral legitimacy to take on this task. No for-profit tech company should be diagnosing or reporting mental illness. Regardless, aren't there laws against such things? Wouldn't HIPAA be relevant here? I do understand the desire to address these troubling issues. I would argue these are symptoms of underlying issues that are better addressed at a deeper level. I'm thinking of the fractured and isolated nature of living in many modern cities, increased loneliness, huge overload of low quality, addictive information, inability to unplug from our devices and be mindful of our current experience, false sense of connectedness found in social media, etc.

anigbrowl 9y ago

I help to run a suicide/depression support group on FB and I do not at all trust FB as a corporation to do a good job on this. Having had friends who've been involuntarily hospitalized and all that goes with that I'm extremely skeptical that this will turn out well. 'But it might save lives!' is true, but it might also wreck lives by institutionalizing people that thought they were venting their feelings to a trusted circle of friends in a private context. (Don't start with 'nothing is private on FB' please.)

anthrowaway 9y ago
For those who aren't familiar, this is how it can play out: 1. Friend^H^H^H^H^H Facebook notices you said something that sounds suicidal to them. 2. They call the police. 3. The police show up and take you to an emergency psychiatric facility (you don't have as much choice in the matter as you think you do, and you can't anticipate the consequences). 4. The facility keeps you in their custody until your friends or relatives can convince them to release you. They are financially incentivized to keep you there. You have limited contact with the outside world. You are effectively a prisoner. 5. Once you get out, a. You get to explain why you didn't show up for N days of work b. You get a bill in the mail for thousands of dollars c. The courts will not help you. Based on a true story. Automating this is not acceptable.
- rootsudo 9y ago
  Agree 100%. Happened to me too when I went to the ER for a panic attack. I reported the doctor and he told me to later go fuck myself. What a world. I got baker acted for going to the ER for a panic attack because of my anxiety.

AlexandrB 9y ago

> But if we're going to use AI in those fields anyway, we may as well extend it to trying to save lives. Absolutely, but the incentives of the organization that does so need to be aligned with the people it's trying to help. Facebook's incentives are aligned with those of their advertisers not their users. Even if this system is being conceived by well-meaning people with the best intentions the institutional pressure will be to monetize any information gathered and any conclusions arrived at from that information. It won't happen right away, but it's inevitable. For a contemporary example look at Google's transformation from "Don't be evil" to "we're going to gather tracking data on you wherever and whenever we can".

ThrowawayP 9y ago

Bear in mind that what will occur is that armed police will be dispatched to take the unfortunate individual into custody and transport them to a hospital to be involuntarily locked up in the mental health ward for psychiatric evaluation and it can be some number of days before they get a judicial hearing to attempt to secure their own release. (See https://en.wikipedia.org/wiki/Involuntary_commitment_internationally#United_States https://en.wikipedia.org/wiki/Involuntary_commitment_interna...) By all reports, it can be a highly traumatic and humiliating process. All this, based on some algorithm of unknown quality flagging posts for evaluation and reporting by moderators with no psychiatric or other medical training. And somehow this is supposed to be a good thing?