6 ms·

Facebook built a monster it cannot control. Their monster is out of the cage and it's making frightening changes to society, rapidly and globally. I have an e

Facebook built a monster it cannot control. Their monster is out of the cage and it's making frightening changes to society, rapidly and globally. I have an extremely strong bias in favor of free expression, but Facebook altered the playing field. Do we really want ISIS beheading videos and terrorist recruitment to circulate freely, amplified and accelerated by Facebook's social graph? [0] Do we really want disinformation fanned out instantly to friends of friends that incites mob violence? [1] No, I don't want to live in a world where a faceless ML automatically censors everything I do. But if that's the only way to control the monster Facebook created, then maybe it's better to switch off the monster entirely. [0] https://www.theguardian.com/news/2017/may/22/facebook-moderator-guidelines-extreme-content-analysis https://www.theguardian.com/news/2017/may/22/facebook-modera... [1] https://www.nytimes.com/2018/07/18/technology/facebook-to-remove-misinformation-that-leads-to-violence.html https://www.nytimes.com/2018/07/18/technology/facebook-to-re...

adrianmonk 8y ago
I sometimes wonder if people asked the same questions when postal services and telephones were invented. People are writing letters to each other and conspiring about things, and no one can even track it! People are calling each other on the telephone and plotting crimes with no one the wiser! I suppose in at least one way they did, because mail fraud is a crime with special extra penalties. But overall, it seems like we've accepted the idea that humans are humans, some do good things and some do bad things, and greater communication amplifies their ability to do both. Are we now saying that we've amplified as much as we want to, and we should stop? Or maybe we're saying that, because this is all taking place on a computer, we now think it should be a solvable problem, when it wasn't before with a telephone.
- escherplex 8y ago
  And by extension, maybe as the POGO comic strip observed We have met the enemy and he is us
- panarky 8y ago
  This is a great example, thank you. The good people who operate the postal service, courier services and telephone networks also deploy very capable security tools to ensure their services aren't used to create mayhem. It's not perfect, as occasionally FedEx will deliver a bomb, or the post office will deliver an envelope with anthrax powder, or a threat will be delivered by telephone. But you can tell that the security tools are operating effectively because these events are rare and isolated. When was the last time 50 people died because of DHL or Verizon? In contrast, Facebook released their creation on the world without the tools needed to keep it safe. Now they've lost control and are trying to deflect blame on "the industry". We cannot allow Facebook to escape accountability. It's their responsibility, not "the community" or "the industry". If they cannot fix it, and fast, then they must be rigorously regulated to disable features that cause more harm than good. Zuckerberg's latest strategy is to encrypt everything so even Facebook can't read it. This is one more cynical attempt to force others to bear the negative externalities of the monster he created.
  - deno 8y ago
    What has Facebook exactly “unleashed” upon the world? Facebook didn’t invent live streaming. You think someone determined won’t be able to do a one-off stream without Facebook? > In contrast, Facebook released their creation on the world without the tools needed to keep it safe. Instead we should be more like China. They keep their citizens safe, right? Yes, free speech has negative externalities. But imagine world without it.
    - panarky 8y ago
      > Facebook didn't invent live streaming When you understand why the terrorist selected Facebook to livestream instead of hundreds of other services, then you'll understand why Facebook is the problem and not livestreaming itself.
      - deno 8y ago
        If you have an argument to make then make it. The live stream was viewed meager 200 times. The logistical purpose of a live stream is that it’s hard to upload videos when you’re in a shoot out with police or dead. Perhaps you should blame 4G networks and portable cameras for enabling this tragedy.
- NeedMoreTea 8y ago
  There is an important quantitative difference compared with post and phone though. If I want to start an alt newsletter and post it out, or conspire via phone, I am very limited in reach. A mailshot in the thousands could take days to stuff, address and stamp, and the costs escalate quickly. Few aside from fellow nutters and conspiracy theorists will know my little alt postal newsletter even exists. Social gives unlimited reach if it goes viral. Thus the overriding aim of every alt and conspiracy group is to go viral and achieve global notoriety. Few knew or cared and certainly didn't have access to what a plane hijackers side of the story was in the 1980s and earlier. Or their propaganda. There might be short extracts in follow up reports. Giving them the oxygen of publicity was almost universally considered a bad thing. I entirely share the view of the NZ Prime Minister in refusing to use his name. Then the US services showed up. Yes we should stop as we've amplified too far. What many Americans call outrageous censorship, often conveniently ignoring it being a private service rather than speaking against a government, much of the rest of the world might call reasonable moderation. If FB can't moderate and block it, or find a way for users to moderate it, perhaps it's time to limit, restrict and censure. It's certain that if FB won't find a way to do it reasonably and fairly, and soon, governments will start to compel them and others. Each event and atrocity makes it that little more certain.
  - roenxi 8y ago
    That argument has a massive unstated assumption in it - that there are substantial communities of people who want to go out and cause irrational harm; but the only thing stopping them is communication difficulties. Restricting Facebook will not stop insane lone-wolf style attacks. That seems unlikely. More relevantly, it seems likely that if such communities exist there is a reason they exist and solving that reason should be an urgent priority.
    - NeedMoreTea 8y ago
      No such assumption exists here. It doesn't matter in the least if it is a substantial community or not - this was a one-off event from an individual. Still doesn't follow that he should be enabled. Nevertheless there are groups who wish to cause harm - anti vaxxers and up who benefit from the reach of social. You;re right that restrictions won't stop lone wolf madness, but publicising their actual message and videos encourages every other copy-cat to be "outrageous enough". The phenomenon of copycats is well enough known and seen in all sorts of things, even atrocity and suicide methods.
- NikkiA 8y ago
  Early telephone was inherently tracable, since you had to interact with an operator, who know who you were, who you were calling, and logged the call for billing purposes. Public telephones added a slight wrinkle, but not much of one. It wasn't until the operator system was removed that conversations might appear to be untracable, but by then automatic logging and billing was in place and thus calls were as tracable as they were in the early days, and the public already knew that the police could/would 'trace calls' before this switchover, it had become a movie trope. So no, I don't imagine people were as concerned, at least with telephones. Post probably followed a similar curve - early post was person to person and involved an intermediary (horse rider/carriage driver) who probably kept some records, although in the very early days likely knew all of the people involved. Not until centuries later did national post systems come along with some semblance of anonymity to the act of posting a letter.
- tgsovlerkhgsel 8y ago
  Not just that, we also recognized the importance of private communications, and created postal secrecy laws.
lvs 8y ago
Essentially, this is a subset of the argument that more prevalent global communication has actually been far worse for society and far easier to exploit than expected by tech optimists. It's not a new argument, but it's still one we're trying to find our way out of a paper bag about. Yesterday somebody on the train was really adamant that I should google something about Michael Jackson being alive and reincarnated -- truly believed it with all his heart. Where is this disinfo coming from? How can we stop it? The horrifying answer would be that it's coming from everywhere, and that we can't stop it.
mr_toad 8y ago
What you say about Facebook other people say about the internet as a whole. And what some people say about the internet, others say about society as a whole. Censor Facebook, or all of the Internet? Or all social discourse?
- panarky 8y ago
  For now, let's focus on the bad actor that is responsible for 95% of the weaponization of social media. After we solve the Facebook problem, then we can see what the next biggest threat is, and address it.
  - ng12 8y ago
    Yep. And the next biggest threat, and the next biggest, until we're at a point where humans can't be trusted with their own private conversations (and eventually private thoughts).
- Mirioron 8y ago
  To keep society civil we need a strong leader that will protect us from immorality and evil! ^ This is how you get authoritarianism. Authoritarianism usually ends in suffering. I will take a free society over that, even if it comes with disgusting content.
Diederich 8y ago
Your points are well taken. I believe, more generally, that the problems we're seeing with FB and social media overall were an entirely inevitable result of, effectively, free and unlimited exchange of information among most of the people on this planet. Even more generally, I think that the effectively free and unlimited ability to exchange information was an entirely inevitable result of the arc of technology as it exists here on planet earth. Our ability to magically beam information and emotions, at effectively no cost, and with effectively no limits, is, I believe, making us crazy, because our evolutionary neurology is still pretty miuch the same as it has been for the past 100,000 years. For the record, I firmly believe that FB and the other social media companies, and tech in general, have objectively produced more good than bad. It's just that our still relatively primitive brains can't handle this new kind of bad.
aeternus 8y ago
The Facebook amplification was relatively minor here. They 'amplified' it to 4000 people and acted quickly to limit the amplification. The real amplification was caused by people, not by platforms. The amazing stat here is that a staggering 1.5 million people attempted to re-uploaded it. Suppose the attacker uploaded it to 8chan instead? I doubt the final numbers would be much different. People are going to decide for themselves whether or not they want to watch something on the internet.
- panarky 8y ago
  That's what the deceptive Facebook PR would lead us to believe. It's very carefully worded to give that impression, but it doesn't actually say that. Facebook provided that specific and meaningless fact to distract us from the truly important facts that they're still hiding. (1) How many minutes from the beginning of the livestream until the last re-uploaded copy was removed? (2) How many Facebook users viewed even part of the livestream or any of the re-uploaded copies? The answers are likely (1) thousands of minutes and (2) millions of views. Naturally these two numbers are not to be found anywhere in Facebook's deceptive explanation. Facebook let 300k people re-upload the video without being stopped at upload. Facebook pushed it to their friends, and when their friends interacted with it Facebook pushed it to friends of friends. That fanout push to feeds isn't comparable to 8chan.

adrianmonk 8y ago

I sometimes wonder if people asked the same questions when postal services and telephones were invented. People are writing letters to each other and conspiring about things, and no one can even track it! People are calling each other on the telephone and plotting crimes with no one the wiser! I suppose in at least one way they did, because mail fraud is a crime with special extra penalties. But overall, it seems like we've accepted the idea that humans are humans, some do good things and some do bad things, and greater communication amplifies their ability to do both. Are we now saying that we've amplified as much as we want to, and we should stop? Or maybe we're saying that, because this is all taking place on a computer, we now think it should be a solvable problem, when it wasn't before with a telephone.

escherplex 8y ago
And by extension, maybe as the POGO comic strip observed We have met the enemy and he is us
panarky 8y ago
This is a great example, thank you. The good people who operate the postal service, courier services and telephone networks also deploy very capable security tools to ensure their services aren't used to create mayhem. It's not perfect, as occasionally FedEx will deliver a bomb, or the post office will deliver an envelope with anthrax powder, or a threat will be delivered by telephone. But you can tell that the security tools are operating effectively because these events are rare and isolated. When was the last time 50 people died because of DHL or Verizon? In contrast, Facebook released their creation on the world without the tools needed to keep it safe. Now they've lost control and are trying to deflect blame on "the industry". We cannot allow Facebook to escape accountability. It's their responsibility, not "the community" or "the industry". If they cannot fix it, and fast, then they must be rigorously regulated to disable features that cause more harm than good. Zuckerberg's latest strategy is to encrypt everything so even Facebook can't read it. This is one more cynical attempt to force others to bear the negative externalities of the monster he created.
- deno 8y ago
  What has Facebook exactly “unleashed” upon the world? Facebook didn’t invent live streaming. You think someone determined won’t be able to do a one-off stream without Facebook? > In contrast, Facebook released their creation on the world without the tools needed to keep it safe. Instead we should be more like China. They keep their citizens safe, right? Yes, free speech has negative externalities. But imagine world without it.
  - panarky 8y ago
    > Facebook didn't invent live streaming When you understand why the terrorist selected Facebook to livestream instead of hundreds of other services, then you'll understand why Facebook is the problem and not livestreaming itself.
    - deno 8y ago
      If you have an argument to make then make it. The live stream was viewed meager 200 times. The logistical purpose of a live stream is that it’s hard to upload videos when you’re in a shoot out with police or dead. Perhaps you should blame 4G networks and portable cameras for enabling this tragedy.
NeedMoreTea 8y ago
There is an important quantitative difference compared with post and phone though. If I want to start an alt newsletter and post it out, or conspire via phone, I am very limited in reach. A mailshot in the thousands could take days to stuff, address and stamp, and the costs escalate quickly. Few aside from fellow nutters and conspiracy theorists will know my little alt postal newsletter even exists. Social gives unlimited reach if it goes viral. Thus the overriding aim of every alt and conspiracy group is to go viral and achieve global notoriety. Few knew or cared and certainly didn't have access to what a plane hijackers side of the story was in the 1980s and earlier. Or their propaganda. There might be short extracts in follow up reports. Giving them the oxygen of publicity was almost universally considered a bad thing. I entirely share the view of the NZ Prime Minister in refusing to use his name. Then the US services showed up. Yes we should stop as we've amplified too far. What many Americans call outrageous censorship, often conveniently ignoring it being a private service rather than speaking against a government, much of the rest of the world might call reasonable moderation. If FB can't moderate and block it, or find a way for users to moderate it, perhaps it's time to limit, restrict and censure. It's certain that if FB won't find a way to do it reasonably and fairly, and soon, governments will start to compel them and others. Each event and atrocity makes it that little more certain.
- roenxi 8y ago
  That argument has a massive unstated assumption in it - that there are substantial communities of people who want to go out and cause irrational harm; but the only thing stopping them is communication difficulties. Restricting Facebook will not stop insane lone-wolf style attacks. That seems unlikely. More relevantly, it seems likely that if such communities exist there is a reason they exist and solving that reason should be an urgent priority.
  - NeedMoreTea 8y ago
    No such assumption exists here. It doesn't matter in the least if it is a substantial community or not - this was a one-off event from an individual. Still doesn't follow that he should be enabled. Nevertheless there are groups who wish to cause harm - anti vaxxers and up who benefit from the reach of social. You;re right that restrictions won't stop lone wolf madness, but publicising their actual message and videos encourages every other copy-cat to be "outrageous enough". The phenomenon of copycats is well enough known and seen in all sorts of things, even atrocity and suicide methods.
NikkiA 8y ago
Early telephone was inherently tracable, since you had to interact with an operator, who know who you were, who you were calling, and logged the call for billing purposes. Public telephones added a slight wrinkle, but not much of one. It wasn't until the operator system was removed that conversations might appear to be untracable, but by then automatic logging and billing was in place and thus calls were as tracable as they were in the early days, and the public already knew that the police could/would 'trace calls' before this switchover, it had become a movie trope. So no, I don't imagine people were as concerned, at least with telephones. Post probably followed a similar curve - early post was person to person and involved an intermediary (horse rider/carriage driver) who probably kept some records, although in the very early days likely knew all of the people involved. Not until centuries later did national post systems come along with some semblance of anonymity to the act of posting a letter.
tgsovlerkhgsel 8y ago
Not just that, we also recognized the importance of private communications, and created postal secrecy laws.

lvs 8y ago

Essentially, this is a subset of the argument that more prevalent global communication has actually been far worse for society and far easier to exploit than expected by tech optimists. It's not a new argument, but it's still one we're trying to find our way out of a paper bag about. Yesterday somebody on the train was really adamant that I should google something about Michael Jackson being alive and reincarnated -- truly believed it with all his heart. Where is this disinfo coming from? How can we stop it? The horrifying answer would be that it's coming from everywhere, and that we can't stop it.

mr_toad 8y ago

What you say about Facebook other people say about the internet as a whole. And what some people say about the internet, others say about society as a whole. Censor Facebook, or all of the Internet? Or all social discourse?

panarky 8y ago
For now, let's focus on the bad actor that is responsible for 95% of the weaponization of social media. After we solve the Facebook problem, then we can see what the next biggest threat is, and address it.
- ng12 8y ago
  Yep. And the next biggest threat, and the next biggest, until we're at a point where humans can't be trusted with their own private conversations (and eventually private thoughts).
Mirioron 8y ago
To keep society civil we need a strong leader that will protect us from immorality and evil! ^ This is how you get authoritarianism. Authoritarianism usually ends in suffering. I will take a free society over that, even if it comes with disgusting content.

Diederich 8y ago

Your points are well taken. I believe, more generally, that the problems we're seeing with FB and social media overall were an entirely inevitable result of, effectively, free and unlimited exchange of information among most of the people on this planet. Even more generally, I think that the effectively free and unlimited ability to exchange information was an entirely inevitable result of the arc of technology as it exists here on planet earth. Our ability to magically beam information and emotions, at effectively no cost, and with effectively no limits, is, I believe, making us crazy, because our evolutionary neurology is still pretty miuch the same as it has been for the past 100,000 years. For the record, I firmly believe that FB and the other social media companies, and tech in general, have objectively produced more good than bad. It's just that our still relatively primitive brains can't handle this new kind of bad.

aeternus 8y ago

The Facebook amplification was relatively minor here. They 'amplified' it to 4000 people and acted quickly to limit the amplification. The real amplification was caused by people, not by platforms. The amazing stat here is that a staggering 1.5 million people attempted to re-uploaded it. Suppose the attacker uploaded it to 8chan instead? I doubt the final numbers would be much different. People are going to decide for themselves whether or not they want to watch something on the internet.

panarky 8y ago
That's what the deceptive Facebook PR would lead us to believe. It's very carefully worded to give that impression, but it doesn't actually say that. Facebook provided that specific and meaningless fact to distract us from the truly important facts that they're still hiding. (1) How many minutes from the beginning of the livestream until the last re-uploaded copy was removed? (2) How many Facebook users viewed even part of the livestream or any of the re-uploaded copies? The answers are likely (1) thousands of minutes and (2) millions of views. Naturally these two numbers are not to be found anywhere in Facebook's deceptive explanation. Facebook let 300k people re-upload the video without being stopped at upload. Facebook pushed it to their friends, and when their friends interacted with it Facebook pushed it to friends of friends. That fanout push to feeds isn't comparable to 8chan.