10 ms·

Facebook and the Normalization of Deviance

89 points by mitchbob 6y ago

darig 6y ago
The New Yorker should drop the "the". It's cleaner. Normalizing the amplified prejudiced attacks of perceived deviance is not the same as normalizing deviance.
jchw 6y ago
Archive link: https://archive.is/hnBQa https://archive.is/hnBQa
- etaioinshrdlu 6y ago
  I'm not complaining, but I'd like to point out that I think this violates HN's terms of service: "You agree to not use the Site to: email or otherwise upload any content that (i) infringes any intellectual property or other proprietary rights of any party;" It seems like sharing a link to copyright infringing content (archive.is) could be considered Contributory copyright infringement (https://en.wikipedia.org/wiki/Contributory_copyright_infringement https://en.wikipedia.org/wiki/Contributory_copyright_infring...) By analogy, how is this so different from posting a link to a pirated movie originally from Netflix? Because it's text? I think we have a case of normalization of deviance right here on HN. Illegal or stolen content is posted in the comments here all the time.
  - jchw 6y ago
    Rules were made to be broken. In this case, dang themselves has commented on the matter[1]. Complaining about paywalls is off topic and discouraged, and linking archives is tolerated. I disagree with your sentiment vehemently. Firstly, law is not morality; people break the law all the time, every day, at their own discretion. It was made by humans and is not infallible. Secondly, I can’t just do a few GET requests to Netflix and get the full movie, yet this archive.is scraper has no trouble. So I guess it wasn’t very protected to begin with. The internet giveth, and the internet taketh away. They can get their better protection by not serving full articles publicly. They presumably do not because they would like to eat their cake and have it too with regards to indexing. This is hugely off topic so apologies but there is a reasonable chance I am not replying beyond this point. [1]: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989
  - madars 6y ago
    Not sure what you mean here. Opening the original link in an incognito window yields full text of the article (maybe you are out of free articles, and they are counting?) Submitting articles to archive.is or archive.org serves the important function of maintaining an accurate historical record, and prevent link rot.
deleted 6y ago
[deleted]
pope_meat 6y ago
We mustn't forget the cambridge analytica shenanigans either.
deleted 6y ago
[deleted]
sneak 6y ago
I think the real problem is that a small set of companies are exercising control over what people see and read. Facebook's censorship (which includes algorithmic timelines, prioritizing some things and censoring others) is an existential danger to the wellbeing of our society. Any organization that can decide what links and text can be shared from neighbor to neighbor or from friend to friend is in a position of massive power. They even prohibit links to certain tweets in DMs.
- im_down_w_otp 6y ago
  Aren't their algorithms all designed to optimize for engagement? Isn't engagement just the measure of how much time you spend on and interacting with their system? Doesn't that mean that they're not deciding what to keep feeding us, but rather that we are? It seems like what they've done is basically automate, "Give the people what they want.", as it were. We produce the clickbait, we click on the clickbait, then we get fed more and more of what we click on. I tend to think that this probably boils down to rent-seeking via exploiting self-destructive human behavior/biology, so I'm not too fond of it for those reasons. But, that doesn't seem very much like Facebook censoring things as it does them feeding everyone exactly what people indicate to them with their clicks and comments they'd like to be fed. I suppose there's an interesting discussion about what lengths companies like Facebook should go to save people from themselves, but to me they're basically another form of vice monetization like cigarettes, alcohol, etc.
  - Animats 6y ago
    It seems like what they've done is basically automate, "Give the people what they want.", as it were. Yes. Facebook profits from giving people what they want, but did not create the demand. What's scary is the quantity of demand for nuttiness. The other big feature of social media is that it brings diffuse minorities together. Before social media, outliers were unlikely to meet those with similar inclinations. Social media powered cohesion of groups from gays to Proud Boys to flat-earthers. Fox News probably influences more people, anyway. Should they be "brought under control?" I'm more worried about the trend towards censorship everywhere.
Ozzie_osman 6y ago
This is the "move fast and break things", "grow at all cost", "ask for forgiveness instead of beg for permission" culture. Facebook is the most obvious and visible example of this, but I fear it's permeated far beyond that in our industry.
- cutemonster 6y ago
  It's "walk over dead bodies"
jchw 6y ago
Note for headline-only readers: the “normalization of deviance” here refers to organizational issues, referencing the term as it was coined in regards to the events leading up to the Challenger incident. It is not, as far as I can tell, intended to be a stigmatizing remark about behavior considered “deviant.” It’s probably worth noting this since it’s easy to interpret it that way from the headline.
staticassertion 6y ago
I was just thinking the other day about what it must be like to work for Facebook on their security team (I'm in security myself). On an infosec team I have an idea of who the attackers are - there may be multiple, ranging from disgruntled employees to nation states. I think about their incentives, capabilities, and how to reduce risk. At Facebook... I'm working for the attacker. You're protecting users from some set of other attackers, of course, but like, if the idea here is "reduce harm to the user", you're working for one of those attackers. Facebook is essentially normalizing a breach. It's not really particularly different from some threat actor gaining access to a database, except that the users "consent" (if such a thing is possible - as if users understand the implications of their consent, not to mention the tracking that happens without consent). As the post notes, and as Facebook states, " Longer term, though, we expect more scraping incidents and think it’s important to . . . normalize the fact that this activity happens regularly." Of course, this makes it clear that protecting users isn't the goal, protecting the company's interests is the goal. It feels like engineers and others may be in need of a hippocratic oath - a commitment that makes it clear that, while you may be employed by some entity, you must do no harm. You have ethical responsibilities that extend beyond the scope of your employment to that entity. > Neither will I administer a poison to anybody when asked to do so, nor will I suggest such a course.
- deleted 6y ago
  [deleted]
- ryandrake 6y ago
  Brave post! Seems whenever you imply on HN that developers should follow some code of ethics, and be held responsible for the ethical impact of the systems they make, people look at you like you have a horn growing out of your head. “But what if my manager TOLD me to write that evil code??” “Hey, I’m not making the requirements, I just implement them!” According to a lot of replies you will get, we are just helpless developer drones, with no say in what we are building.
  - justapassenger 6y ago
    > Brave post! Not commenting on op’s point, but let’s be honest - saying that Facebook is evil on HN is as brave as saying that Fox News is evil on CNN.
op03 6y ago
Problem solving in NASA is a very different story compared to problem solving at Facebook. NASA hires rocket scientists to build rockets. Enough talent existed in the military working on that stuff, even before anyone started day dreaming of flying to space. Facebook on the other hand hires computer scientists, programmers and an assortment of random over-energetic mindlessly ambitious buffoons (too ignorant to fully comprehend how ignorant they are outside their domain) to work on Sociology, Political Science and Social Psych problems. What solutions can anyone expect? Many of said created problems are barely studied by domain experts at such scales. So expect solutions to always be half baked and create their own new issues for a long time, before any real breakthroughs happen. Its like waiting for Meteorologists to announce they have finally worked out how to control the path of hurricanes. Ofcourse the difference here is Meteorologists dont create and inject chaos into society. Facebook does and is well rewarded for it. Its high time they are punished.
Barrin92 6y ago
There's some real crazy quotes in that piece >"According to the newspaper, Zhang was fired for “spending too much time focused on uprooting civic fake engagement and not enough time on the priorities outlined by management.”"[...] >"Facebook’s products enabled corrupt governments to create fake followers and fake “likes,” which then triggered Facebook’s algorithms to boost their propaganda and legitimacy. According to the Guardian, when Zhang alerted >higher-ups about how this was being used by the government of Honduras, an executive told her, “I don’t think Honduras is big on people’s minds here.” Just ponder that attitude for a second and ask yourself if it can truly be said that Facebook is still in any way shape or form aligned with human flourishing. And more importantly small countries should take note. Their communication infrastructure is controlled by people who treat them like pocket change.
- yeetman21 6y ago
  If he was tasked with a job regarding domestic (US) activities, why should he be looking into Honduras? >ask yourself if it can truly be said that Facebook is still in any way shape or form aligned with human flourishing. FB contributes a lot to many open source projects such as Godot and OCP. Please stop being so dramatic
  - techbio 6y ago
    Just FYI, something like 15% of Hondurans live in the US [1], and even higher for Salvadorans, and not unrelated, the murder rate down there is extremely dramatic. [1] https://en.wikipedia.org/wiki/Honduran_diaspora https://en.wikipedia.org/wiki/Honduran_diaspora
  - DreamScatter 6y ago
    Imagine replacing Facebook with Hitler in your statements: > Hitler contributes a lot to many open source projects such as Godot and OCP. Please stop being so dramatic Sure, fine, they contribute to open source, but that does not mean they are aligned with human flourishing.
  - cutemonster 6y ago
    If you get some open source breadcrumbs, then you're fine with attacks on civilians and ethnic cleansing? (Facebook, Erdogan (in Turkey), Myanmar) "being so dramatic" you said about things like that?
loopz 6y ago
It's easy to villify, but we need to recognize the top-down management culture is doing something other than what we say. It is not servant leadership, but stealing worker responsibility putting it in the hands of others, externalizing costs, etc. The resulting culture follows from that, but is reported green upwards.
poisonborz 6y ago
For soldiers and medical personnel there are ethical guidances and law to follow. Why not make one for developers?
- Nicksil 6y ago
  Professional engineers, too.
- MeinBlutIstBlau 6y ago
  How do you create ethics for something like programming? Coding is not what needs ethics, it's management.
fallingknife 6y ago
Still don't think facebook did anything wrong. What's the difference between their recommendation algorithm and Netflix? If people want things you don't like then your beef is with them not facebook.
- cutemonster 6y ago
  Websearch for "Facebook Myanmar genocide" and "Facebook WhatsApp lynchings" -- Netflix doesn't make such things happen
mensetmanusman 6y ago
People complaining about FB but not discussing a viable alternative, or even what we have to change to enable a potential future viable alternative isn’t super helpful.
lucasnortj 6y ago
social media is cancer, do what jaron lanier says and delete it all

89 points by mitchbob 6y ago

The New Yorker should drop the "the". It's cleaner. Normalizing the amplified prejudiced attacks of perceived deviance is not the same as normalizing deviance.

jchw 6y ago

Archive link: https://archive.is/hnBQa https://archive.is/hnBQa

etaioinshrdlu 6y ago
I'm not complaining, but I'd like to point out that I think this violates HN's terms of service: "You agree to not use the Site to: email or otherwise upload any content that (i) infringes any intellectual property or other proprietary rights of any party;" It seems like sharing a link to copyright infringing content (archive.is) could be considered Contributory copyright infringement (https://en.wikipedia.org/wiki/Contributory_copyright_infringement https://en.wikipedia.org/wiki/Contributory_copyright_infring...) By analogy, how is this so different from posting a link to a pirated movie originally from Netflix? Because it's text? I think we have a case of normalization of deviance right here on HN. Illegal or stolen content is posted in the comments here all the time.
- jchw 6y ago
  Rules were made to be broken. In this case, dang themselves has commented on the matter[1]. Complaining about paywalls is off topic and discouraged, and linking archives is tolerated. I disagree with your sentiment vehemently. Firstly, law is not morality; people break the law all the time, every day, at their own discretion. It was made by humans and is not infallible. Secondly, I can’t just do a few GET requests to Netflix and get the full movie, yet this archive.is scraper has no trouble. So I guess it wasn’t very protected to begin with. The internet giveth, and the internet taketh away. They can get their better protection by not serving full articles publicly. They presumably do not because they would like to eat their cake and have it too with regards to indexing. This is hugely off topic so apologies but there is a reasonable chance I am not replying beyond this point. [1]: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989
- madars 6y ago
  Not sure what you mean here. Opening the original link in an incognito window yields full text of the article (maybe you are out of free articles, and they are counting?) Submitting articles to archive.is or archive.org serves the important function of maintaining an accurate historical record, and prevent link rot.

deleted 6y ago

[deleted]

pope_meat 6y ago

We mustn't forget the cambridge analytica shenanigans either.

deleted 6y ago

[deleted]

sneak 6y ago

I think the real problem is that a small set of companies are exercising control over what people see and read. Facebook's censorship (which includes algorithmic timelines, prioritizing some things and censoring others) is an existential danger to the wellbeing of our society. Any organization that can decide what links and text can be shared from neighbor to neighbor or from friend to friend is in a position of massive power. They even prohibit links to certain tweets in DMs.

im_down_w_otp 6y ago
Aren't their algorithms all designed to optimize for engagement? Isn't engagement just the measure of how much time you spend on and interacting with their system? Doesn't that mean that they're not deciding what to keep feeding us, but rather that we are? It seems like what they've done is basically automate, "Give the people what they want.", as it were. We produce the clickbait, we click on the clickbait, then we get fed more and more of what we click on. I tend to think that this probably boils down to rent-seeking via exploiting self-destructive human behavior/biology, so I'm not too fond of it for those reasons. But, that doesn't seem very much like Facebook censoring things as it does them feeding everyone exactly what people indicate to them with their clicks and comments they'd like to be fed. I suppose there's an interesting discussion about what lengths companies like Facebook should go to save people from themselves, but to me they're basically another form of vice monetization like cigarettes, alcohol, etc.
- Animats 6y ago
  It seems like what they've done is basically automate, "Give the people what they want.", as it were. Yes. Facebook profits from giving people what they want, but did not create the demand. What's scary is the quantity of demand for nuttiness. The other big feature of social media is that it brings diffuse minorities together. Before social media, outliers were unlikely to meet those with similar inclinations. Social media powered cohesion of groups from gays to Proud Boys to flat-earthers. Fox News probably influences more people, anyway. Should they be "brought under control?" I'm more worried about the trend towards censorship everywhere.

Ozzie_osman 6y ago

This is the "move fast and break things", "grow at all cost", "ask for forgiveness instead of beg for permission" culture. Facebook is the most obvious and visible example of this, but I fear it's permeated far beyond that in our industry.

cutemonster 6y ago
It's "walk over dead bodies"

jchw 6y ago

Note for headline-only readers: the “normalization of deviance” here refers to organizational issues, referencing the term as it was coined in regards to the events leading up to the Challenger incident. It is not, as far as I can tell, intended to be a stigmatizing remark about behavior considered “deviant.” It’s probably worth noting this since it’s easy to interpret it that way from the headline.

staticassertion 6y ago

I was just thinking the other day about what it must be like to work for Facebook on their security team (I'm in security myself). On an infosec team I have an idea of who the attackers are - there may be multiple, ranging from disgruntled employees to nation states. I think about their incentives, capabilities, and how to reduce risk. At Facebook... I'm working for the attacker. You're protecting users from some set of other attackers, of course, but like, if the idea here is "reduce harm to the user", you're working for one of those attackers. Facebook is essentially normalizing a breach. It's not really particularly different from some threat actor gaining access to a database, except that the users "consent" (if such a thing is possible - as if users understand the implications of their consent, not to mention the tracking that happens without consent). As the post notes, and as Facebook states, " Longer term, though, we expect more scraping incidents and think it’s important to . . . normalize the fact that this activity happens regularly." Of course, this makes it clear that protecting users isn't the goal, protecting the company's interests is the goal. It feels like engineers and others may be in need of a hippocratic oath - a commitment that makes it clear that, while you may be employed by some entity, you must do no harm. You have ethical responsibilities that extend beyond the scope of your employment to that entity. > Neither will I administer a poison to anybody when asked to do so, nor will I suggest such a course.

deleted 6y ago
[deleted]
ryandrake 6y ago
Brave post! Seems whenever you imply on HN that developers should follow some code of ethics, and be held responsible for the ethical impact of the systems they make, people look at you like you have a horn growing out of your head. “But what if my manager TOLD me to write that evil code??” “Hey, I’m not making the requirements, I just implement them!” According to a lot of replies you will get, we are just helpless developer drones, with no say in what we are building.
- justapassenger 6y ago
  > Brave post! Not commenting on op’s point, but let’s be honest - saying that Facebook is evil on HN is as brave as saying that Fox News is evil on CNN.

op03 6y ago

Problem solving in NASA is a very different story compared to problem solving at Facebook. NASA hires rocket scientists to build rockets. Enough talent existed in the military working on that stuff, even before anyone started day dreaming of flying to space. Facebook on the other hand hires computer scientists, programmers and an assortment of random over-energetic mindlessly ambitious buffoons (too ignorant to fully comprehend how ignorant they are outside their domain) to work on Sociology, Political Science and Social Psych problems. What solutions can anyone expect? Many of said created problems are barely studied by domain experts at such scales. So expect solutions to always be half baked and create their own new issues for a long time, before any real breakthroughs happen. Its like waiting for Meteorologists to announce they have finally worked out how to control the path of hurricanes. Ofcourse the difference here is Meteorologists dont create and inject chaos into society. Facebook does and is well rewarded for it. Its high time they are punished.

Barrin92 6y ago

There's some real crazy quotes in that piece >"According to the newspaper, Zhang was fired for “spending too much time focused on uprooting civic fake engagement and not enough time on the priorities outlined by management.”"[...] >"Facebook’s products enabled corrupt governments to create fake followers and fake “likes,” which then triggered Facebook’s algorithms to boost their propaganda and legitimacy. According to the Guardian, when Zhang alerted >higher-ups about how this was being used by the government of Honduras, an executive told her, “I don’t think Honduras is big on people’s minds here.” Just ponder that attitude for a second and ask yourself if it can truly be said that Facebook is still in any way shape or form aligned with human flourishing. And more importantly small countries should take note. Their communication infrastructure is controlled by people who treat them like pocket change.

yeetman21 6y ago
If he was tasked with a job regarding domestic (US) activities, why should he be looking into Honduras? >ask yourself if it can truly be said that Facebook is still in any way shape or form aligned with human flourishing. FB contributes a lot to many open source projects such as Godot and OCP. Please stop being so dramatic
- techbio 6y ago
  Just FYI, something like 15% of Hondurans live in the US [1], and even higher for Salvadorans, and not unrelated, the murder rate down there is extremely dramatic. [1] https://en.wikipedia.org/wiki/Honduran_diaspora https://en.wikipedia.org/wiki/Honduran_diaspora
- DreamScatter 6y ago
  Imagine replacing Facebook with Hitler in your statements: > Hitler contributes a lot to many open source projects such as Godot and OCP. Please stop being so dramatic Sure, fine, they contribute to open source, but that does not mean they are aligned with human flourishing.
- cutemonster 6y ago
  If you get some open source breadcrumbs, then you're fine with attacks on civilians and ethnic cleansing? (Facebook, Erdogan (in Turkey), Myanmar) "being so dramatic" you said about things like that?

loopz 6y ago

It's easy to villify, but we need to recognize the top-down management culture is doing something other than what we say. It is not servant leadership, but stealing worker responsibility putting it in the hands of others, externalizing costs, etc. The resulting culture follows from that, but is reported green upwards.

poisonborz 6y ago

For soldiers and medical personnel there are ethical guidances and law to follow. Why not make one for developers?

Nicksil 6y ago
Professional engineers, too.
MeinBlutIstBlau 6y ago
How do you create ethics for something like programming? Coding is not what needs ethics, it's management.

fallingknife 6y ago

Still don't think facebook did anything wrong. What's the difference between their recommendation algorithm and Netflix? If people want things you don't like then your beef is with them not facebook.

cutemonster 6y ago
Websearch for "Facebook Myanmar genocide" and "Facebook WhatsApp lynchings" -- Netflix doesn't make such things happen

mensetmanusman 6y ago

People complaining about FB but not discussing a viable alternative, or even what we have to change to enable a potential future viable alternative isn’t super helpful.

lucasnortj 6y ago

social media is cancer, do what jaron lanier says and delete it all