4 ms·

What were we (technologists) thinking when we built these toxic social media platforms and algorithms? Tristan Harris gave a compelling talk a couple of years

What were we (technologists) thinking when we built these toxic social media platforms and algorithms? Tristan Harris gave a compelling talk a couple of years ago that dug into this question without assigning moral blame on us. [0] He's identified several principles, probably familiar to most of us and seemingly innocent, that he says have led us to where we are now: * Give users what they 'want' * Disrupt everything * Technology is neutral * Who are we to choose what our userbase does with our platform? * Value growth at all costs * Design our interfaces to convert users * Obsess over metrics Tristan is part of the Center for Humane Technology, a nonprofit that, in my opinion, deserves more of our attention. [0]: https://www.youtube.com/watch?v=YQh2FQ7MZdA&t=22m51s https://www.youtube.com/watch?v=YQh2FQ7MZdA&t=22m51s ; timestamped for the specific question, but the whole talk is worth a watch

emilsedgh 5y ago
> What were we (technologists) thinking when we built these toxic social media platforms and algorithms? If OKR A reaches B, we'd be valued at C, which would mean my stock option would be worth D. That's what everyone was, is, and will be thinking.
- iamdamian 5y ago
  I think that's part of it. But surely if a team was handed an OKR to 'build ship control software that has a 50% chance of sinking a ship' or 'drive up teen mental health problems by 50%', few people would have gone along with it. I think the trick is that most people don't want to feel like the bad guy. They have to be able to convince themselves that what they're doing isn't harmful in order to avoid internal moral conflicts. The principles we hear about tech & neutrality give us enough leeway to feel okay about our actions even if the outcomes are toxic. (Which, to your point, lets us cash in when we hit our targets.)
  - exdsq 5y ago
    Also there’s a lot of hindsight here. What might look bad now is likely an emergent property of what at the time wasn’t bad at all (show you news sources people care about, etc). I can totally see people who hate what Facebook has become having no problems working on it a decade ago.
ComradePhil 5y ago
shadowgovt 5y ago
We were mostly thinking that the stuff that came before was hard to use and had a steep learning curve. And being decentralized, it was way harder to iterate on than a wholly-owned service. Twitter (the technology) is okay. It turns out people work poorly when you put them all in the same room and let them shout at each other.
- unsupp0rted 5y ago
  Yours is a point that bears repeating: Twitter is fine (not that I use it). What's not fine is people: they prefer to be in a place in which they can yell at each other about how much more virtuous they are than their villains. It feels good to vilify somebody. It feels good to verbally punch a nazi. I don't know what we can replace Twitter with that will make people less interested in public displays of virtue as they stand up to their ideological enemies.
  - shadowgovt 5y ago
    One option is the thing we used to have: private displays of virtue in distributed systems. When Reddit is hosting racists, it's easy to blame Reddit (and, more specifically, easy to put pressure on them to deny those users a forum on Reddit's servers). When such content is on a distributed, federated system, it's up to individual service providers how much of a tolerance they'll have for speech others consider completely unacceptable. Some will choose not to host it, some will perhaps choose to host it (and others can choose to peer with them or not)... But in general, there will be more variety of opinions represented in what is "acceptable behavior" when there are many distributed systems instead of a few centralized ones. This does, of course, increase the complexity of accessing them... ... honestly, this probably mirrors societal development as a whole. In a town of frontiersmen, you barely have enough time to worry about what goes on in your neighbor's house. In a city, where you can hear your neighbor through the walls (and have deeply-intertwined interdependency networks through which pressure can be applied), you both have more reason to care and more tools to pressure someone to conform. ... all of that having been said, to take a step back from the mechanical and consider the philosophical, I'm not sure it's an inherently virtuous thing that my frontier neighbor could be out there doing Texas Chainsaw Massacre stuff and there's little structural pressure that can be applied to get him to cut it out...
cloutchaser 5y ago
> Tristan is part of the Center for Humane Technology, a nonprofit that, in my opinion, deserves more of our attention. You mean the guy who has a Netflix documentary AND has been on Joe rogan? I literally can’t think of bigger audience coverage in 2022 than those two
- exdsq 5y ago
  I imagine you’ve thought about this too, based on your HN username! :P
ironmagma 5y ago
“Wow, these are cool technologies to be working with, and problems at scale!” This is why I intentionally don’t give a crap about things being “at scale” or the technology stack in the abstract anymore. It has to be coupled with something else that makes the job worthwhile.
dan-robertson 5y ago
Didn’t a lot of it seem like a good idea at the time? When you look at early Facebook, I think people got a lot out of it. Even Facebook today can be a way for older people (say those who weren’t students when it was new) to reconnect with old acquaintances or lost friends. If you try to imagine early Twitter it seemed like something people liked. Surely microblogging was easier to get into than microblogging. And the centralisation made it easier for friends to follow each other. I don’t think it was particularly obvious how it might go wrong. For example there wasn’t much of an algorithm for a while (feeds were chronological) except for a ‘trending’ section. But HN has a trending-like section – the front page – and it seems to work out ok. Basically, I claim that ‘we’ thought they were a good idea and that they fitted in with the optimistic world-connecting zeitgeist of the early internet. I don’t really know what else people were thinking (well obviously there was some level of ‘we think these numbers correspond to a good service for our users and investors so we want to make them go up’ and ‘we sell ads to make money’) This isn’t to say that the results were good but I think the efforts were generally serious and well-intentioned. If you look at something which is ‘anti-Facebook’, the GDPR, I think a similar thing can be said: the law was carefully written with deliberate goals and it’s authors seemed to seriously believe it would achieve them. I think we’ll find it didn’t work as well as intended but I hope I won’t browse HN in 2032 and see someone saying ‘what were lawmakers thinking when they drafted these terrible data-protection laws?’
- ShamelessC 5y ago
  A lot of it seemed like a pretty bad idea at the time. Capitalists built it anyway. "We" had little say in the matter aside from the ability to quit in protest to likely be replaced.
  - dan-robertson 5y ago
    I think this must be the root of our disagreement because I basically don’t believe this. I believe neither that it seemed like a bad idea at the time nor that employees could do little, though I sympathise with the many reasons employees do do little.
pmoriarty 5y ago
* Spying on users * Advertising * Massive inequality
specialist 5y ago
Harris is well intentioned, perhaps, but those principles are non sequiturs. Preferential attachment creates winner-takes-all outcomes. aka Power law distribution of attention. It's something like a law of nature. The stuff Harris lists are second order effects, at best. Twitter and others did two things: First, they reduced transaction costs. Discovery, foraging, communication. All the costs fell thru the floor. Every one, including me, thought this was a great idea, and expected great outcomes. Not until Clay Shirky do I even recall a concern or criticism about this. Second, recommenders accelerated winner-takes-all. Given the high stakes and shortened shelf lifes, gaming the system became all out war. @jack wrote: > centralizing discovery and identity into corporations really damaged the internet. The only part right here is forfeiting a core pillar of civil society -- authenicity (identity) -- to a paper clip maximizing corporation that has no regard for or responsibility to that society. We let the fox guard the hen house.
- deleted 5y ago
  [deleted]
notacoward 5y ago
I'm surprised that "information wants to be free" isn't on this list. It used to be a common mantra. Now it sounds quite quaint. What a lot of critics of Twitter, Facebook, etc. seem to miss is that a lot of their growth was related to the idea that only good could come from people knowing more about each other. People criticized Facebook for not making their social graph available to anyone who could sign up for an API key. The result, of course, was Cambridge Analytica. We've learned about the dangers since then, of course, but back in the early oughts it was an idea with significant traction among the very same crowd who can't keep themselves from commenting "Twitbook is a cancer on society" no matter how relevant it is (or isn't) to the story at hand today. Intellectual fashions change just like any other.

emilsedgh 5y ago

> What were we (technologists) thinking when we built these toxic social media platforms and algorithms? If OKR A reaches B, we'd be valued at C, which would mean my stock option would be worth D. That's what everyone was, is, and will be thinking.

iamdamian 5y ago
I think that's part of it. But surely if a team was handed an OKR to 'build ship control software that has a 50% chance of sinking a ship' or 'drive up teen mental health problems by 50%', few people would have gone along with it. I think the trick is that most people don't want to feel like the bad guy. They have to be able to convince themselves that what they're doing isn't harmful in order to avoid internal moral conflicts. The principles we hear about tech & neutrality give us enough leeway to feel okay about our actions even if the outcomes are toxic. (Which, to your point, lets us cash in when we hit our targets.)
- exdsq 5y ago
  Also there’s a lot of hindsight here. What might look bad now is likely an emergent property of what at the time wasn’t bad at all (show you news sources people care about, etc). I can totally see people who hate what Facebook has become having no problems working on it a decade ago.

ComradePhil 5y ago

shadowgovt 5y ago

We were mostly thinking that the stuff that came before was hard to use and had a steep learning curve. And being decentralized, it was way harder to iterate on than a wholly-owned service. Twitter (the technology) is okay. It turns out people work poorly when you put them all in the same room and let them shout at each other.

unsupp0rted 5y ago
Yours is a point that bears repeating: Twitter is fine (not that I use it). What's not fine is people: they prefer to be in a place in which they can yell at each other about how much more virtuous they are than their villains. It feels good to vilify somebody. It feels good to verbally punch a nazi. I don't know what we can replace Twitter with that will make people less interested in public displays of virtue as they stand up to their ideological enemies.
- shadowgovt 5y ago
  One option is the thing we used to have: private displays of virtue in distributed systems. When Reddit is hosting racists, it's easy to blame Reddit (and, more specifically, easy to put pressure on them to deny those users a forum on Reddit's servers). When such content is on a distributed, federated system, it's up to individual service providers how much of a tolerance they'll have for speech others consider completely unacceptable. Some will choose not to host it, some will perhaps choose to host it (and others can choose to peer with them or not)... But in general, there will be more variety of opinions represented in what is "acceptable behavior" when there are many distributed systems instead of a few centralized ones. This does, of course, increase the complexity of accessing them... ... honestly, this probably mirrors societal development as a whole. In a town of frontiersmen, you barely have enough time to worry about what goes on in your neighbor's house. In a city, where you can hear your neighbor through the walls (and have deeply-intertwined interdependency networks through which pressure can be applied), you both have more reason to care and more tools to pressure someone to conform. ... all of that having been said, to take a step back from the mechanical and consider the philosophical, I'm not sure it's an inherently virtuous thing that my frontier neighbor could be out there doing Texas Chainsaw Massacre stuff and there's little structural pressure that can be applied to get him to cut it out...

cloutchaser 5y ago

> Tristan is part of the Center for Humane Technology, a nonprofit that, in my opinion, deserves more of our attention. You mean the guy who has a Netflix documentary AND has been on Joe rogan? I literally can’t think of bigger audience coverage in 2022 than those two

exdsq 5y ago
I imagine you’ve thought about this too, based on your HN username! :P

ironmagma 5y ago

“Wow, these are cool technologies to be working with, and problems at scale!” This is why I intentionally don’t give a crap about things being “at scale” or the technology stack in the abstract anymore. It has to be coupled with something else that makes the job worthwhile.

dan-robertson 5y ago

Didn’t a lot of it seem like a good idea at the time? When you look at early Facebook, I think people got a lot out of it. Even Facebook today can be a way for older people (say those who weren’t students when it was new) to reconnect with old acquaintances or lost friends. If you try to imagine early Twitter it seemed like something people liked. Surely microblogging was easier to get into than microblogging. And the centralisation made it easier for friends to follow each other. I don’t think it was particularly obvious how it might go wrong. For example there wasn’t much of an algorithm for a while (feeds were chronological) except for a ‘trending’ section. But HN has a trending-like section – the front page – and it seems to work out ok. Basically, I claim that ‘we’ thought they were a good idea and that they fitted in with the optimistic world-connecting zeitgeist of the early internet. I don’t really know what else people were thinking (well obviously there was some level of ‘we think these numbers correspond to a good service for our users and investors so we want to make them go up’ and ‘we sell ads to make money’) This isn’t to say that the results were good but I think the efforts were generally serious and well-intentioned. If you look at something which is ‘anti-Facebook’, the GDPR, I think a similar thing can be said: the law was carefully written with deliberate goals and it’s authors seemed to seriously believe it would achieve them. I think we’ll find it didn’t work as well as intended but I hope I won’t browse HN in 2032 and see someone saying ‘what were lawmakers thinking when they drafted these terrible data-protection laws?’

ShamelessC 5y ago
A lot of it seemed like a pretty bad idea at the time. Capitalists built it anyway. "We" had little say in the matter aside from the ability to quit in protest to likely be replaced.
- dan-robertson 5y ago
  I think this must be the root of our disagreement because I basically don’t believe this. I believe neither that it seemed like a bad idea at the time nor that employees could do little, though I sympathise with the many reasons employees do do little.

pmoriarty 5y ago

* Spying on users * Advertising * Massive inequality

specialist 5y ago

Harris is well intentioned, perhaps, but those principles are non sequiturs. Preferential attachment creates winner-takes-all outcomes. aka Power law distribution of attention. It's something like a law of nature. The stuff Harris lists are second order effects, at best. Twitter and others did two things: First, they reduced transaction costs. Discovery, foraging, communication. All the costs fell thru the floor. Every one, including me, thought this was a great idea, and expected great outcomes. Not until Clay Shirky do I even recall a concern or criticism about this. Second, recommenders accelerated winner-takes-all. Given the high stakes and shortened shelf lifes, gaming the system became all out war. @jack wrote: > centralizing discovery and identity into corporations really damaged the internet. The only part right here is forfeiting a core pillar of civil society -- authenicity (identity) -- to a paper clip maximizing corporation that has no regard for or responsibility to that society. We let the fox guard the hen house.

deleted 5y ago
[deleted]

notacoward 5y ago

I'm surprised that "information wants to be free" isn't on this list. It used to be a common mantra. Now it sounds quite quaint. What a lot of critics of Twitter, Facebook, etc. seem to miss is that a lot of their growth was related to the idea that only good could come from people knowing more about each other. People criticized Facebook for not making their social graph available to anyone who could sign up for an API key. The result, of course, was Cambridge Analytica. We've learned about the dangers since then, of course, but back in the early oughts it was an idea with significant traction among the very same crowd who can't keep themselves from commenting "Twitbook is a cancer on society" no matter how relevant it is (or isn't) to the story at hand today. Intellectual fashions change just like any other.