10 ms·

Manipulating Human Psychology To Turn Users Into Addicts

313 points by coloneltcb 9y ago

wppick 9y ago
Stories like this might appear to hurt Facebook and similar companies, but they actually help create a barrier to entry for the "next Facebook" to not be able to use the same strategy.
- Devolver 9y ago
  If my writing makes it less likely that some would-be Facebook competitor can use mass-scale psychological manipulation to get an edge, I can live with that.
- chroem- 9y ago
  Is that necessarily a bad thing? I feel like social media has destroyed an enormous amount of value by making it harder to have authentic relationships.
- Danihan 9y ago
  I highly doubt a blog post will have that effect.
  - JoshMnem 9y ago
    Blog posts start conversations.
  - Devolver 9y ago
    Me too. But you know what they say about the pen and the sword? Maybe we need to update the saying: “the keyboard and a content database can be mightier than the predator drone” ?
- oiuplk 9y ago
  The new Facebook may be kinder and gentler and not designed to exploit you. The killer feature may be that it does not exploit your human flaws. People are always surprised what new needs we have for technology. Now we need software and services that respect us as humans.
- opportune 9y ago
  In other news, it's really hard for the "next British East India Company" to enter the Chinese Opium market
spaceribs 9y ago
If you can name a successful product which does not manipulate users to continue using that product, then it's not a product: * I've been manipulated to bring an umbrella because I don't like to get rained on. * I've been manipulated into liking broadband because it provides the internet faster than dialup. * I've been manipulated into owning a car because it's a fast convenient mode of transportation. This is the basics of products and services that provide value, this isn't magical or new, and this book is probably being more honest than most industries are. The point the book was making is that if you don't want a product to be flash in the pan garbage, follow some simple guidelines which ensure that your service is actually providing some sort of value to end users, or your end users will eventually leave and find better things to do with their time and resources (Unless you're selling heroin I guess). I don't consider Facebook sending me an email when I get a personal message to be exploitative, but I find no value at all when they send me something every time one of my friends fart. I specifically avoid interacting with those "hey someone did something" emails because they are annoying and make me want to stop being on facebook altogether. That's the result of abusing the hook model, users go away because it's annoying and unproductive.
- maxxxxx 9y ago
  Manipulation had always been there but what's new is that the best minds of a generation are working on exploiting human weaknesses with the help of technology to a new level. Same for surveillance. It was always there but now it can be done on an absolutely unprecedented level.
- dj-wonk 9y ago
  In standard / common usage, selling and marketing an umbrella to someone is not "manipulation". Your definition of "manipulate" is not what most people mean in these contexts. Your use of "manipulate" in such a way muddies the waters and distracts from the core issue. Putting word choice aside, the core issue here is _not_ about people's preferences (e.g. many people don't like to get rained on). Rather, the core issue is about unknown and unwelcome influences, particularly influences that operate on parts of our minds that are hard to combat due to human nature. Facebook's manipulation of our attention is conceptually related to how casinos trick people into staying longer than they would otherwise.
ttoinou 9y ago
The original article seems fine. I don't see how it was immoral. Seemed more like "amoral" https://medium.com/the-mission/the-morality-of-manipulation-c3115fb2bb3d https://medium.com/the-mission/the-morality-of-manipulation-... In the meantime, users will have to judge the yet unknown consequences for themselves, while creators will have to live with the moral repercussions of how they spend their professional lives. https://en.wikipedia.org/wiki/Amorality https://en.wikipedia.org/wiki/Amorality
- scythe 9y ago
  People are amoral, acts are immoral. An act by an amoral person can certainly be immoral.
dj-wonk 9y ago
I am glad to see more people caring about long-term effects of their use of social software. More of us are mindful in how we engage with technology. Now that I've said the positive, let me talk about the negatives. I bet things will get worse before they get better. At present, the commercial pressures on social media platforms to manipulate "users" dwarf human pushback against these "dark art" techniques. I don't see any easy solutions. As with many collective challenges, we need increased awareness to effect change. This is arguably a political problem; we need social organizing as a counterbalancing force.
- TheOtherHobbes 9y ago
  We don't need "increased awareness", we need a social decision system (aka "an economy") that rewards positive social outcomes and disincentivises negative social outcomes. At the moment we have the opposite. No one cares how you make your first billion or two, or how people suffer as a side effect. They only care that you have a billion or two. Toxic side-effects remain invisible and uncounted. There's a certain irony in attacking EA for building a game economy that's closer to the effort/reward profile of the real economy - for both players and owners - than any other game economy to date.
  - dj-wonk 9y ago
    First, a question for you: What values would you like your "social decision system" to embody? Second, I advocate for increased awareness because awareness is the first step towards rational thought. Many psychological manipulation techniques work best on the irrational parts of our brains, so often a useful first step is to engage the rational mind. That said, I'm not claiming awareness guarantees systemic change.
  - AckSyn 9y ago
    > rewards positive social outcomes and disincentivises negative social outcomes Therein lies the problem. Who is to decide what is a social positive or negative among some of the more nuanced and innocuous bits of society? This is not a road we should travel down any further than we have already.
  - 9y ago
ncantelmo 9y ago
My knee-jerk reaction to the book Hooked (which I've somewhat shamefully avoided reading out of a sort of premeditated disgust) was based on the general arguments laid out in this blog post. I believe the tech world has a huge problem on its hands here, and that problem doesn't get anywhere near the attention it deserves. Large swaths of the population are turning into zombies and tech is the facilitator. Try looking at the drivers turning through a busy intersection some time. It's obscene how many are staring down at their phones rather than at whatever might be in the road around the corner. That said, I don't believe the problem is simply that certain products remind or encourage people to use them at times. Rather, it's that there is a class of truly addictive products that don't do much (if anything) to help people manage the resulting dependency. And that approach is celebrated far more than it's scrutinized. Coincidentally, I wrote the following on Twitter yesterday: "Any potentially addictive software/service should offer a way for users to lock themselves out of their own account for a predetermined set of hours." [0] It was meant in my usual tongue-in-cheek style, but I do think that a solution for breaking tech product dependency is sorely needed and hope this discussion gains some steam. [0] https://twitter.com/ncantelmo/status/930876372620374017 https://twitter.com/ncantelmo/status/930876372620374017
- cokeandsympathy 9y ago
  You might want to check out some of the criticism advanced by people such as Tristan Harris on the tension between the user behaviors that tech companies optimize their product design, and the goals users hope to accomplish through their use. I do in some ways support Nir in that we should learn how to harness these types of technologies for our own good, to 'supercharge' our ability to accomplish what we want, but I do not trust any company dependent on ad-revenue (or backed by Wall Street in general) to help with that. I'd also recommend checking out The Attention Merchants by Tim Wu and Addiction by Design by Natasha Schull for background on the history of the commodification of attention and of the
- deleted 9y ago
  [deleted]
- sjellis 9y ago
  "Any potentially addictive software/service should offer a way for users to lock themselves out of their own account for a predetermined set of hours." It's really noticeable that Hacker News provides exactly this feature, and absolutely no other site that I've seen does.
acchow 9y ago
This is a fairly similar argument to saying drug dealers are bad because they profit off your addiction. I don't know if I agree or disagree with that, but at what point do we actually acknowledge agency?
- scythe 9y ago
  Drug dealers don't have intricate control over the effects of the drugs they're selling. I imagine if synthetic drugs were legal in general rather than specific compounds you might see similar problems pop up (e.g. 3-fluoroethamphetamine, a compound screened in rats to be maximally addictive) but it's reasonable to advocate for a drug market that avoids this.
  - emmab 9y ago
    > Drug dealers don't have intricate control over the effects of the drugs they're selling. yet
- intended 9y ago
  I think Agency is always understood to be a part of the equation - and the people with the agency are not the issue. By analogy - If the population of a city was exposed intentionally, to a cold virus - it is not the people with natural resistance, we are concerned about. We are glad those people are not affected, and know they are not part of the % affected by the virus - and then we focus on the people affected by it. --- If I were to engage with your point in the spirit it is intended, there are several issues - 1) Tech recruits the best of the best. IT pays for the smartest people in the world (and this is not hyperbole), to get past agency. This is not hard core drug dealer style sale of addiction. This is candy crush, happy smiley imagery, "just one more game", style behavioral addiction. Most people don't even know they are being addicted/affected 2) This is all at internet scale. Just on a probabilistic level, the casualties/infection is going to vastly outperform personal agency. 3) The network effects of having more people doing this, at this scale, pushes it to crisis levels very quickly. It wouldn't matter what the few immune or capable can do.
m3kw9 9y ago
When max profits means max attention from users this will naturally get public companies like FB to craft products this way
denkmoon 9y ago
The thin end of the wedge here is advertising. It is deliberately manipulating people using psychological insights, often targeting weaknesses to create a recurrent cash flow. Yet the entertainment and content-creating side of the internet relies very explicitly on ad revenue. Are creators that rely on revenue generated by manipulating human psychology morally bankrupt?
- cyberpunk0 9y ago
  Yes they are completely morally bankrupt
- SomeStupidPoint 9y ago
  They invented a whole language to paper over the fact that their entire field is weaponized psychology employed against their customers. Do a noun for noun substitution to unpack a marketing meeting, and you'll want to puke. Frankly, it's shameful for any company that says they care about their customers to employ marketing psychology tactics. Marketing and advertising are (in their modern, virulent form) simply an attempt to hack viewers to shift the locus of control from the customer to the business. (Other objectionable things about tech have this kind of behavior, too.) Frankly, it's just disgusting and I can't blame customers for really viewing companies negatively based on their advertising practices. (If all that wasn't bad enough, online advertising often comes with literal malware ontop.)
diego_moita 9y ago
When you think about technology critics none seems more relevant today than Aldous Huxley. The traditional Hollywood criticism on technology goes along the lines of Frankenstein, Metropolis, 2001 A Space Odyssey, Terminator and Matrix: technology will control us. I find it dumb, frankly. Then there is the political criticism along the lines of George Orwell and some Luddites: technology is a tool for control and domination either by governments or evil capitalists. It is not totally wrong (e.g.: China, Turkey) but not the exact true sometimes. But the Huxley criticism is, for me, the most interesting: technology will entertain us to stupidity, apathy and alienation. This seems to be very much the case in here.
- zitterbewegung 9y ago
  We are in a society that has the distractions of Huxleys world with the surveliance of Orwell. The only thing different is the fact that no one expected our computers to get so small . That we would stare at screens that watch us every hour of every day. The anime psycho pass is much more relevant .
  - chiefalchemist 9y ago
    I think it's easy to get hung up on the means (that each authored). But as we know, the prediction business is a tough one. For example, Orwell (in 1984) focused on the filtering, manipulation, constriction and reconstruction of information in order to continuouly redraw the history / truth. But today the truth is something that's often just as fuzzy because we're overwhelmed with too much information and too many distractions. Different means, same ends (read: confused and manipulated proles). But it's the ends that matter. Both Huxley and Orwell have warned us. The irony is, at this point, too few are paying attention.
    - WalterBright 9y ago
      The truth has always been fuzzy/slippery. There never was a golden age of unbiased nothing-but-the-correct-truth journalism. The only change now is it's a little more obvious.
zaptheimpaler 9y ago
IMO tech has simply become the dominant form of media. People who would earlier spend 3 hours/day watching TV might now spend 4 hours/day on Netflix/FB/Twitter etc. The fundamental problem is not tech, it is the incentives of those who pay for the media we consume. Advertisers pay for media and it is in their interest to make the medium as addictive as possible. And when there is enough money to be made as an employee or shareholder of ad companies like FB/Google, people look the other way as long as they benefit. The root problem is simple greed, and its a very old problem. Things are not going to change anytime soon as long as people are able to rationalize doing a small amount of evil for enough money/prestige. (not to say I wouldn't do the same) Thats why these discussions seem pointless to me. Ultimately the people paying/investing in a company decide how a company operates, and they do not give a f* about the next article telling us how bad the product is. Even ordinary shareholders will happily buy FB stock if its projected to go up, nevermind why that is. When you operate within an economic system that is built solely to optimize revenue... it optimizes revenue, at all social costs.
- snowpanda 9y ago
  I agree with you, but I think it's more than just greed. These companies consist of many employees. Depending on how you look at it, all those employees are contributing to the problem indirectly by working for the company. So one question that comes to mind is are these employees greedy too? What about the interns that get minimum wage? Are they not simply trying to make a living? So I think the first part of your comment is more accurate in the sense that people rationalize things. In that regard, I think we should look at schools emphasizing values in terms of work ethics. When I was in college I took a business ethics class. I realized that was the first time in my entire education that I was learning things about anything even remotely related to this subject, not to mention, the class was optional. That's a sad thing to happen in only the last years before entering the permanent work force.
  - zaptheimpaler 9y ago
    The fact is if someone works for one of those companies, they are contributing to the problem. Maybe out of greed, maybe necessity, maybe ignorance. Moral judgements aside, it really doesn't matter why as long as they choose to work there - the company still benefits and the cycle still continues. I'm sure teaching ethics would help.. but then again I don't think all the recent revelations about FB/Google/Twitter have done anything to make hiring more difficult for them. At most they pay 10-20% more and get the people they need. Parts of wall street are unethical and yet they get all the employees they need given the right salary.
a-guest 9y ago
And then today in YC a blog about "Startup Ideas", there is a suggestion for a "Social Network for Children". https://blog.ycombinator.com/13-startup-ideas/ https://blog.ycombinator.com/13-startup-ideas/ Please, no.
Blazespinnaker 9y ago
Gamification can provide value. Stack overflow is all about addiction and value. The blog title was manipulative in its hyperbole, btw.
- ben_w 9y ago
  I wouldn’t call stackoverflow addictive - if anything I use it too little! - but I do have that problem with some of the linked network sites on stackexchange. “Why yes, who would win in a fight between Aquaman and The Flash is more interesting than why the third argument of the ExampleFooAPI constructor doesn’t behave itself!” (I’m glad adblock lets me hide that block. Really helped me avoid those distractions).
bryanrasmussen 9y ago
There's a thing in essays where the author signifies cutting through the bull with some earthy and down-home talk, but I think this sentence "It’s too soon to tell, but it ain’t look so pretty from here." is a step too far.
- bryanrasmussen 9y ago
  actually maybe the article just needs an editor or at least a proofreader.
  - Devolver 9y ago
    Chose to use this grammatically broken slang very consciously.
- creo 9y ago
  It's not too far. Maybe You don't feel effects of antisocial media, but i do. Few years ago if friends wanted to go swimming. They were calling each other, there was human interaction. Now it's event on facebook. If You dont have favebook you're basically not a friend anymore, because noone is going to call You and invite. This is the effect of social media on our civilisation. It's like in '60 doctors were recommending smoking as stress reliever and everyone agreed until they knew better. http://fortune.com/2016/06/22/loneliness-is-a-modern-day-epidemic/ http://fortune.com/2016/06/22/loneliness-is-a-modern-day-epi... https://www.cbsnews.com/news/many-americans-are-lonely-survey-finds/ https://www.cbsnews.com/news/many-americans-are-lonely-surve...
bryanrasmussen 9y ago
I was going to clap for this story, the first time ever (despite my misgivings regarding grammar and spelling - I'm sure I've made errors in this post, but none as egregious as the ones I've found in the essay) but anyway it turns out to clap on medium I need to create an account. I decided to forgo the opportunity out of respect for the author's evident wishes.
- hohenheim 9y ago
  Ironically, clapping, is the very device Medium uses to keep writers writing, the same stimuli that releases dopamine in other platforms. And yet, the author decided to write his article on the very site that uses techniques Nir talks about to promote engagement.
  - Devolver 9y ago
    I agree. There is a contradiction here. And I am nearly as uncomfortable with centralized publishing platforms as I am with mass scale behavior design for profit. And yet. I originally published on my own site and it didn’t get much traction. But when I edited it substantially and published as a direct response to Nir’s post, it reached 30,000 people, 6000 of whom finished the whole thing...in less than 24 hours. Since the goal for me is spreading the message and fueling the converstion, I gotta go where distribution follows.
    - bryanrasmussen 9y ago
      hmm, sounds like that distribution thing is highly addictive /I kid
      - Devolver 9y ago
        I’d be lying if I said that reaching an audience and seeing my post generate (mostly) positive, thoughtful discussion didn’t feel good. But that feeling (for me, at least) is closer to gratitude and hopefulness than it is to ego gratification. The claps and notifications of retweets are, however, quite addicting, in the truest sense of the word. I find myself checking twitter more than is reasonable to see where it’s going. And while there is some upside there (connecting with and engaging in conversation with like-minded peeps), the frequency of my checking is highly disproportionate to that Value. Ugh.
CodeWriter23 9y ago
Nir Eyal didn’t invent anything. He simply adapted mass media techniques to the App era.
PeOe 9y ago
I´ve read a book recently called "Hooked" by Nir Eyal. https://www.amazon.de/Hooked-How-Build-Habit-Forming-Products/dp/0241184835/ref=sr_1_1?ie=UTF8&qid=1510910504&sr=8-1&keywords=nir+eyal+hooked https://www.amazon.de/Hooked-How-Build-Habit-Forming-Product... It´s exactly about this topic. He also gives quite a few examples of the achievements of Apple and so on. Really interesting.
- placebo 9y ago
  The entire post is addressed to him, and not in a positive way. I tend to agree...
yodsanklai 9y ago
For me, the main problem is that I need my computer for what I consider "legit" activities such as working or communicating with my friends. Starting from there, it's easy to get trapped into other "toxic" activities such as facebook or news. I'd like to be able to just put the computer aside for a few days but it's almost impossible.
deleted 9y ago
[deleted]
uoaei 9y ago
The thing is, it doesn't seem like much of this is even intentional. A/B tests (probably inspired by some sort of thought experiment re: 'getting more engagement') tell a story that says you have more users if you do this thing, when that thing just happens to be addictive.
- randomsearch 9y ago
  I used to think that, until Sean Parker recently stated it was a conscious, deliberate, decision to exploit people.
westmeal 9y ago
I've never agreed with an article quite as much as this one.
- xkcdef 9y ago
  Also, I would like to point out three things to folks who are asking "What about personal responsibility? What about agency?" here and on the article comments. Facebook's shadow profiles take away any 'agency' and 'personal responsibility' out of the equation. While I don't yet know Facebook using shadow profiles to make someone's life worse off, their penchant for saying "Screw it" to any ethical considerations makes me feel its only a matter of time before even that crutch goes away. Secondly, there is the complexity. You cannot take personal responsibility for something that you don't fully comprehend. At the same time, we are at a point where AI produces data insights that even the AI builders cannot reason about. For example, how does someone take responsibility for clicking the Like button on a post and then finding out that most of the people who liked the post were (say) closet anarchists which was then correlated with them being the best targets for some kind of subversive advertising from (say) the NRA? Thirdly, there is the asymmetry involved in "moderate use". It is possible for Facebook to create "automated notifications" implemented at a very tiny cost to them in terms of their resources/profits while the effort needed on your part to make sure you don't turn into a Pavlovian "notifications" dog comes at an extremely high time cost to you.

313 points by coloneltcb 9y ago

Stories like this might appear to hurt Facebook and similar companies, but they actually help create a barrier to entry for the "next Facebook" to not be able to use the same strategy.

Devolver 9y ago
If my writing makes it less likely that some would-be Facebook competitor can use mass-scale psychological manipulation to get an edge, I can live with that.
chroem- 9y ago
Is that necessarily a bad thing? I feel like social media has destroyed an enormous amount of value by making it harder to have authentic relationships.
Danihan 9y ago
I highly doubt a blog post will have that effect.
- JoshMnem 9y ago
  Blog posts start conversations.
- Devolver 9y ago
  Me too. But you know what they say about the pen and the sword? Maybe we need to update the saying: “the keyboard and a content database can be mightier than the predator drone” ?
oiuplk 9y ago
The new Facebook may be kinder and gentler and not designed to exploit you. The killer feature may be that it does not exploit your human flaws. People are always surprised what new needs we have for technology. Now we need software and services that respect us as humans.
opportune 9y ago
In other news, it's really hard for the "next British East India Company" to enter the Chinese Opium market

spaceribs 9y ago

If you can name a successful product which does not manipulate users to continue using that product, then it's not a product: * I've been manipulated to bring an umbrella because I don't like to get rained on. * I've been manipulated into liking broadband because it provides the internet faster than dialup. * I've been manipulated into owning a car because it's a fast convenient mode of transportation. This is the basics of products and services that provide value, this isn't magical or new, and this book is probably being more honest than most industries are. The point the book was making is that if you don't want a product to be flash in the pan garbage, follow some simple guidelines which ensure that your service is actually providing some sort of value to end users, or your end users will eventually leave and find better things to do with their time and resources (Unless you're selling heroin I guess). I don't consider Facebook sending me an email when I get a personal message to be exploitative, but I find no value at all when they send me something every time one of my friends fart. I specifically avoid interacting with those "hey someone did something" emails because they are annoying and make me want to stop being on facebook altogether. That's the result of abusing the hook model, users go away because it's annoying and unproductive.

maxxxxx 9y ago
Manipulation had always been there but what's new is that the best minds of a generation are working on exploiting human weaknesses with the help of technology to a new level. Same for surveillance. It was always there but now it can be done on an absolutely unprecedented level.
dj-wonk 9y ago
In standard / common usage, selling and marketing an umbrella to someone is not "manipulation". Your definition of "manipulate" is not what most people mean in these contexts. Your use of "manipulate" in such a way muddies the waters and distracts from the core issue. Putting word choice aside, the core issue here is _not_ about people's preferences (e.g. many people don't like to get rained on). Rather, the core issue is about unknown and unwelcome influences, particularly influences that operate on parts of our minds that are hard to combat due to human nature. Facebook's manipulation of our attention is conceptually related to how casinos trick people into staying longer than they would otherwise.

ttoinou 9y ago

The original article seems fine. I don't see how it was immoral. Seemed more like "amoral" https://medium.com/the-mission/the-morality-of-manipulation-c3115fb2bb3d https://medium.com/the-mission/the-morality-of-manipulation-... In the meantime, users will have to judge the yet unknown consequences for themselves, while creators will have to live with the moral repercussions of how they spend their professional lives. https://en.wikipedia.org/wiki/Amorality https://en.wikipedia.org/wiki/Amorality

scythe 9y ago
People are amoral, acts are immoral. An act by an amoral person can certainly be immoral.

dj-wonk 9y ago

I am glad to see more people caring about long-term effects of their use of social software. More of us are mindful in how we engage with technology. Now that I've said the positive, let me talk about the negatives. I bet things will get worse before they get better. At present, the commercial pressures on social media platforms to manipulate "users" dwarf human pushback against these "dark art" techniques. I don't see any easy solutions. As with many collective challenges, we need increased awareness to effect change. This is arguably a political problem; we need social organizing as a counterbalancing force.

TheOtherHobbes 9y ago
We don't need "increased awareness", we need a social decision system (aka "an economy") that rewards positive social outcomes and disincentivises negative social outcomes. At the moment we have the opposite. No one cares how you make your first billion or two, or how people suffer as a side effect. They only care that you have a billion or two. Toxic side-effects remain invisible and uncounted. There's a certain irony in attacking EA for building a game economy that's closer to the effort/reward profile of the real economy - for both players and owners - than any other game economy to date.
- dj-wonk 9y ago
  First, a question for you: What values would you like your "social decision system" to embody? Second, I advocate for increased awareness because awareness is the first step towards rational thought. Many psychological manipulation techniques work best on the irrational parts of our brains, so often a useful first step is to engage the rational mind. That said, I'm not claiming awareness guarantees systemic change.
- AckSyn 9y ago
  > rewards positive social outcomes and disincentivises negative social outcomes Therein lies the problem. Who is to decide what is a social positive or negative among some of the more nuanced and innocuous bits of society? This is not a road we should travel down any further than we have already.
- 9y ago

ncantelmo 9y ago

My knee-jerk reaction to the book Hooked (which I've somewhat shamefully avoided reading out of a sort of premeditated disgust) was based on the general arguments laid out in this blog post. I believe the tech world has a huge problem on its hands here, and that problem doesn't get anywhere near the attention it deserves. Large swaths of the population are turning into zombies and tech is the facilitator. Try looking at the drivers turning through a busy intersection some time. It's obscene how many are staring down at their phones rather than at whatever might be in the road around the corner. That said, I don't believe the problem is simply that certain products remind or encourage people to use them at times. Rather, it's that there is a class of truly addictive products that don't do much (if anything) to help people manage the resulting dependency. And that approach is celebrated far more than it's scrutinized. Coincidentally, I wrote the following on Twitter yesterday: "Any potentially addictive software/service should offer a way for users to lock themselves out of their own account for a predetermined set of hours." [0] It was meant in my usual tongue-in-cheek style, but I do think that a solution for breaking tech product dependency is sorely needed and hope this discussion gains some steam. [0] https://twitter.com/ncantelmo/status/930876372620374017 https://twitter.com/ncantelmo/status/930876372620374017

cokeandsympathy 9y ago
You might want to check out some of the criticism advanced by people such as Tristan Harris on the tension between the user behaviors that tech companies optimize their product design, and the goals users hope to accomplish through their use. I do in some ways support Nir in that we should learn how to harness these types of technologies for our own good, to 'supercharge' our ability to accomplish what we want, but I do not trust any company dependent on ad-revenue (or backed by Wall Street in general) to help with that. I'd also recommend checking out The Attention Merchants by Tim Wu and Addiction by Design by Natasha Schull for background on the history of the commodification of attention and of the
deleted 9y ago
[deleted]
sjellis 9y ago
"Any potentially addictive software/service should offer a way for users to lock themselves out of their own account for a predetermined set of hours." It's really noticeable that Hacker News provides exactly this feature, and absolutely no other site that I've seen does.

acchow 9y ago

This is a fairly similar argument to saying drug dealers are bad because they profit off your addiction. I don't know if I agree or disagree with that, but at what point do we actually acknowledge agency?

scythe 9y ago
Drug dealers don't have intricate control over the effects of the drugs they're selling. I imagine if synthetic drugs were legal in general rather than specific compounds you might see similar problems pop up (e.g. 3-fluoroethamphetamine, a compound screened in rats to be maximally addictive) but it's reasonable to advocate for a drug market that avoids this.
- emmab 9y ago
  > Drug dealers don't have intricate control over the effects of the drugs they're selling. yet
intended 9y ago
I think Agency is always understood to be a part of the equation - and the people with the agency are not the issue. By analogy - If the population of a city was exposed intentionally, to a cold virus - it is not the people with natural resistance, we are concerned about. We are glad those people are not affected, and know they are not part of the % affected by the virus - and then we focus on the people affected by it. --- If I were to engage with your point in the spirit it is intended, there are several issues - 1) Tech recruits the best of the best. IT pays for the smartest people in the world (and this is not hyperbole), to get past agency. This is not hard core drug dealer style sale of addiction. This is candy crush, happy smiley imagery, "just one more game", style behavioral addiction. Most people don't even know they are being addicted/affected 2) This is all at internet scale. Just on a probabilistic level, the casualties/infection is going to vastly outperform personal agency. 3) The network effects of having more people doing this, at this scale, pushes it to crisis levels very quickly. It wouldn't matter what the few immune or capable can do.

m3kw9 9y ago

When max profits means max attention from users this will naturally get public companies like FB to craft products this way

denkmoon 9y ago

The thin end of the wedge here is advertising. It is deliberately manipulating people using psychological insights, often targeting weaknesses to create a recurrent cash flow. Yet the entertainment and content-creating side of the internet relies very explicitly on ad revenue. Are creators that rely on revenue generated by manipulating human psychology morally bankrupt?

cyberpunk0 9y ago
Yes they are completely morally bankrupt
SomeStupidPoint 9y ago
They invented a whole language to paper over the fact that their entire field is weaponized psychology employed against their customers. Do a noun for noun substitution to unpack a marketing meeting, and you'll want to puke. Frankly, it's shameful for any company that says they care about their customers to employ marketing psychology tactics. Marketing and advertising are (in their modern, virulent form) simply an attempt to hack viewers to shift the locus of control from the customer to the business. (Other objectionable things about tech have this kind of behavior, too.) Frankly, it's just disgusting and I can't blame customers for really viewing companies negatively based on their advertising practices. (If all that wasn't bad enough, online advertising often comes with literal malware ontop.)

diego_moita 9y ago

When you think about technology critics none seems more relevant today than Aldous Huxley. The traditional Hollywood criticism on technology goes along the lines of Frankenstein, Metropolis, 2001 A Space Odyssey, Terminator and Matrix: technology will control us. I find it dumb, frankly. Then there is the political criticism along the lines of George Orwell and some Luddites: technology is a tool for control and domination either by governments or evil capitalists. It is not totally wrong (e.g.: China, Turkey) but not the exact true sometimes. But the Huxley criticism is, for me, the most interesting: technology will entertain us to stupidity, apathy and alienation. This seems to be very much the case in here.

zitterbewegung 9y ago
We are in a society that has the distractions of Huxleys world with the surveliance of Orwell. The only thing different is the fact that no one expected our computers to get so small . That we would stare at screens that watch us every hour of every day. The anime psycho pass is much more relevant .
- chiefalchemist 9y ago
  I think it's easy to get hung up on the means (that each authored). But as we know, the prediction business is a tough one. For example, Orwell (in 1984) focused on the filtering, manipulation, constriction and reconstruction of information in order to continuouly redraw the history / truth. But today the truth is something that's often just as fuzzy because we're overwhelmed with too much information and too many distractions. Different means, same ends (read: confused and manipulated proles). But it's the ends that matter. Both Huxley and Orwell have warned us. The irony is, at this point, too few are paying attention.
  - WalterBright 9y ago
    The truth has always been fuzzy/slippery. There never was a golden age of unbiased nothing-but-the-correct-truth journalism. The only change now is it's a little more obvious.

zaptheimpaler 9y ago

IMO tech has simply become the dominant form of media. People who would earlier spend 3 hours/day watching TV might now spend 4 hours/day on Netflix/FB/Twitter etc. The fundamental problem is not tech, it is the incentives of those who pay for the media we consume. Advertisers pay for media and it is in their interest to make the medium as addictive as possible. And when there is enough money to be made as an employee or shareholder of ad companies like FB/Google, people look the other way as long as they benefit. The root problem is simple greed, and its a very old problem. Things are not going to change anytime soon as long as people are able to rationalize doing a small amount of evil for enough money/prestige. (not to say I wouldn't do the same) Thats why these discussions seem pointless to me. Ultimately the people paying/investing in a company decide how a company operates, and they do not give a f* about the next article telling us how bad the product is. Even ordinary shareholders will happily buy FB stock if its projected to go up, nevermind why that is. When you operate within an economic system that is built solely to optimize revenue... it optimizes revenue, at all social costs.

snowpanda 9y ago
I agree with you, but I think it's more than just greed. These companies consist of many employees. Depending on how you look at it, all those employees are contributing to the problem indirectly by working for the company. So one question that comes to mind is are these employees greedy too? What about the interns that get minimum wage? Are they not simply trying to make a living? So I think the first part of your comment is more accurate in the sense that people rationalize things. In that regard, I think we should look at schools emphasizing values in terms of work ethics. When I was in college I took a business ethics class. I realized that was the first time in my entire education that I was learning things about anything even remotely related to this subject, not to mention, the class was optional. That's a sad thing to happen in only the last years before entering the permanent work force.
- zaptheimpaler 9y ago
  The fact is if someone works for one of those companies, they are contributing to the problem. Maybe out of greed, maybe necessity, maybe ignorance. Moral judgements aside, it really doesn't matter why as long as they choose to work there - the company still benefits and the cycle still continues. I'm sure teaching ethics would help.. but then again I don't think all the recent revelations about FB/Google/Twitter have done anything to make hiring more difficult for them. At most they pay 10-20% more and get the people they need. Parts of wall street are unethical and yet they get all the employees they need given the right salary.

a-guest 9y ago

And then today in YC a blog about "Startup Ideas", there is a suggestion for a "Social Network for Children". https://blog.ycombinator.com/13-startup-ideas/ https://blog.ycombinator.com/13-startup-ideas/ Please, no.

Blazespinnaker 9y ago

Gamification can provide value. Stack overflow is all about addiction and value. The blog title was manipulative in its hyperbole, btw.

ben_w 9y ago
I wouldn’t call stackoverflow addictive - if anything I use it too little! - but I do have that problem with some of the linked network sites on stackexchange. “Why yes, who would win in a fight between Aquaman and The Flash is more interesting than why the third argument of the ExampleFooAPI constructor doesn’t behave itself!” (I’m glad adblock lets me hide that block. Really helped me avoid those distractions).

bryanrasmussen 9y ago

There's a thing in essays where the author signifies cutting through the bull with some earthy and down-home talk, but I think this sentence "It’s too soon to tell, but it ain’t look so pretty from here." is a step too far.

bryanrasmussen 9y ago
actually maybe the article just needs an editor or at least a proofreader.
- Devolver 9y ago
  Chose to use this grammatically broken slang very consciously.
creo 9y ago
It's not too far. Maybe You don't feel effects of antisocial media, but i do. Few years ago if friends wanted to go swimming. They were calling each other, there was human interaction. Now it's event on facebook. If You dont have favebook you're basically not a friend anymore, because noone is going to call You and invite. This is the effect of social media on our civilisation. It's like in '60 doctors were recommending smoking as stress reliever and everyone agreed until they knew better. http://fortune.com/2016/06/22/loneliness-is-a-modern-day-epidemic/ http://fortune.com/2016/06/22/loneliness-is-a-modern-day-epi... https://www.cbsnews.com/news/many-americans-are-lonely-survey-finds/ https://www.cbsnews.com/news/many-americans-are-lonely-surve...

bryanrasmussen 9y ago

I was going to clap for this story, the first time ever (despite my misgivings regarding grammar and spelling - I'm sure I've made errors in this post, but none as egregious as the ones I've found in the essay) but anyway it turns out to clap on medium I need to create an account. I decided to forgo the opportunity out of respect for the author's evident wishes.

hohenheim 9y ago
Ironically, clapping, is the very device Medium uses to keep writers writing, the same stimuli that releases dopamine in other platforms. And yet, the author decided to write his article on the very site that uses techniques Nir talks about to promote engagement.
- Devolver 9y ago
  I agree. There is a contradiction here. And I am nearly as uncomfortable with centralized publishing platforms as I am with mass scale behavior design for profit. And yet. I originally published on my own site and it didn’t get much traction. But when I edited it substantially and published as a direct response to Nir’s post, it reached 30,000 people, 6000 of whom finished the whole thing...in less than 24 hours. Since the goal for me is spreading the message and fueling the converstion, I gotta go where distribution follows.
  - bryanrasmussen 9y ago
    hmm, sounds like that distribution thing is highly addictive /I kid
    - Devolver 9y ago
      I’d be lying if I said that reaching an audience and seeing my post generate (mostly) positive, thoughtful discussion didn’t feel good. But that feeling (for me, at least) is closer to gratitude and hopefulness than it is to ego gratification. The claps and notifications of retweets are, however, quite addicting, in the truest sense of the word. I find myself checking twitter more than is reasonable to see where it’s going. And while there is some upside there (connecting with and engaging in conversation with like-minded peeps), the frequency of my checking is highly disproportionate to that Value. Ugh.

CodeWriter23 9y ago

Nir Eyal didn’t invent anything. He simply adapted mass media techniques to the App era.

PeOe 9y ago

I´ve read a book recently called "Hooked" by Nir Eyal. https://www.amazon.de/Hooked-How-Build-Habit-Forming-Products/dp/0241184835/ref=sr_1_1?ie=UTF8&qid=1510910504&sr=8-1&keywords=nir+eyal+hooked https://www.amazon.de/Hooked-How-Build-Habit-Forming-Product... It´s exactly about this topic. He also gives quite a few examples of the achievements of Apple and so on. Really interesting.

placebo 9y ago
The entire post is addressed to him, and not in a positive way. I tend to agree...

yodsanklai 9y ago

For me, the main problem is that I need my computer for what I consider "legit" activities such as working or communicating with my friends. Starting from there, it's easy to get trapped into other "toxic" activities such as facebook or news. I'd like to be able to just put the computer aside for a few days but it's almost impossible.

deleted 9y ago

[deleted]

uoaei 9y ago

The thing is, it doesn't seem like much of this is even intentional. A/B tests (probably inspired by some sort of thought experiment re: 'getting more engagement') tell a story that says you have more users if you do this thing, when that thing just happens to be addictive.

randomsearch 9y ago
I used to think that, until Sean Parker recently stated it was a conscious, deliberate, decision to exploit people.

westmeal 9y ago

I've never agreed with an article quite as much as this one.

xkcdef 9y ago
Also, I would like to point out three things to folks who are asking "What about personal responsibility? What about agency?" here and on the article comments. Facebook's shadow profiles take away any 'agency' and 'personal responsibility' out of the equation. While I don't yet know Facebook using shadow profiles to make someone's life worse off, their penchant for saying "Screw it" to any ethical considerations makes me feel its only a matter of time before even that crutch goes away. Secondly, there is the complexity. You cannot take personal responsibility for something that you don't fully comprehend. At the same time, we are at a point where AI produces data insights that even the AI builders cannot reason about. For example, how does someone take responsibility for clicking the Like button on a post and then finding out that most of the people who liked the post were (say) closet anarchists which was then correlated with them being the best targets for some kind of subversive advertising from (say) the NRA? Thirdly, there is the asymmetry involved in "moderate use". It is possible for Facebook to create "automated notifications" implemented at a very tiny cost to them in terms of their resources/profits while the effort needed on your part to make sure you don't turn into a Pavlovian "notifications" dog comes at an extremely high time cost to you.