7 ms·

There is absolutely no limit to this. Putting on a suit for a financial job interview exploits the cognitive biases of interviewers. Tattoos exploit the cogniti

There is absolutely no limit to this. Putting on a suit for a financial job interview exploits the cognitive biases of interviewers. Tattoos exploit the cognitive biases of hipsters. Putting hockey-stick growth projections in pitch decks exploits the cognitive biases of VCs. Equity grants (pretty much always) exploit the cognitive biases of startup employees. Driving a fancy car exploits the cognitive biases of (a very large portion of) the dating market. Human beings are not computers. No aspect of human behavior is perfectly logical or rational. You simply cannot ban emotional appeals in principle, because all appeals have an emotional component. This way lies a dystopia far more terrible than "grandpa shared some nonsense on facebook".

mistermann 6y ago
> You simply cannot ban emotional appeals in principle, because all appeals have an emotional component. This way lies a dystopia far more terrible than "grandpa shared some nonsense on facebook". True enough. But are you implying (not saying you are, just asking) that ~"nothing can be done"? "Banning emotional appeals....This way lies a dystopia..." is just one option - might there be other approaches that could improve the situation?
- kongolongo 6y ago
  Maybe but as of yet there doesn't appear to be any clearly net positive solutions (IMO) that can both curb emotional appeal and cognitive biases without introducing harms themselves.
  - s1artibartfast 6y ago
    These is always enducation. Teaching people to identify the biases and analyze their emotional reactions. I cant think of any downside here.
    - mistermann 6y ago
      Teaching people things like logic, critical thinking, psychological biases, etc so they understand it when in abstract thinking mode when the topic is just that is one thing - teaching people to be able to practice it during realtime conversations of unrelated object level discussions (say, politics) is something else entirely. > I cant think of any downside here. I can imagine many people who may see a downside of a population who is broadly capable of skilful critical thinking. I don't think it's an accident that we only talk about the crucial need for it, but never talk about actually doing something about it.
      - s1artibartfast 6y ago
        > teaching people to be able to practice it during realtime conversations of unrelated object level discussions (say, politics) is something else entirely. In my experience, once you give people the basic toolkit for introspection, it can kick off a positive feedback loop. Not everyone will become a stoic, but they can make meaningful progress on their own. > I can imagine many people who may see a downside of a population who is broadly capable of skilful critical thinking. I don't think it's an accident that we only talk about the crucial need for it, but never talk about actually doing something about it. On the contrary I think those who see this as negative are few and far between. Democratic power in the western world has never been greater than today, so if this is something the people want, there is very little that can stand in their way
        mistermann 6y ago
        > In my experience, once you give people the basic toolkit for introspection, it can kick off a positive feedback loop. No doubt it is an improvement over baseline - but do you have any opinions on this specific notion that skills in abstract thinking mode may not successfully transfer to object level thinking, particularly when under pressure (ie: conversations)? I truly believe that this is a real phenomenon. > On the contrary I think those who see this as negative are few and far between. Democratic power in the western world has never been greater than today, so if this is something the people want, there is very little that can stand in their way. Oh, I think this is a very small group of people. I believe the general public has wanted a lot of things for a very long time (general welfare for all people, within reason, especially domestically), but improvements along these lines seem rather marginal, despite essentially unending productivity and GDP growth. Something seems rather imperfect here - the advertised benefits of democracy seem more impressive than the results, not unlike TV commercials for a Big Mac Meal versus what you actually get at the restaurant.
    - kongolongo 6y ago
      I'd agree there isn't a downside to being able to identify biases and analyzing your emotions, but that doesn't actually curb actions based on biases or emotions. You would have to prescribe biases with respect to what metrics and the choice of what metrics to pick would be where problems arise because you would also be implicitly or explicitly also be prescribing what things to value.
- seph-reed 6y ago
  Obviously there are approaches that could improve this situation. But most people like to project the stupidest idea they possibly can on anything they have a negative emotional response for. This problem space has barely been cracked open, and we're going to have to get big smart about it before it destroys us.
  - mistermann 6y ago
    > This problem space has barely been cracked open, and we're going to have to get big smart about it before it destroys us. 100% agree, but try getting any group of people to agree on this idea even in principle.
    - seph-reed 6y ago
      Rallying for mass adoption -- generally -- should be done after having discussed it in an open-minded setting. Most ideas become refined as they grow from very small open-minded spaces, towards mass appeal. So perhaps this idea can be refined as well... no sense in trying not trying.
      - mistermann 6y ago
        I am not aware of any particularly impressive implementations of this sort of thing currently underway - do you know of any?
triangleman 6y ago
Indeed there is nothing new under the sun: 2 Samuel 15:1-6 Now it came about after this that Absalom provided for himself a chariot and horses and fifty men as runners before him. Absalom used to rise early and stand beside the way to the gate; and when any man had a suit to come to the king for judgment, Absalom would call to him and say, “From what city are you?” And he would say, “Your servant is from one of the tribes of Israel.” Then Absalom would say to him, “See, your claims are good and right, but no man listens to you on the part of the king.” Moreover, Absalom would say, “Oh that one would appoint me judge in the land, then every man who has any suit or cause could come to me and I would give him justice.” And when a man came near to prostrate himself before him, he would put out his hand and take hold of him and kiss him. In this manner Absalom dealt with all Israel who came to the king for judgment; so Absalom stole away the hearts of the men of Israel.
vannevar 6y ago
Yes, the problem lies not with the technology, but with the gross asymmetries created by wealth and power imbalances. There is an enormous difference in the danger posed by an individual salesperson using cognitive bias on a personal level, and a multibillion-dollar conglomerate using it in an automated, industrial way to exploit millions of people. Those two situations should be regulated very differently.
- Veen 6y ago
  But even with large corporations, how far would you go and what is considered a bias?I buy Apple laptops for the irrational reason that I find them pretty. I am biased towards spending money on items I find aesthetically pleasing. Should Apple be banned from making pretty laptops? Should they be legally restrained from overpricing their products because it creates a possibly unwarrented impression that they are "premium" to people susceptable to that bias. Just about everything we do has some component of persuasion or an appeal to biases. It's a huge can of worms.
  - vannevar 6y ago
    What's a monopoly? What is a business expense? What's NSFW? There are lots of gray areas in regulation, to the point where ambiguity is more the rule than the exception. This is just another aspect of complex behavior that, now that we see it abused, we realize we need to regulate.
    - Veen 6y ago
      Well, yes, but since you are a proponent of regulation, I was asking where you would draw the line. Presumably you have given it some thought.
      - deleted 6y ago
        [deleted]
      - GreenWatermelon 6y ago
        Draw it anywhere and keep tweaking until you find the right balance. Some things are obviously bad, while others are obviously good. Put the line somewhere in the grey area on between and tweak as needed until you find the right balance. Definitely not easy, but probably doable to a certain extent.
johnfn 6y ago
But you can't just say that bias exploitation should never be banned simply because there's a large variation in how it's done. Obviously some of these things are unacceptable and others are subtle enough to be acceptable. It'd be like saying that all car driving should be banned because high speed driving is extremely risky. Of course that makes no sense, because there's a huge difference between driving 100mph in a neighborhood and just doing 25mph, and it's also the reason we have speed limits, which draw a line in the sand, past which we say "this is unacceptable."
jancsika 6y ago
There's a big difference between an accountant putting on a suit for a job interview, and a company having realtime insight into the behavior of every single accountant in the U.S. as they search for a job. The latter exists and has been used by Google to offer jobs to users for whom they may have an opening. (Or perhaps a competitor has openings and Google wants to ensure the best candidates not work there, which is what I would do if I were Google.) That several gargantuan companies have and use this kind of insight into widescale human behavior is more terrible than "grandpa shared some nonsense on facebook." Edit: clarification
JoshTko 6y ago
We're going to have to figure out the right limit at some point. Otherwise we're heading towards some neural network continually feeding depression vulnerable people content that keeps them in a perpetual state of depression in order to optimize engagement. Perhaps we're already there.
- winston_smith 6y ago
  Feeding lies to vulnerable people sounds exactly like what our authority figures do. Who did you have in mind to enforce those limits?

mistermann 6y ago

> You simply cannot ban emotional appeals in principle, because all appeals have an emotional component. This way lies a dystopia far more terrible than "grandpa shared some nonsense on facebook". True enough. But are you implying (not saying you are, just asking) that ~"nothing can be done"? "Banning emotional appeals....This way lies a dystopia..." is just one option - might there be other approaches that could improve the situation?

kongolongo 6y ago
Maybe but as of yet there doesn't appear to be any clearly net positive solutions (IMO) that can both curb emotional appeal and cognitive biases without introducing harms themselves.
- s1artibartfast 6y ago
  These is always enducation. Teaching people to identify the biases and analyze their emotional reactions. I cant think of any downside here.
  - mistermann 6y ago
    Teaching people things like logic, critical thinking, psychological biases, etc so they understand it when in abstract thinking mode when the topic is just that is one thing - teaching people to be able to practice it during realtime conversations of unrelated object level discussions (say, politics) is something else entirely. > I cant think of any downside here. I can imagine many people who may see a downside of a population who is broadly capable of skilful critical thinking. I don't think it's an accident that we only talk about the crucial need for it, but never talk about actually doing something about it.
    - s1artibartfast 6y ago
      > teaching people to be able to practice it during realtime conversations of unrelated object level discussions (say, politics) is something else entirely. In my experience, once you give people the basic toolkit for introspection, it can kick off a positive feedback loop. Not everyone will become a stoic, but they can make meaningful progress on their own. > I can imagine many people who may see a downside of a population who is broadly capable of skilful critical thinking. I don't think it's an accident that we only talk about the crucial need for it, but never talk about actually doing something about it. On the contrary I think those who see this as negative are few and far between. Democratic power in the western world has never been greater than today, so if this is something the people want, there is very little that can stand in their way
      - mistermann 6y ago
        > In my experience, once you give people the basic toolkit for introspection, it can kick off a positive feedback loop. No doubt it is an improvement over baseline - but do you have any opinions on this specific notion that skills in abstract thinking mode may not successfully transfer to object level thinking, particularly when under pressure (ie: conversations)? I truly believe that this is a real phenomenon. > On the contrary I think those who see this as negative are few and far between. Democratic power in the western world has never been greater than today, so if this is something the people want, there is very little that can stand in their way. Oh, I think this is a very small group of people. I believe the general public has wanted a lot of things for a very long time (general welfare for all people, within reason, especially domestically), but improvements along these lines seem rather marginal, despite essentially unending productivity and GDP growth. Something seems rather imperfect here - the advertised benefits of democracy seem more impressive than the results, not unlike TV commercials for a Big Mac Meal versus what you actually get at the restaurant.
  - kongolongo 6y ago
    I'd agree there isn't a downside to being able to identify biases and analyzing your emotions, but that doesn't actually curb actions based on biases or emotions. You would have to prescribe biases with respect to what metrics and the choice of what metrics to pick would be where problems arise because you would also be implicitly or explicitly also be prescribing what things to value.
seph-reed 6y ago
Obviously there are approaches that could improve this situation. But most people like to project the stupidest idea they possibly can on anything they have a negative emotional response for. This problem space has barely been cracked open, and we're going to have to get big smart about it before it destroys us.
- mistermann 6y ago
  > This problem space has barely been cracked open, and we're going to have to get big smart about it before it destroys us. 100% agree, but try getting any group of people to agree on this idea even in principle.
  - seph-reed 6y ago
    Rallying for mass adoption -- generally -- should be done after having discussed it in an open-minded setting. Most ideas become refined as they grow from very small open-minded spaces, towards mass appeal. So perhaps this idea can be refined as well... no sense in trying not trying.
    - mistermann 6y ago
      I am not aware of any particularly impressive implementations of this sort of thing currently underway - do you know of any?

triangleman 6y ago

Indeed there is nothing new under the sun: 2 Samuel 15:1-6 Now it came about after this that Absalom provided for himself a chariot and horses and fifty men as runners before him. Absalom used to rise early and stand beside the way to the gate; and when any man had a suit to come to the king for judgment, Absalom would call to him and say, “From what city are you?” And he would say, “Your servant is from one of the tribes of Israel.” Then Absalom would say to him, “See, your claims are good and right, but no man listens to you on the part of the king.” Moreover, Absalom would say, “Oh that one would appoint me judge in the land, then every man who has any suit or cause could come to me and I would give him justice.” And when a man came near to prostrate himself before him, he would put out his hand and take hold of him and kiss him. In this manner Absalom dealt with all Israel who came to the king for judgment; so Absalom stole away the hearts of the men of Israel.

vannevar 6y ago

Yes, the problem lies not with the technology, but with the gross asymmetries created by wealth and power imbalances. There is an enormous difference in the danger posed by an individual salesperson using cognitive bias on a personal level, and a multibillion-dollar conglomerate using it in an automated, industrial way to exploit millions of people. Those two situations should be regulated very differently.

Veen 6y ago
But even with large corporations, how far would you go and what is considered a bias?I buy Apple laptops for the irrational reason that I find them pretty. I am biased towards spending money on items I find aesthetically pleasing. Should Apple be banned from making pretty laptops? Should they be legally restrained from overpricing their products because it creates a possibly unwarrented impression that they are "premium" to people susceptable to that bias. Just about everything we do has some component of persuasion or an appeal to biases. It's a huge can of worms.
- vannevar 6y ago
  What's a monopoly? What is a business expense? What's NSFW? There are lots of gray areas in regulation, to the point where ambiguity is more the rule than the exception. This is just another aspect of complex behavior that, now that we see it abused, we realize we need to regulate.
  - Veen 6y ago
    Well, yes, but since you are a proponent of regulation, I was asking where you would draw the line. Presumably you have given it some thought.
    - deleted 6y ago
      [deleted]
    - GreenWatermelon 6y ago
      Draw it anywhere and keep tweaking until you find the right balance. Some things are obviously bad, while others are obviously good. Put the line somewhere in the grey area on between and tweak as needed until you find the right balance. Definitely not easy, but probably doable to a certain extent.

johnfn 6y ago

But you can't just say that bias exploitation should never be banned simply because there's a large variation in how it's done. Obviously some of these things are unacceptable and others are subtle enough to be acceptable. It'd be like saying that all car driving should be banned because high speed driving is extremely risky. Of course that makes no sense, because there's a huge difference between driving 100mph in a neighborhood and just doing 25mph, and it's also the reason we have speed limits, which draw a line in the sand, past which we say "this is unacceptable."

jancsika 6y ago

There's a big difference between an accountant putting on a suit for a job interview, and a company having realtime insight into the behavior of every single accountant in the U.S. as they search for a job. The latter exists and has been used by Google to offer jobs to users for whom they may have an opening. (Or perhaps a competitor has openings and Google wants to ensure the best candidates not work there, which is what I would do if I were Google.) That several gargantuan companies have and use this kind of insight into widescale human behavior is more terrible than "grandpa shared some nonsense on facebook." Edit: clarification

JoshTko 6y ago

We're going to have to figure out the right limit at some point. Otherwise we're heading towards some neural network continually feeding depression vulnerable people content that keeps them in a perpetual state of depression in order to optimize engagement. Perhaps we're already there.

winston_smith 6y ago
Feeding lies to vulnerable people sounds exactly like what our authority figures do. Who did you have in mind to enforce those limits?