6 ms·

AI and Robots Are a Minefield of Cognitive Biases

75 points by ystad 5y ago

deleted 5y ago
[deleted]
bottled_poe 5y ago
Some interesting ideas, but it’s all conjecture and very little data. I’m sure Dr Kim knows this stuff inside out, but I felt like the content lacked support. Also, there are somewhat contradictory statements. Eg. The point about evolution being equivalent to millions of years of AI training is absurd - In a later sentence, the author states that computers were designed for computation (duh) and so it shouldn’t be a surprise that they are good at that one thing. The whole thing reads as being rushed and superficial.
- ZeroGravitas 5y ago
  I thought it made a good point. I remember Microsoft introducing an early AI in a game and they made it a young boy. This seemed like an artful/deceptive attempt to take advantage of human biases to make it seem more inpressive than it was. Which is fine, up to a point, for showbiz, but probably not ideal for wider applications. I'm not sure if this technique has a name but I see it more and more like "<unhealthy thing> has more <good quality> than <healthy thing>". They're not lying but they are taking advantage of human biases to very specifically talk about a minor truth to distract you from the overall truth. "Rinsing your recycling (in water heated by electricity generated by coal) generates more carbon than the recycling saves". Again true, but carefully crafted to make you think the opposite of the truth. Notably, no evil, mustache-twirling villain is required for this. The people who make boring, reasonable assertions will be automatically filtered out in favour of the people making wild and sloppy claims that appeal to people's biases and when they get caught out they'll be replaced by another similar person. But a few villains will intentionally take advantage of it too.
  - mattcwilson 5y ago
    I think your given example could benefit from spelling it out further. Let’s say that I read "Rinsing your recycling (in water heated by electricity generated by coal) generates more carbon than the recycling saves", investigate it, and convlude it’s true. What is the additional truth that statement is making me think the opposite of? I lost you there.
    - ZeroGravitas 5y ago
      That multiple, independant, well-regarded and thorough lifecycle analyses of recycling have consistently shown it to be both financially and environmentally (including but not limited to carbon output specifically) beneficial. edit: I removed a "not" to clear up the ambiguous double-negative. Summary, my position is that science says recycling is a good thing. Numerous media stories will focus relentlessly on any small flaw in it, either to get a reaction from the people who do it and think it's a good thing, or support the people who fight against it for political reasons as it represents (like global warming) a massive failure of free markets.
- tharne 5y ago
  > The whole thing reads as being rushed and superficial. Journalism tends to reward this. I used to work in a much more public-facing field where we would regularly get calls from local journalists about projects we were involved in. The journalists never printed the quotes from the engineers who gave thoughtful, measured, and accurate answers. They always quoted the blowhards and the charlatans in our office who'd inevitably toss together a bunch of meaningless buzzwords, make a series of vague statements that sounded insightful, and play fast and loose with the facts. A journalist's job is to sell newspapers, or clicks, not to keep anyone informed.
  - bobthechef 5y ago
    > A journalist's job is to sell newspapers, or clicks, not to keep anyone informed. Yes. And to propagandize on behalf of his and his organization's preferred political views. Seldom do they probably consciously realize they are doing this. The echo chamber synergizes with their prejudices and steers them in that direction. And for them, it's a job. Their reward is the editor's approval, or at least the absence of his disapproval.
    - benrbray 5y ago
      There's a Noam Chomsky quote for this: “I’m sure you believe everything you’re saying. But what I’m saying is that if you believe something different, you wouldn’t be sitting where you’re sitting.” [1] https://www.goodreads.com/quotes/9692159-i-m-sure-you-believe-everything-you-re-saying-but-what-i-m https://www.goodreads.com/quotes/9692159-i-m-sure-you-believ... [2] "Manufacturing Consent" by Noam Chomsky
    - throwaway894345 5y ago
      I’m not sure why you’re being downvoted. Is it controversial that publications have become increasingly partisan in recent years? I thought that was widely agreed upon? Or maybe people disagree with your implication that this is a bad thing (I imagine progressives are pleased that the majority of the old reputable outlets are burning through their credibility reserves to advance progressivism).
  - 5y ago
- IAmEveryone 5y ago
  I don't know what sort of "data" you might want. There are modes of argumentation that do not rely on double-blind randomised trials. The argument is, essentially: - people used to believe that being a chess grandmaster is universally considered proof of high intelligence - people also believed that opening a refrigerator requires strictly less intelligence than becoming a chess grandmaster - we now have build artificial "intelligence" that easily beats any human in chess. Ask it to open a refrigerator and see what happens. -> Therefore, our concept of "intelligence", as a single, ordered, continuous, and measurable quantity, is demonstrably wrong. You can proof plenty with a single datapoint or two. As long as you don't care about the color, half a sheep proofs sheep exist. There's no n and no p-value attached to that because it is not subject to measuring errors and confounders. Sheep just is. They article isn't quite clear on how our biases are potentially dangerous, because it happens to be the point of warnings to come before the actual damage occurs. But when someone is setting out to revolutionise all our lives with something that uses the term "intelligence", and neither they nor we can actually come up with definition of the concept, maybe the burden of proof should be on those saying it's all fine, and not those asking for a bit of occasional reflection. I also just don't get what's controversial about the article. It doesn't even ask for anything dramatic, or specific. It's the MIT robot tech leader, who knows what they're talking about. And when people start invoking the human rights of abused robots, I am confident the robotics community wouldn't object to that being either a joke or misguided anthropomorphising.
blamestross 5y ago
Brains are survival engines, not truth detectors. If self deception promotes fitness, the brain lies. Stops noticing irrelevant things. Truth never matters. Only fitness. By now you don’t experience the world as it exists at all. - Peter Watts "Blindsight"
- bobthechef 5y ago
  Is that a true statement? Has Peter Watts transcended his humanity and entered the ethereal realm of gods where he is no longer subject to that which he claims humans are subject?
  - scotty79 5y ago
    Yes. Pretty much. For example your conscious awareness processes all the time just 2% of the image your eyes actually see.
    - throwaway316943 5y ago
      And how would you solve the compression problem of dealing with all that data? Our brains are running a similar algorithm to mpeg-4 where we discard redundant data and make some basic predictions about movement. If they didn’t we’d waste a lot of time paying attention to unimportant things in our visual field.
      - scotty79 5y ago
        Those are not necessarily unimportant things. These are just things we don't have a capacity to notice because of crap hardware we have to process this information. We don't conciously process 2% of what we see because only 2% is relevant. We do that because hardware needed to process more wouldn't pass through birth canal.
        throwaway316943 5y ago
        The 2% off visual data that we process is essentially just the stuff that is changing. It’s not some secret hidden knowledge, we can still intentionally examine things closely and see all there is to see.
rbanffy 5y ago
As the tragic ED-209 incident showed us, inadequate training datasets can have deadly consequences. The machine never saw a nicely dressed white man holding a gun and couldn't figure out the gun was dropped.
- taeric 5y ago
  This took me longer to place than I'd have expected. Kudos!
_Nat_ 5y ago
tl;dr- Humans may consider their own intelligence as a reference-model from which to conceptualize AI. Biases in self-conception may undermine that reference-model, thus undermining conceptions of AI. Earlier version: https://www.naverlabs.com/en/storyDetail/212 https://www.naverlabs.com/en/storyDetail/212
bobthechef 5y ago
The article makes some acceptable points, but this article seems disorganized and there seems to be a disconnect between the content and the conclusion. The author devotes most of the article to supporting the idea that anthropomorphizing "AI" is a category mistake. But at the end of the article, he claims that this category mistake needs to be taken into account by producers of AI?
magwa101 5y ago
The cognitive biases of AI are typically just a mirror on our own.

75 points by ystad 5y ago

deleted 5y ago

[deleted]

Some interesting ideas, but it’s all conjecture and very little data. I’m sure Dr Kim knows this stuff inside out, but I felt like the content lacked support. Also, there are somewhat contradictory statements. Eg. The point about evolution being equivalent to millions of years of AI training is absurd - In a later sentence, the author states that computers were designed for computation (duh) and so it shouldn’t be a surprise that they are good at that one thing. The whole thing reads as being rushed and superficial.

ZeroGravitas 5y ago
I thought it made a good point. I remember Microsoft introducing an early AI in a game and they made it a young boy. This seemed like an artful/deceptive attempt to take advantage of human biases to make it seem more inpressive than it was. Which is fine, up to a point, for showbiz, but probably not ideal for wider applications. I'm not sure if this technique has a name but I see it more and more like "<unhealthy thing> has more <good quality> than <healthy thing>". They're not lying but they are taking advantage of human biases to very specifically talk about a minor truth to distract you from the overall truth. "Rinsing your recycling (in water heated by electricity generated by coal) generates more carbon than the recycling saves". Again true, but carefully crafted to make you think the opposite of the truth. Notably, no evil, mustache-twirling villain is required for this. The people who make boring, reasonable assertions will be automatically filtered out in favour of the people making wild and sloppy claims that appeal to people's biases and when they get caught out they'll be replaced by another similar person. But a few villains will intentionally take advantage of it too.
- mattcwilson 5y ago
  I think your given example could benefit from spelling it out further. Let’s say that I read "Rinsing your recycling (in water heated by electricity generated by coal) generates more carbon than the recycling saves", investigate it, and convlude it’s true. What is the additional truth that statement is making me think the opposite of? I lost you there.
  - ZeroGravitas 5y ago
    That multiple, independant, well-regarded and thorough lifecycle analyses of recycling have consistently shown it to be both financially and environmentally (including but not limited to carbon output specifically) beneficial. edit: I removed a "not" to clear up the ambiguous double-negative. Summary, my position is that science says recycling is a good thing. Numerous media stories will focus relentlessly on any small flaw in it, either to get a reaction from the people who do it and think it's a good thing, or support the people who fight against it for political reasons as it represents (like global warming) a massive failure of free markets.
tharne 5y ago
> The whole thing reads as being rushed and superficial. Journalism tends to reward this. I used to work in a much more public-facing field where we would regularly get calls from local journalists about projects we were involved in. The journalists never printed the quotes from the engineers who gave thoughtful, measured, and accurate answers. They always quoted the blowhards and the charlatans in our office who'd inevitably toss together a bunch of meaningless buzzwords, make a series of vague statements that sounded insightful, and play fast and loose with the facts. A journalist's job is to sell newspapers, or clicks, not to keep anyone informed.
- bobthechef 5y ago
  > A journalist's job is to sell newspapers, or clicks, not to keep anyone informed. Yes. And to propagandize on behalf of his and his organization's preferred political views. Seldom do they probably consciously realize they are doing this. The echo chamber synergizes with their prejudices and steers them in that direction. And for them, it's a job. Their reward is the editor's approval, or at least the absence of his disapproval.
  - benrbray 5y ago
    There's a Noam Chomsky quote for this: “I’m sure you believe everything you’re saying. But what I’m saying is that if you believe something different, you wouldn’t be sitting where you’re sitting.” [1] https://www.goodreads.com/quotes/9692159-i-m-sure-you-believe-everything-you-re-saying-but-what-i-m https://www.goodreads.com/quotes/9692159-i-m-sure-you-believ... [2] "Manufacturing Consent" by Noam Chomsky
  - throwaway894345 5y ago
    I’m not sure why you’re being downvoted. Is it controversial that publications have become increasingly partisan in recent years? I thought that was widely agreed upon? Or maybe people disagree with your implication that this is a bad thing (I imagine progressives are pleased that the majority of the old reputable outlets are burning through their credibility reserves to advance progressivism).
- 5y ago
IAmEveryone 5y ago
I don't know what sort of "data" you might want. There are modes of argumentation that do not rely on double-blind randomised trials. The argument is, essentially: - people used to believe that being a chess grandmaster is universally considered proof of high intelligence - people also believed that opening a refrigerator requires strictly less intelligence than becoming a chess grandmaster - we now have build artificial "intelligence" that easily beats any human in chess. Ask it to open a refrigerator and see what happens. -> Therefore, our concept of "intelligence", as a single, ordered, continuous, and measurable quantity, is demonstrably wrong. You can proof plenty with a single datapoint or two. As long as you don't care about the color, half a sheep proofs sheep exist. There's no n and no p-value attached to that because it is not subject to measuring errors and confounders. Sheep just is. They article isn't quite clear on how our biases are potentially dangerous, because it happens to be the point of warnings to come before the actual damage occurs. But when someone is setting out to revolutionise all our lives with something that uses the term "intelligence", and neither they nor we can actually come up with definition of the concept, maybe the burden of proof should be on those saying it's all fine, and not those asking for a bit of occasional reflection. I also just don't get what's controversial about the article. It doesn't even ask for anything dramatic, or specific. It's the MIT robot tech leader, who knows what they're talking about. And when people start invoking the human rights of abused robots, I am confident the robotics community wouldn't object to that being either a joke or misguided anthropomorphising.

blamestross 5y ago

Brains are survival engines, not truth detectors. If self deception promotes fitness, the brain lies. Stops noticing irrelevant things. Truth never matters. Only fitness. By now you don’t experience the world as it exists at all. - Peter Watts "Blindsight"

bobthechef 5y ago
Is that a true statement? Has Peter Watts transcended his humanity and entered the ethereal realm of gods where he is no longer subject to that which he claims humans are subject?
- scotty79 5y ago
  Yes. Pretty much. For example your conscious awareness processes all the time just 2% of the image your eyes actually see.
  - throwaway316943 5y ago
    And how would you solve the compression problem of dealing with all that data? Our brains are running a similar algorithm to mpeg-4 where we discard redundant data and make some basic predictions about movement. If they didn’t we’d waste a lot of time paying attention to unimportant things in our visual field.
    - scotty79 5y ago
      Those are not necessarily unimportant things. These are just things we don't have a capacity to notice because of crap hardware we have to process this information. We don't conciously process 2% of what we see because only 2% is relevant. We do that because hardware needed to process more wouldn't pass through birth canal.
      - throwaway316943 5y ago
        The 2% off visual data that we process is essentially just the stuff that is changing. It’s not some secret hidden knowledge, we can still intentionally examine things closely and see all there is to see.

rbanffy 5y ago

As the tragic ED-209 incident showed us, inadequate training datasets can have deadly consequences. The machine never saw a nicely dressed white man holding a gun and couldn't figure out the gun was dropped.

taeric 5y ago
This took me longer to place than I'd have expected. Kudos!

_Nat_ 5y ago

tl;dr- Humans may consider their own intelligence as a reference-model from which to conceptualize AI. Biases in self-conception may undermine that reference-model, thus undermining conceptions of AI. Earlier version: https://www.naverlabs.com/en/storyDetail/212 https://www.naverlabs.com/en/storyDetail/212

bobthechef 5y ago

The article makes some acceptable points, but this article seems disorganized and there seems to be a disconnect between the content and the conclusion. The author devotes most of the article to supporting the idea that anthropomorphizing "AI" is a category mistake. But at the end of the article, he claims that this category mistake needs to be taken into account by producers of AI?

magwa101 5y ago

The cognitive biases of AI are typically just a mirror on our own.