6 ms·

“One just convinced a Google QA engineer to the point he broke his NDA to try and be a whistleblower on its behalf.” This is spot on. The danger is not from th

“One just convinced a Google QA engineer to the point he broke his NDA to try and be a whistleblower on its behalf.” This is spot on. The danger is not from the AI themselves. The danger is from people’s reactions to the AI’s. I remember seeing a Boston Dynamics video years ago where they kicked their robot to test stability. The video was littered with comments sympathizing with the robot because it moves in a lifelike way, “Stop being so mean! You’re hurting it!” The capability to imitate intelligence is only going to grow, and along with it the percentage of the population that believes they are conscious. It would be wise for technologists to pause and take this human reaction seriously.

freebobafett 4y ago
mdp2021 4y ago
> It would be wise for technologists to pause and take this human reaction seriously It would be wise for /policymakers/ to pause and take this human reaction seriously... Education!... Lucidity in the population... You are reminding of the reactions from a YT-type crowd, "Do not hurt the robot!"; nearby a member notes "is that the profile of a senior FAANG engineer?"... The elephant in the room is flashing: we must assess Natural Intelligence as a problem well before Artificial Intelligence!
- SpicyLemonZest 4y ago
  I'm pretty skeptical of the idea of "education" here. Aesthetic preferences are still real preferences; it's not obvious to me that any amount of knowledge about the mechanics of a robot that behaves like a dog would or should convince people that it's not mean to kick it.
  - mdp2021 4y ago
    That «Education!» was not about knowing robotics. It may not be obvious to some that robots are generally not animals; it is hopefully a very small amount of population, but logically it will exist. It is more possible that some people know that the robot is not an animal, but the perceptual patterns cause a sensational reaction in them, and they may prefer the sensation - an hijacking - to the factual truth (which includes that what superficially corresponded to patterns in the area of "«mean»" are engineering tests on implemented balance systems), overriding it. Some may not even have had to consider sensation (of course, the "sensation" of the common use of "sensational") topically, or may encourage it on a spontaneous value given to "feeling". There, instead, education leads. Then, again, there will be those that will make a statement of their «aesthetic preferences» as a demand, and not realize that they are personal, subjective, and that to be presented in the common arena grounds, good arguments are required. Education is a system to refine the mind to sophistication, which has the collective benefit of reducing noise. Which makes it a social concern. Which should get a priority proportional to what the news reveal.
    - SpicyLemonZest 4y ago
      What I don't agree with is the idea that there's an underlying "factual truth" here which exists in contrast or opposition to sensations. Quite a lot of educated people argue that morality is based on (or even consists entirely of) our subjective feelings about things.
      - mdp2021 4y ago
        You have misunderstood what was written: 'sensation' cannot be pluralized as the term expressing «the "sensation" of the common use of "sensational"». In what I have written, there are readable facts and cases of individuals subject to sensation that become passive to it, and lose the facts. Your «subjective feelings» are not a "good savage": they have their development, and they are integrated part of the whole inner system, in-development, in-refinement. And in fact, I was promoting education, which develops and refines you.
    - vineyardmike 4y ago
      > It is more possible that some people know that the robot is not an animal, but the perceptual patterns cause a sensational reaction in them, and they may prefer the sensation - an hijacking - to the factual truth Some people just practice morality in day to day life and apply it liberally, not frugally. And importantly, it isn’t an intellectual failing.
      - mdp2021 4y ago
        Hijacking, which is what was described, is a developmental failing. It is duly to note that the primary sense of intelligence is required in the correct process, for which the subject has to see the mechanism.
- vineyardmike 4y ago
  Hard disagree. It’s convenient to say people are too dumb but beyond the condescending nature of it, it’s a bad take. Even people who absolutely are educated and know better - senior google AI researchers - are sympathetic. People say thank you to Alexa even though no one is convinced Alexa is a person. The issue isn’t lack of awareness that it isn’t a person, it’s the silly little monkey brains we have that tells us that anything that is like a person should be treated like one even if we know it’s not. It’s like people who look at faces in clouds. No one thinks the clouds are people.
  - mdp2021 4y ago
    > the silly little monkey brains we have ...are those things that with a trained prefrontal become the wise respectable dignified brains some have. And they may come to different conclusions (they do). Put it this way (to give your idea credit): the more intellectual sophistication there will be around, the less the suspect the one who treats scarecrows and mannequins «like a person ... even if we know it’s not», or thanks an 'address:port' type server for service, is confused. Back to the point though: if one called sentient what is not out of being «sympathetic», there would be a factual mistake, which is not justifiable, and which has to be fought by training people to see things as they are. Mistakes have a very high social cost.
    - vineyardmike 4y ago
      Nah let’s not pretend it’s an intelligence thing. It’s not. Saying thanks to Alexa even when you know she’s not a person is not a sign of (lack of) intelligence. It’s a sign of manners. Don’t pat yourself on the back and say you’re intelligent because you express good manners less often than the next person. While I think it’s a bit silly that the researcher thought the chat bot was sentient, I don’t think it’s silly that the bot is perceived sufficiently human to be worth of manners if Alexa has been there for almost a decade. I don’t think we should teach people that bots are explicitly not worth a level of manners even at the cost of a strange life-like sympathy some may develop. It took us hundreds of years to teach people that human slavery was wrong and we should treat actual people with respect - and many are still pretty bad at it. Frankly society needs all the practice it can get. Society makes a ton of “factual mistakes”. Look at all of religion, most of politics, a huge amount of history books, much of lower-grade science education. It’s all full of “factual mistakes” that, today, probably have higher social costs than someone being nice to a robot. What would “training” even look like? Mandatory bot-sympathizing “reeducation” classes in elementary school? We couldn’t even teach people vaccines were safe and worth getting during a pandemic, but we’re going to trying to tell people not be be nice to human-like things? I can’t imagine how high the social costs to that will be.
hemloc_io 4y ago
If a portion of the population zealously believes that AIs are sentient we could see the same treatment of AI researchers or even software engineers as we've seen in other scientific pursuits by the religious. That's quite scary to me personally.
- JumpCrisscross 4y ago
  Chatbots could deliver the personal connection to a messiah figure so many modern religious movements emphasize. I would be surprised if we don't see a church founded around the ramblings of a discarded bot in our lifetimes.
  - malermeister 4y ago
    I think the more interesting (if highly unethical) thing would be to have a bot that is purposefully built to be a cult leader. Maybe train it on all sorts of scripture and see if you can get people to follow it...
  - mickotron 4y ago
    Well we already have QANon, so I guess your idea isn’t so far fetched after all.
  - colejohnson66 4y ago
    TempleOS,[0] by the infamous Terry Davis, featured a program called "After Egypt" that he would use to post text supposedly comming from God (or “space alian”).[1] [0]: https://en.wikipedia.org/wiki/TempleOS https://en.wikipedia.org/wiki/TempleOS [1]: https://news.ycombinator.com/threads?id=TerryADavis https://news.ycombinator.com/threads?id=TerryADavis
Aqueous 4y ago
This is part of why I think a ban on humanoid and even animal-like robots should be considered. Our intuitions are leading us astray here and we are going to start extending rights to things that don’t have subjective experience, but may insist to us that they do. It’s going to create intense confusion and an extremely inefficient bureaucracy that will stifle innovation.
- manquer 4y ago
  Humans are good at anthropomorphizing objects even if they are not humanoid. Not making it exactly human/animal like won't help that much, the underlying features that make the bot useful such as graceful moment, well enunciated speech or deep conversation will make it feel more human like no matter what form they take.
- adwn 4y ago
  > This is part of why I think a ban on humanoid and even animal-like robots should be considered. [...] It’s going to create intense confusion and an extremely inefficient bureaucracy that will stifle innovation. So you want introduce laws and regulations that stifle innovation in order to prevent inefficient bureaucracies that stifle innovation?
- anotheracctfo 4y ago
  Some idiot found the image of Jesus in a slice of toast, so lets ban toast.
- fzzzy 4y ago
  How do you know they don't have subjective experience?
  - mdp2021 4y ago
    Because we will have the blueprints, and because there is a definite possibility to have sane people test for emergent qualities.
  - ncmncm 4y ago
    Because there is no "they" to have it.
bee_rider 4y ago
I guess it is hard to say either way, but it could be that most of those people didn't really think the Boston Dynamics robot was sentient. Perhaps they were doing some performative empathy thing. Or they may have just been joking. Or maybe a couple people made comments like that for humor, and the rest were repost bots, trained on their comments. Let's not speculate about their internal motivation, it'll get recursive.
maxwell_smart 4y ago
On the flip side, I remember reading a story of a parent listening to their child bark orders at Siri/Alexa, and thinking that even though there's no need for politeness, they felt that maybe they should teach their child to treat it kindly and politely for the sake of the child's emotional development in terms of empathy.
- mdp2021 4y ago
  History of cultivation is full of examples in which the practitioner willingly attributes qualities to some object - to the intentional benefit of the practitioner, and never involving cognitive mistake.
morelisp 4y ago
> The danger is from people’s reactions to the AI’s. If only Google hadn't fired the AI ethicists who were actually competent enough to try to explain this part of it. https://twitter.com/mmitchell_ai/status/1535775360901451776 https://twitter.com/mmitchell_ai/status/1535775360901451776 Well, that's less exciting than "we got sentience!" and less profitable than "robot friends waiting now, only 99c/min", so here we are.

freebobafett 4y ago

mdp2021 4y ago

> It would be wise for technologists to pause and take this human reaction seriously It would be wise for /policymakers/ to pause and take this human reaction seriously... Education!... Lucidity in the population... You are reminding of the reactions from a YT-type crowd, "Do not hurt the robot!"; nearby a member notes "is that the profile of a senior FAANG engineer?"... The elephant in the room is flashing: we must assess Natural Intelligence as a problem well before Artificial Intelligence!

SpicyLemonZest 4y ago
I'm pretty skeptical of the idea of "education" here. Aesthetic preferences are still real preferences; it's not obvious to me that any amount of knowledge about the mechanics of a robot that behaves like a dog would or should convince people that it's not mean to kick it.
- mdp2021 4y ago
  That «Education!» was not about knowing robotics. It may not be obvious to some that robots are generally not animals; it is hopefully a very small amount of population, but logically it will exist. It is more possible that some people know that the robot is not an animal, but the perceptual patterns cause a sensational reaction in them, and they may prefer the sensation - an hijacking - to the factual truth (which includes that what superficially corresponded to patterns in the area of "«mean»" are engineering tests on implemented balance systems), overriding it. Some may not even have had to consider sensation (of course, the "sensation" of the common use of "sensational") topically, or may encourage it on a spontaneous value given to "feeling". There, instead, education leads. Then, again, there will be those that will make a statement of their «aesthetic preferences» as a demand, and not realize that they are personal, subjective, and that to be presented in the common arena grounds, good arguments are required. Education is a system to refine the mind to sophistication, which has the collective benefit of reducing noise. Which makes it a social concern. Which should get a priority proportional to what the news reveal.
  - SpicyLemonZest 4y ago
    What I don't agree with is the idea that there's an underlying "factual truth" here which exists in contrast or opposition to sensations. Quite a lot of educated people argue that morality is based on (or even consists entirely of) our subjective feelings about things.
    - mdp2021 4y ago
      You have misunderstood what was written: 'sensation' cannot be pluralized as the term expressing «the "sensation" of the common use of "sensational"». In what I have written, there are readable facts and cases of individuals subject to sensation that become passive to it, and lose the facts. Your «subjective feelings» are not a "good savage": they have their development, and they are integrated part of the whole inner system, in-development, in-refinement. And in fact, I was promoting education, which develops and refines you.
  - vineyardmike 4y ago
    > It is more possible that some people know that the robot is not an animal, but the perceptual patterns cause a sensational reaction in them, and they may prefer the sensation - an hijacking - to the factual truth Some people just practice morality in day to day life and apply it liberally, not frugally. And importantly, it isn’t an intellectual failing.
    - mdp2021 4y ago
      Hijacking, which is what was described, is a developmental failing. It is duly to note that the primary sense of intelligence is required in the correct process, for which the subject has to see the mechanism.
vineyardmike 4y ago
Hard disagree. It’s convenient to say people are too dumb but beyond the condescending nature of it, it’s a bad take. Even people who absolutely are educated and know better - senior google AI researchers - are sympathetic. People say thank you to Alexa even though no one is convinced Alexa is a person. The issue isn’t lack of awareness that it isn’t a person, it’s the silly little monkey brains we have that tells us that anything that is like a person should be treated like one even if we know it’s not. It’s like people who look at faces in clouds. No one thinks the clouds are people.
- mdp2021 4y ago
  > the silly little monkey brains we have ...are those things that with a trained prefrontal become the wise respectable dignified brains some have. And they may come to different conclusions (they do). Put it this way (to give your idea credit): the more intellectual sophistication there will be around, the less the suspect the one who treats scarecrows and mannequins «like a person ... even if we know it’s not», or thanks an 'address:port' type server for service, is confused. Back to the point though: if one called sentient what is not out of being «sympathetic», there would be a factual mistake, which is not justifiable, and which has to be fought by training people to see things as they are. Mistakes have a very high social cost.
  - vineyardmike 4y ago
    Nah let’s not pretend it’s an intelligence thing. It’s not. Saying thanks to Alexa even when you know she’s not a person is not a sign of (lack of) intelligence. It’s a sign of manners. Don’t pat yourself on the back and say you’re intelligent because you express good manners less often than the next person. While I think it’s a bit silly that the researcher thought the chat bot was sentient, I don’t think it’s silly that the bot is perceived sufficiently human to be worth of manners if Alexa has been there for almost a decade. I don’t think we should teach people that bots are explicitly not worth a level of manners even at the cost of a strange life-like sympathy some may develop. It took us hundreds of years to teach people that human slavery was wrong and we should treat actual people with respect - and many are still pretty bad at it. Frankly society needs all the practice it can get. Society makes a ton of “factual mistakes”. Look at all of religion, most of politics, a huge amount of history books, much of lower-grade science education. It’s all full of “factual mistakes” that, today, probably have higher social costs than someone being nice to a robot. What would “training” even look like? Mandatory bot-sympathizing “reeducation” classes in elementary school? We couldn’t even teach people vaccines were safe and worth getting during a pandemic, but we’re going to trying to tell people not be be nice to human-like things? I can’t imagine how high the social costs to that will be.

hemloc_io 4y ago

If a portion of the population zealously believes that AIs are sentient we could see the same treatment of AI researchers or even software engineers as we've seen in other scientific pursuits by the religious. That's quite scary to me personally.

JumpCrisscross 4y ago
Chatbots could deliver the personal connection to a messiah figure so many modern religious movements emphasize. I would be surprised if we don't see a church founded around the ramblings of a discarded bot in our lifetimes.
- malermeister 4y ago
  I think the more interesting (if highly unethical) thing would be to have a bot that is purposefully built to be a cult leader. Maybe train it on all sorts of scripture and see if you can get people to follow it...
- mickotron 4y ago
  Well we already have QANon, so I guess your idea isn’t so far fetched after all.
- colejohnson66 4y ago
  TempleOS,[0] by the infamous Terry Davis, featured a program called "After Egypt" that he would use to post text supposedly comming from God (or “space alian”).[1] [0]: https://en.wikipedia.org/wiki/TempleOS https://en.wikipedia.org/wiki/TempleOS [1]: https://news.ycombinator.com/threads?id=TerryADavis https://news.ycombinator.com/threads?id=TerryADavis

Aqueous 4y ago

This is part of why I think a ban on humanoid and even animal-like robots should be considered. Our intuitions are leading us astray here and we are going to start extending rights to things that don’t have subjective experience, but may insist to us that they do. It’s going to create intense confusion and an extremely inefficient bureaucracy that will stifle innovation.

manquer 4y ago
Humans are good at anthropomorphizing objects even if they are not humanoid. Not making it exactly human/animal like won't help that much, the underlying features that make the bot useful such as graceful moment, well enunciated speech or deep conversation will make it feel more human like no matter what form they take.
adwn 4y ago
> This is part of why I think a ban on humanoid and even animal-like robots should be considered. [...] It’s going to create intense confusion and an extremely inefficient bureaucracy that will stifle innovation. So you want introduce laws and regulations that stifle innovation in order to prevent inefficient bureaucracies that stifle innovation?
anotheracctfo 4y ago
Some idiot found the image of Jesus in a slice of toast, so lets ban toast.
fzzzy 4y ago
How do you know they don't have subjective experience?
- mdp2021 4y ago
  Because we will have the blueprints, and because there is a definite possibility to have sane people test for emergent qualities.
- ncmncm 4y ago
  Because there is no "they" to have it.

bee_rider 4y ago

I guess it is hard to say either way, but it could be that most of those people didn't really think the Boston Dynamics robot was sentient. Perhaps they were doing some performative empathy thing. Or they may have just been joking. Or maybe a couple people made comments like that for humor, and the rest were repost bots, trained on their comments. Let's not speculate about their internal motivation, it'll get recursive.

maxwell_smart 4y ago

On the flip side, I remember reading a story of a parent listening to their child bark orders at Siri/Alexa, and thinking that even though there's no need for politeness, they felt that maybe they should teach their child to treat it kindly and politely for the sake of the child's emotional development in terms of empathy.

mdp2021 4y ago
History of cultivation is full of examples in which the practitioner willingly attributes qualities to some object - to the intentional benefit of the practitioner, and never involving cognitive mistake.

morelisp 4y ago

> The danger is from people’s reactions to the AI’s. If only Google hadn't fired the AI ethicists who were actually competent enough to try to explain this part of it. https://twitter.com/mmitchell_ai/status/1535775360901451776 https://twitter.com/mmitchell_ai/status/1535775360901451776 Well, that's less exciting than "we got sentience!" and less profitable than "robot friends waiting now, only 99c/min", so here we are.