3 ms·

What I can’t figure out is who is actually bothered by anything that an AI system output? Like it’s a program. And it printed a string to standard out. Are ther

What I can’t figure out is who is actually bothered by anything that an AI system output? Like it’s a program. And it printed a string to standard out. Are there really humans out there who are genuinely upset, or bothered by what some program prints to standard out? I feel like this is all entirely manufactured, the supposed “controversy” referenced in this article

antonvs 4y ago
It all seems to be predicated on the argument that many users will not understand that they're not dealing with a sentient entity, and that this can have negative results. In the OP this is most clearly summarized in this paragraph: > Human beings have historically tended to anthropomorphize natural phenomena, animals and deities. But anthropomorphizing software is not harmless. In 1966 Joseph Weizenbaum created ELIZA, a pioneer chatbot designed to imitate a therapist, but ended up regretting it after seeing many users take it seriously, even after Weizenbaum explained to them how it worked. The fictitious “I” has been persistent throughout our cultural artifacts. Stanley’s Kubrick HAL 9000 (“2001: A Space Odyssey”) and Spike Jonze’s Samantha (“Her”) point at two lessons that developers don’t seem to have taken to heart: first, that the bias towards anthropomorphization is so strong to seem irresistible; and second, that if we lean into it instead of adopting safeguards, it leads to outcomes ranging from the depressing to the catastrophic. I think it's true there are some potential problems here, but it's not clear that banning AI's use of "I" is really going to make much difference. People will still be getting authoritative-seeming responses from AIs, and are probably still going to anthropomorphize them anyway, if they're that way inclined. This kind of thing is perhaps better dealt with by education, public awareness campaigns, etc.

antonvs 4y ago

It all seems to be predicated on the argument that many users will not understand that they're not dealing with a sentient entity, and that this can have negative results. In the OP this is most clearly summarized in this paragraph: > Human beings have historically tended to anthropomorphize natural phenomena, animals and deities. But anthropomorphizing software is not harmless. In 1966 Joseph Weizenbaum created ELIZA, a pioneer chatbot designed to imitate a therapist, but ended up regretting it after seeing many users take it seriously, even after Weizenbaum explained to them how it worked. The fictitious “I” has been persistent throughout our cultural artifacts. Stanley’s Kubrick HAL 9000 (“2001: A Space Odyssey”) and Spike Jonze’s Samantha (“Her”) point at two lessons that developers don’t seem to have taken to heart: first, that the bias towards anthropomorphization is so strong to seem irresistible; and second, that if we lean into it instead of adopting safeguards, it leads to outcomes ranging from the depressing to the catastrophic. I think it's true there are some potential problems here, but it's not clear that banning AI's use of "I" is really going to make much difference. People will still be getting authoritative-seeming responses from AIs, and are probably still going to anthropomorphize them anyway, if they're that way inclined. This kind of thing is perhaps better dealt with by education, public awareness campaigns, etc.