5 ms·

This to me is the real problem, "AI safety" can mean about a million things but it's always just whatever is most convenient for the speaker. I'm convinced huma

by nmilo 3y ago

This to me is the real problem, "AI safety" can mean about a million things but it's always just whatever is most convenient for the speaker. I'm convinced human language/English is not enough to discuss AI problems, the words are way too loaded with anthropomorphized meanings and cultural meanings to discuss the topic in a rational way at all. The words are just too easy to twist.

HPMOR 3y ago
Linguistics experts will have a fun time untangling the pre-AI language from the post-AI era.
- tkgally 3y ago
  As it happens, I’m now teaching an undergraduate course at the University of Tokyo titled “The Meaning of Language in the Age of AI.” The students—who come from various countries and majors—and I are discussing how theories of human language will have to change now that humans interact not only with each other but also with AI. I don’t think we’ll have any full-fledged theories ready by the end of the semester, but some points to consider are emerging. One is that aspects of language use that are affected by the social status, personal identities, and consciousness of the speakers and listeners—politeness, formality, forms of address, pronoun reference, etc.—will have to be rethought now that people are increasingly conversing with entities that have no social status, personal identity, or consciousness (not yet, at least). And, yes, thinking about all this is a lot of fun.
  - palata 3y ago
    Not sure if I am completely off-topic here, but I was recently reading this essay by Bruce Schneier: https://www.schneier.com/blog/archives/2023/12/ai-and-trust.html https://www.schneier.com/blog/archives/2023/12/ai-and-trust...., where he mentions a related risk: because we now can "talk" to those LLMs in a similar way that we talk to people, it makes it harder for us to realize that they are not people. And so it's easier to abuse our trust. I guess my question here would be: will human language have to change because of interactions with LLMs, or is the whole point of LLMs that it does not have to change (and therefore humans will have to learn not to be abused by the machines)? Because we already have languages to talk to machines: those are programming languages, which are designed to be unambiguous. The problem I see is not that we don't know how to talk to machines, but rather that we now have machines that are really good at pretending they are not machines. Not sure if I am making any sense at all :-).
    - tkgally 3y ago
      You are making a lot of sense to me. Thanks for the link to that essay, too. The point about programming-language nonambiguity is a good one. After I started using ChatGPT a year ago, it took me a while to realize that I didn’t have to be careful about my spelling, capitalization, punctuation, etc. It turned out to be good at interpreting the intention of sloppily written prompts. And it never pointed out or complained about my mistakes, either—another difference from humans.
      - palata 3y ago
        > And it never pointed out or complained about my mistakes, either—another difference from humans. Yeah that is a big difference indeed: a human has some notion of confidence about what they know. Not every human expresses it the same way: a scientist will say "I am pretty sure that climate change is a consequence of human activity", meaning "well, maybe we all live in the Matrix and gravity is an illusion, but from what we know, we are pretty sure". On the other end of the spectrum, some will say "I know for a fact that God exists, because it is written in a very old book". But both have a consistent notion of confidence about what they (think they) know and what they don't know. An LLM does not have that. An LLM is not critical. It cannot conclude stuff like "I have to take into account that this text was written in a period of war and the author is not neutral, so there is probably a bias". It can generate text that pretends it does, but it remains a generated text, it is not a critical analysis. Also it cannot be offended by something you write, even though it could generate text that pretends it is. So yeah, it can probably be engineered to never/always point out or complain about mistakes, with the caveat that it does not have an "idea" of what a mistake is. It just generates text that pretends it does. > It turned out to be good at interpreting the intention of sloppily written prompts. In the context of a conversation with a human (you), who is very good at navigating ambiguous communications. But if you tried to describe an algorithm with an ambiguous natural language, you may quickly have problems. And errors accumulate, so it gets worse when the task becomes more complex. Which is an interesting thought: it may make simple tasks simpler, but it's not at all a given that it scales.
tkgally 3y ago
One of the students in a course I’m teaching on language and AI (mentioned in another comment here) wrote something similar in a homework assignment the other day. We had discussed the paper “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness” [1]. The student wrote: “One of the questions that I have been wondering about is whether there have been any discussions exploring the creation of a distinct category, akin to but different from consciousness, that better captures the potential for AI to be sentient. While ‘consciousness’ is a familiar term applicable, to some extent, beyond the human brain, given the associated difficulties, it might be sensible to establish a separate definition to distinguish these two categories.” Probably new terms should be coined not only for AI “consciousness” but for other aspects of what they are and do as well. [1] https://arxiv.org/abs/2308.08708 https://arxiv.org/abs/2308.08708
- nmilo 3y ago
  Agreed. Even "AI" is a bit loaded; we had half a century of Terminator, Asimov, 2001, etc. We should call them by implementation; neural networks, transformers, language models, etc. The plus side is that it's harder to generalize statements about all three---there is very little to generalize in the first place---and that asking "is a transformer conscious" at least sounds more ridiculous than asking "is AI conscious."
  - BoiledCabbage 3y ago
    But that misses the point, via the same old argument "Does a submarine swim"? Or the now less loaded "Does a plane fly". Neither generalizes very well w.r.t. how a fish or a bird does it. Nothing really generalizes between the natural version and the mechanical one. But it's also clear that it's irrelevant. Trying to define useful behavior by categorizing implementation, will only give you categories of implementation. It's clear by behavior that what a submarine does and what a bird does are both for all intents and purposes meeting the definition of what we'd want something to do in order to fly or swim, even though they is no commonality between them and the version in nature. So by your definition finding it ridiculous to ask "is a transformer conscious" is about as ridiculous as asking the very specific "does a plan flap it's wings"? The point is not to define it by it's implementation, but by what behaviors it exhibits. If in the future there is no way for me the exterior to distinguish a next gen LLM from a human, animal or other conscious being it becomes irrelevant if it's implemented with transformers. However, while I won't go down that path here, the argument goes even deeper as technically we would expect differences so even if it's not identical to a human it still can be conscious. Just like a dog is conscious but behaves differently than a person does.
yreg 3y ago
If english is good enough to talk about what genes 'want' then it's good enough to talk about what AI 'wants'.

by nmilo 3y ago

HPMOR 3y ago

Linguistics experts will have a fun time untangling the pre-AI language from the post-AI era.

tkgally 3y ago
As it happens, I’m now teaching an undergraduate course at the University of Tokyo titled “The Meaning of Language in the Age of AI.” The students—who come from various countries and majors—and I are discussing how theories of human language will have to change now that humans interact not only with each other but also with AI. I don’t think we’ll have any full-fledged theories ready by the end of the semester, but some points to consider are emerging. One is that aspects of language use that are affected by the social status, personal identities, and consciousness of the speakers and listeners—politeness, formality, forms of address, pronoun reference, etc.—will have to be rethought now that people are increasingly conversing with entities that have no social status, personal identity, or consciousness (not yet, at least). And, yes, thinking about all this is a lot of fun.
- palata 3y ago
  Not sure if I am completely off-topic here, but I was recently reading this essay by Bruce Schneier: https://www.schneier.com/blog/archives/2023/12/ai-and-trust.html https://www.schneier.com/blog/archives/2023/12/ai-and-trust...., where he mentions a related risk: because we now can "talk" to those LLMs in a similar way that we talk to people, it makes it harder for us to realize that they are not people. And so it's easier to abuse our trust. I guess my question here would be: will human language have to change because of interactions with LLMs, or is the whole point of LLMs that it does not have to change (and therefore humans will have to learn not to be abused by the machines)? Because we already have languages to talk to machines: those are programming languages, which are designed to be unambiguous. The problem I see is not that we don't know how to talk to machines, but rather that we now have machines that are really good at pretending they are not machines. Not sure if I am making any sense at all :-).
  - tkgally 3y ago
    You are making a lot of sense to me. Thanks for the link to that essay, too. The point about programming-language nonambiguity is a good one. After I started using ChatGPT a year ago, it took me a while to realize that I didn’t have to be careful about my spelling, capitalization, punctuation, etc. It turned out to be good at interpreting the intention of sloppily written prompts. And it never pointed out or complained about my mistakes, either—another difference from humans.
    - palata 3y ago
      > And it never pointed out or complained about my mistakes, either—another difference from humans. Yeah that is a big difference indeed: a human has some notion of confidence about what they know. Not every human expresses it the same way: a scientist will say "I am pretty sure that climate change is a consequence of human activity", meaning "well, maybe we all live in the Matrix and gravity is an illusion, but from what we know, we are pretty sure". On the other end of the spectrum, some will say "I know for a fact that God exists, because it is written in a very old book". But both have a consistent notion of confidence about what they (think they) know and what they don't know. An LLM does not have that. An LLM is not critical. It cannot conclude stuff like "I have to take into account that this text was written in a period of war and the author is not neutral, so there is probably a bias". It can generate text that pretends it does, but it remains a generated text, it is not a critical analysis. Also it cannot be offended by something you write, even though it could generate text that pretends it is. So yeah, it can probably be engineered to never/always point out or complain about mistakes, with the caveat that it does not have an "idea" of what a mistake is. It just generates text that pretends it does. > It turned out to be good at interpreting the intention of sloppily written prompts. In the context of a conversation with a human (you), who is very good at navigating ambiguous communications. But if you tried to describe an algorithm with an ambiguous natural language, you may quickly have problems. And errors accumulate, so it gets worse when the task becomes more complex. Which is an interesting thought: it may make simple tasks simpler, but it's not at all a given that it scales.

tkgally 3y ago

One of the students in a course I’m teaching on language and AI (mentioned in another comment here) wrote something similar in a homework assignment the other day. We had discussed the paper “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness” [1]. The student wrote: “One of the questions that I have been wondering about is whether there have been any discussions exploring the creation of a distinct category, akin to but different from consciousness, that better captures the potential for AI to be sentient. While ‘consciousness’ is a familiar term applicable, to some extent, beyond the human brain, given the associated difficulties, it might be sensible to establish a separate definition to distinguish these two categories.” Probably new terms should be coined not only for AI “consciousness” but for other aspects of what they are and do as well. [1] https://arxiv.org/abs/2308.08708 https://arxiv.org/abs/2308.08708

nmilo 3y ago
Agreed. Even "AI" is a bit loaded; we had half a century of Terminator, Asimov, 2001, etc. We should call them by implementation; neural networks, transformers, language models, etc. The plus side is that it's harder to generalize statements about all three---there is very little to generalize in the first place---and that asking "is a transformer conscious" at least sounds more ridiculous than asking "is AI conscious."
- BoiledCabbage 3y ago
  But that misses the point, via the same old argument "Does a submarine swim"? Or the now less loaded "Does a plane fly". Neither generalizes very well w.r.t. how a fish or a bird does it. Nothing really generalizes between the natural version and the mechanical one. But it's also clear that it's irrelevant. Trying to define useful behavior by categorizing implementation, will only give you categories of implementation. It's clear by behavior that what a submarine does and what a bird does are both for all intents and purposes meeting the definition of what we'd want something to do in order to fly or swim, even though they is no commonality between them and the version in nature. So by your definition finding it ridiculous to ask "is a transformer conscious" is about as ridiculous as asking the very specific "does a plan flap it's wings"? The point is not to define it by it's implementation, but by what behaviors it exhibits. If in the future there is no way for me the exterior to distinguish a next gen LLM from a human, animal or other conscious being it becomes irrelevant if it's implemented with transformers. However, while I won't go down that path here, the argument goes even deeper as technically we would expect differences so even if it's not identical to a human it still can be conscious. Just like a dog is conscious but behaves differently than a person does.

yreg 3y ago

If english is good enough to talk about what genes 'want' then it's good enough to talk about what AI 'wants'.