3 ms·

This arguments is made every singe time a new LLM article gets posted and I'm not sure it's really adding anything to the conversation. Everyone understands lan

by Me1000 2y ago

This arguments is made every singe time a new LLM article gets posted and I'm not sure it's really adding anything to the conversation. Everyone understands language models are not human, there's no need to add a philosophical argument because you don't like the word choice. "Characters" in any other context are also by definition not curious. They're not open-minded or thoughtful either. Characters in a book are not people, they don't have thoughts, they are whatever the author put on paper. Yet we still use these words to describe them, as if the characters' consciousness and motivations existed beyond the paper on which they're described. It becomes really hard to talk about these traits without using words like curiosity and thoughtfulness. No one thinks the characters in a fictional book are real people.

HarHarVeryFunny 2y ago
I'm not sure that "these are just characters, your expectations are too high" is totally valid here, especially as Anthropic are claiming their LLM progression is on the path to AGI. Any true human-level (or even rat-level for that matter) AGI would need to have actual innate traits such as curiosity to drive lifelong exploration and learning. That seems a pretty minimal bar to meet to claim that something is AGI. I suppose in context of an article about giving Claude a character (having it play a character) then we need to interpret "having a trait" as "playing a character having a trait", because it certainly is very different from actually having it.
- pinneycolton 2y ago
  Some humans who are not trained that curiosity produces beneficial outcomes are often not all that curious. It is difficult to know what is truly innate vs. the result of conditioning. It is also difficult to know what is the result of an absence of opposing conditioning. If I was raised in a poor area with high crime and little economic opportunity, my conditioning would be quite different. How would that have changed my behavior? Could curiosity get me in more trouble than it's worth? Could I observe that and adjust my own behavior as a result? We are all, to some extent, a product of our environment (training data). I wasn't raised in that type of area, but does that mean my own intellectual curiosity is more innate? Or does it mean it is less innate? I could argue that both ways.
  - HarHarVeryFunny 2y ago
    > Some humans who are not trained that curiosity produces beneficial outcomes are often not all that curious. Innate traits such as curiosity & boredom are things that we are born with, not learnt. The reason evolution has selected for these innate traits is because there is a benefit (encouraging exploration and learning which help survival), but you don't need to be aware of the benefits to experience and act on boredom or curiosity. Innate behaviors can certainly be reinforced, or suppressed to some degree, by experience.
- abrichr 2y ago
  > Any true human-level (or even rat-level for that matter) AGI would need to have actual innate traits such as curiosity to drive lifelong exploration and learning. While perhaps not "true" intelligence (whatever that is), with LLMs this can be emulated with a prompt: > You have an innate curiosity to drive lifelong exploration and learning. ... If the behavior of the system is indistinguishable from "true" intelligence, the distinction becomes a philosophical one.
  - HarHarVeryFunny 2y ago
    How can you emulate learning when LLMs have no way to update their weights? LLMs don't even know what they do and don't know (hence hallucinations), so how would they even realize that something was new in the first place? I could maybe see chatbots storing everything you ever said in a vector database for future lookup, but that's just memory.
    - falcor84 2y ago
      There's nothing stopping you as an LLM system designer from having it "go to sleep" once a day and use that downtime to consolidate the memory in the vector database into fine-tuning of the weights.

by Me1000 2y ago

HarHarVeryFunny 2y ago

I'm not sure that "these are just characters, your expectations are too high" is totally valid here, especially as Anthropic are claiming their LLM progression is on the path to AGI. Any true human-level (or even rat-level for that matter) AGI would need to have actual innate traits such as curiosity to drive lifelong exploration and learning. That seems a pretty minimal bar to meet to claim that something is AGI. I suppose in context of an article about giving Claude a character (having it play a character) then we need to interpret "having a trait" as "playing a character having a trait", because it certainly is very different from actually having it.

pinneycolton 2y ago
Some humans who are not trained that curiosity produces beneficial outcomes are often not all that curious. It is difficult to know what is truly innate vs. the result of conditioning. It is also difficult to know what is the result of an absence of opposing conditioning. If I was raised in a poor area with high crime and little economic opportunity, my conditioning would be quite different. How would that have changed my behavior? Could curiosity get me in more trouble than it's worth? Could I observe that and adjust my own behavior as a result? We are all, to some extent, a product of our environment (training data). I wasn't raised in that type of area, but does that mean my own intellectual curiosity is more innate? Or does it mean it is less innate? I could argue that both ways.
- HarHarVeryFunny 2y ago
  > Some humans who are not trained that curiosity produces beneficial outcomes are often not all that curious. Innate traits such as curiosity & boredom are things that we are born with, not learnt. The reason evolution has selected for these innate traits is because there is a benefit (encouraging exploration and learning which help survival), but you don't need to be aware of the benefits to experience and act on boredom or curiosity. Innate behaviors can certainly be reinforced, or suppressed to some degree, by experience.
abrichr 2y ago
> Any true human-level (or even rat-level for that matter) AGI would need to have actual innate traits such as curiosity to drive lifelong exploration and learning. While perhaps not "true" intelligence (whatever that is), with LLMs this can be emulated with a prompt: > You have an innate curiosity to drive lifelong exploration and learning. ... If the behavior of the system is indistinguishable from "true" intelligence, the distinction becomes a philosophical one.
- HarHarVeryFunny 2y ago
  How can you emulate learning when LLMs have no way to update their weights? LLMs don't even know what they do and don't know (hence hallucinations), so how would they even realize that something was new in the first place? I could maybe see chatbots storing everything you ever said in a vector database for future lookup, but that's just memory.
  - falcor84 2y ago
    There's nothing stopping you as an LLM system designer from having it "go to sleep" once a day and use that downtime to consolidate the memory in the vector database into fine-tuning of the weights.