6 ms·

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not eve

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

pixl97 3y ago
A consistent stateful experience may be needed, but not sure about continuous time. I mean human consciousness doesn't do that.
- jncfhnb 3y ago
  I would argue it needs to be at least somewhat continuous. Perhaps discrete on some granularity but if something is just a function waiting to be called it’s not an intelligent entity. The entity is the calling itself.
- haswell 3y ago
  Human consciousness does though, e.g. the flow state. F1 drivers are a good example. We tend to not experience continuous time because we repeatedly get distracted by our thoughts, but entering the continuous stream of now is possible with practice and is one of the aims of many meditators.
  - int_19h 3y ago
    Human consciousness is capable of it, but since most humans aren't in it much of the time, it would appear that it's not a prerequisite for true sentience.
  - krisoft 3y ago
    What does it mean to “experience continous time”? How do you know that F1 drivers experience it?
wongarsu 3y ago
You could imagine an LLM being called in a loop with a prompt like You observe: {new input} You remember: {from previous output} React to this in the following format: My inner thoughts: [what do you think about the current state] I want to remember: [information that is important for your future actions] Things I do: [Actions you want to take] Things I say: [What I want to say to the user] ... Not sure if that would qualify as an AGI as we currently define it. Given a sufficiently good LLM with good reasoning capabilities such a setup might be able to It would be able to do many of the things we currently expect AGIs to be able to do (given a sufficiently good LLM with good reasoning capabilities), including planning and learning new knowledge and new skills (by collecting and storing positive and negative examples in its "memory"). But its learning would be limited, and I'm sure as soon as it exists we would agree that it's not AGI
- lgas 3y ago
  It works better to give it access to functions to call for actions and remembering stuff, but this approach does provide some interesting results.
- sophiabits 3y ago
  This already exists (in a slightly different prompt format); it's the underlying idea behind ReAct: https://react-lm.github.io https://react-lm.github.io As you say, I'm skeptical this counts as AGI. Although I admit that I don't have a particularly rock solid definition of what _would_ constitute true AGI.
- marquisdepolis 3y ago
  (Author here). I tried creating something similar in order to solve wordle etc, and the interesting part is that it is insufficient still. That's part of the mystery.
PopePompus 3y ago
I don't understand why you believe it must experience continuous time. If you had a system which clearly could reason, which could learn new tasks on its own, which didn't hallucinate any more than humans do, but it was only active for the period required for it to complete an assigned task, and was completely dormant otherwise, why would that dormant period disqualify it as AGI? I agree that such a system should probably not be considered conscious, but I think it's an open question whether or not consciousness is required for intelligence.
- kaibee 3y ago
  I think its note worthy that humans actually fail this test... We have to go dormant for 8 hours every day.
  - Hunpeter 3y ago
    Yes, but our brain is still working and processing information at those times as well, isn't it? Even if not in the same way as it does when we're conscious.
    - PopePompus 3y ago
      What about general anesthesia? I had a major operation during which most of my brain was definitely offline for at least 8 hours.
      - autoexec 3y ago
        Anesthesia shouldn't take your brain offline. It just makes you unconscious, paralyzes you, and gives you amnesia. Your brain is still active under general anesthesia. What you were thinking or feeling for those 8 hours was just forgotten.
        crest 3y ago
        [citation needed].
adrianN 3y ago
I try my best not to experience continuous time for at least eight hours a day.
- jncfhnb 3y ago
  Then for at least eight hours a day you don’t qualify as a generally intelligent system.
  - card_zero 3y ago
    If I spend some amount of the day bathing, some amount of it scratching, some amount of it thinking vaguely about racoons without any clear conclusions, and a lot of it drinking tea, I wonder how many seconds remain during which I qualified as generally intelligent.
    - jncfhnb 3y ago
      I feel you qualify during all of those waking seconds
      - card_zero 3y ago
        Racoons are said to be intelligent because they're good at opening locks. On the other hand, when they have food and are within ten feet of a pool of water, they will dip the food in the water and rub it between their paws for no reason. They can reason about the locks, but not about the food. Meanwhile, I in theory can reason about anything, but in practice I wouldn't count on it. Whereas an LLM can't reason, but it's very sharp and always ready to react appropriately.
        jncfhnb 3y ago
        I’m not familiar with this raccoon behavior but it sure doesn’t sound like it’s done without reason. An LLM is never ready to react to anything because it’s just a matrix that needs a higher level system to invoke it.
naasking 3y ago
Some good prompt-reply interactions are probably fed back in to subsequent training runs, so they're still stateful/have memory in a way, there's just a long delay.
- jncfhnb 3y ago
  That’s not the AGI’s state. That’s just some past information.
  - naasking 3y ago
    State is a function of accumulated past information.
    - jncfhnb 3y ago
      State is a function of accumulated past. That does not mean that having some past written down makes you stateful. A stateful thing has to incorporate the ongoing changes.
      - naasking 3y ago
        Which is what I described: some successful prompt-replies are fed back into subsequent training runs.
        jncfhnb 3y ago
        No… that implies the model never has active state and is being replaced with a different, stateless model. This is similar to the difference between Actor.happy = True And Actor = happier(Actor)
        naasking 3y ago
        Both of your examples are stateful systems from the outside, given a suitable choice of timeframe, the latter one is just how purely functional systems represent state. Theoretically they can simulate each other, and the endpoint you use to access Actor will still reference the latest Actor. The only reason you're calling them different is because you insist on using a specific timeframe to exclude considering one as stateful, and I'm pointing out that that isn't strictly necessary.

pixl97 3y ago

A consistent stateful experience may be needed, but not sure about continuous time. I mean human consciousness doesn't do that.

jncfhnb 3y ago
I would argue it needs to be at least somewhat continuous. Perhaps discrete on some granularity but if something is just a function waiting to be called it’s not an intelligent entity. The entity is the calling itself.
haswell 3y ago
Human consciousness does though, e.g. the flow state. F1 drivers are a good example. We tend to not experience continuous time because we repeatedly get distracted by our thoughts, but entering the continuous stream of now is possible with practice and is one of the aims of many meditators.
- int_19h 3y ago
  Human consciousness is capable of it, but since most humans aren't in it much of the time, it would appear that it's not a prerequisite for true sentience.
- krisoft 3y ago
  What does it mean to “experience continous time”? How do you know that F1 drivers experience it?

wongarsu 3y ago

You could imagine an LLM being called in a loop with a prompt like You observe: {new input} You remember: {from previous output} React to this in the following format: My inner thoughts: [what do you think about the current state] I want to remember: [information that is important for your future actions] Things I do: [Actions you want to take] Things I say: [What I want to say to the user] ... Not sure if that would qualify as an AGI as we currently define it. Given a sufficiently good LLM with good reasoning capabilities such a setup might be able to It would be able to do many of the things we currently expect AGIs to be able to do (given a sufficiently good LLM with good reasoning capabilities), including planning and learning new knowledge and new skills (by collecting and storing positive and negative examples in its "memory"). But its learning would be limited, and I'm sure as soon as it exists we would agree that it's not AGI

lgas 3y ago
It works better to give it access to functions to call for actions and remembering stuff, but this approach does provide some interesting results.
sophiabits 3y ago
This already exists (in a slightly different prompt format); it's the underlying idea behind ReAct: https://react-lm.github.io https://react-lm.github.io As you say, I'm skeptical this counts as AGI. Although I admit that I don't have a particularly rock solid definition of what _would_ constitute true AGI.
marquisdepolis 3y ago
(Author here). I tried creating something similar in order to solve wordle etc, and the interesting part is that it is insufficient still. That's part of the mystery.

PopePompus 3y ago

I don't understand why you believe it must experience continuous time. If you had a system which clearly could reason, which could learn new tasks on its own, which didn't hallucinate any more than humans do, but it was only active for the period required for it to complete an assigned task, and was completely dormant otherwise, why would that dormant period disqualify it as AGI? I agree that such a system should probably not be considered conscious, but I think it's an open question whether or not consciousness is required for intelligence.

kaibee 3y ago
I think its note worthy that humans actually fail this test... We have to go dormant for 8 hours every day.
- Hunpeter 3y ago
  Yes, but our brain is still working and processing information at those times as well, isn't it? Even if not in the same way as it does when we're conscious.
  - PopePompus 3y ago
    What about general anesthesia? I had a major operation during which most of my brain was definitely offline for at least 8 hours.
    - autoexec 3y ago
      Anesthesia shouldn't take your brain offline. It just makes you unconscious, paralyzes you, and gives you amnesia. Your brain is still active under general anesthesia. What you were thinking or feeling for those 8 hours was just forgotten.
      - crest 3y ago
        [citation needed].

adrianN 3y ago

I try my best not to experience continuous time for at least eight hours a day.

jncfhnb 3y ago
Then for at least eight hours a day you don’t qualify as a generally intelligent system.
- card_zero 3y ago
  If I spend some amount of the day bathing, some amount of it scratching, some amount of it thinking vaguely about racoons without any clear conclusions, and a lot of it drinking tea, I wonder how many seconds remain during which I qualified as generally intelligent.
  - jncfhnb 3y ago
    I feel you qualify during all of those waking seconds
    - card_zero 3y ago
      Racoons are said to be intelligent because they're good at opening locks. On the other hand, when they have food and are within ten feet of a pool of water, they will dip the food in the water and rub it between their paws for no reason. They can reason about the locks, but not about the food. Meanwhile, I in theory can reason about anything, but in practice I wouldn't count on it. Whereas an LLM can't reason, but it's very sharp and always ready to react appropriately.
      - jncfhnb 3y ago
        I’m not familiar with this raccoon behavior but it sure doesn’t sound like it’s done without reason. An LLM is never ready to react to anything because it’s just a matrix that needs a higher level system to invoke it.

naasking 3y ago

Some good prompt-reply interactions are probably fed back in to subsequent training runs, so they're still stateful/have memory in a way, there's just a long delay.

jncfhnb 3y ago
That’s not the AGI’s state. That’s just some past information.
- naasking 3y ago
  State is a function of accumulated past information.
  - jncfhnb 3y ago
    State is a function of accumulated past. That does not mean that having some past written down makes you stateful. A stateful thing has to incorporate the ongoing changes.
    - naasking 3y ago
      Which is what I described: some successful prompt-replies are fed back into subsequent training runs.
      - jncfhnb 3y ago
        No… that implies the model never has active state and is being replaced with a different, stateless model. This is similar to the difference between Actor.happy = True And Actor = happier(Actor)
        naasking 3y ago
        Both of your examples are stateful systems from the outside, given a suitable choice of timeframe, the latter one is just how purely functional systems represent state. Theoretically they can simulate each other, and the endpoint you use to access Actor will still reference the latest Actor. The only reason you're calling them different is because you insist on using a specific timeframe to exclude considering one as stateful, and I'm pointing out that that isn't strictly necessary.