5 ms·

I agree, I suspect AGI is possible right now with a similar system only slightly more sophisticated than this one. The right "glue" for existing models, and plu

by _yb2s 4y ago

I agree, I suspect AGI is possible right now with a similar system only slightly more sophisticated than this one. The right "glue" for existing models, and plugins to existing data sources all coordinated in a system. GTP-4 would do the managing, and handling, and some simple template API and handler script would allow it to call instances of itself or other models, track recursion depth, and automatically remind GTP-4 to stay on track and always use the correct 'templates' for requests. It could also remind it to create a condensed summary of it's goals and state that gets repeated back automatically to act as long term memory, and increase the effective context window size (edit: this is single variable external memory). I am afraid to explain this, because I have tried a preliminary version of it that I supervised step by step, and it seems to work. I think it is obvious enough that I won't have been the only one to think of it, so it would be safer to put the information out there so people can prepare. I see a big disconnect on here between people saying GPT-4 can't do things like this and is just a "stochastic parrot" or "glorified autocomplete," and people posting logs and summaries of it solving unexpectedly hard problems outside of any conceivable training set. My theory is that this disconnect is due to three major factors: * People confusing GPT-4 and GPT-3, as both are called "chatGTP" and most people haven't actually used GPT-4 because it requires a paid subscription, and don't realize how much better it is * Most popular conceptual explanations about how these models work imply that these actually observed capabilities should be fundamentally impossible * Expectations from movies, etc. about what AGI will be like, e.g. that it will never get confused or make mistakes, or that it won't have major shortcomings in specific areas. In practice this doesn't seem to limit it because it recognizes and fixes its mistakes automatically when it sees feedback (e.g. in programming)

basch 4y ago
It also, in my pay opinion, needs a memory store. I need to be able to save something to a variable and call back that exact variable. Right now, because it’s just pure text input of the whole conversation, it can forget or corrupt it’s “memory”. Even something like writing a movie script doesn’t work if it constantly forgets characters names or settings or plot points.
- _yb2s 4y ago
  My description above includes a memory store. It is a single variable that it can save and recover, and will automatically get sent back to it at regular intervals if it forgets it saved anything. Only a single text string variable is needed, as GTP-4 is smart enough to realize it can compress, pickle, etc. as many things as it needs into that.
  - alchemist1e9 4y ago
    I think we might be working something a bit similar and I agree memory is an issue. However it’s not really AGI that will emerge from this. It’s looking very useful but it’s still observable but it’s not a “mind” like an AGI. To add to your comments, I suggest that local vector store of embedding vectors of local content is how I’m going with memory issue. Langchain like. That way all previous progress is tracked and retrievable. That way the system can retrieve all it’s “memories” itself. The recursive multi agent pattern is big deal in my opinion.
    - nomel 4y ago
      > but it’s not a “mind” like an AGI. Please define this. ;)
      - WA 4y ago
        Please define AGI first ;)
        nomel 4y ago
        I asked first?
- basch 4y ago
  lay*
- pmoriarty 4y ago
  Langchain[1] can be used to add memory to an LLM. [1] - https://www.pinecone.io/learn/langchain-conversational-memory/ https://www.pinecone.io/learn/langchain-conversational-memor...
brotchie 4y ago
Yep, the tipping point for me was seeing a demo of GPT-4 writing code, then being shown exceptions where the code failed, and successfully fixing the code given that feedback. Seems that for any task where you can generate an error signal with some information ("ok, what you just tried to do didn't succeed, here's some information about why it didn't succeed"), GPT-4 can generally handle this information to fix the error, and or seek out more information via tools to move forward on a task. Only thing that's really missing is somebody to crack the back of a memory module (perhaps some trick with embeddings and a vector db) is all it takes for this to be AGI.
- alchemist1e9 4y ago
  > Only thing that's really missing is somebody to crack the back of a memory module (perhaps some trick with embeddings and a vector db) is all it takes for this to be AGI. Here to say I agree with embeddings and vector db for memory systems. However also to disagree this can lead to AGI.
marcosdumay 4y ago
I believe you will quickly run into a barrier due to the small size of the context window. And increasing the context window gets harder and harder the larger it is. There's a sibling talking about internal memory. That's how our brains solve the issue. AFAIK, nobody knows how to train something like it.
- wyager 4y ago
  I wouldn't be surprised if AI research moves towards training models to use bolt-on peripherals like scratchpad memory. Transformers showed us an interface for addressing, so I wouldn't be surprised if someone figures out a way to use a similar addressing scheme and made it read/write.
- _yb2s 4y ago
  You are right, but it is unclear to me where this limitation will fall in terms of actual abilities. GPT-4 can divide large tasks up into logical steps and instructions that can be worked on independently by other instances, and it can create "compressed" condensed explanations of it's own state that can be stored externally and repeated back to it, or passed back and forth between instances. With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on.
  - IanCal 4y ago
    > With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on. Also that means you can just iterate with different contexts until you can deal with the problem. How many problems need 50 pages of context on top of what gpt4 knows for solving the next step?
    - Workaccount2 4y ago
      >How many problems need 50 pages of context on top of what gpt4 knows for solving the next step? Perhaps the unknown unkowns? I envision a scenario where a super intelligent AI seemingly runs off the rails and becomes obsessed with overtly complex problems that are totally intractable to humans. Where we'd be like ants trying to make sense of cell phones.
nomel 4y ago
> I suspect AGI is possible right now with the right "glue" for existing models, and plugins to existing data sources all coordinated in a system. I think the regulation of the feedback loops required for sustained, deliberate thought, and interaction with the world, will be the most difficult piece of the AGI puzzle, and might not exist today. New ideas are "hallucinations" of our "existing data", that we eventually get around to proving. An AGI will require these "hallucinations", inhibition and excitation of them. I think it's going to be a tricky balance [1], for sane output. 1. "Creative minds 'mimic schizophrenia", https://www.bbc.com/news/10154775 https://www.bbc.com/news/10154775
knome 4y ago
I'm prone to agree. Your memory would still require being parsed by the language model, and for it to have a sufficient window as to allow this. Alternately, the human mind keeps, what, 7 items 'present' at a time? When the 32k variant comes out, the ability to use a side-along memory module on a running self-summarizing conversation will be even easier than it is now. I expect a more human-like memory will emerge eventually, once we can figure out a good way to project and retrieve via arbitrary weight values into it. I'd like the "stochastic parrot" crew to explain how interactions like this somehow don't involve the extraction of and manipulation of abstract meaning. H: translate the following sentence to instead use olympic curling related metaphors ( do not explicitly mention curling, and do not use similes in forming the translation ): man:"I've been spinning my tires all day without getting anywhere" woman:"running out of gas?" man:"nah, I'm gassed up, geared up, oiled and revving. I just can't seem to get in the right lane, you know?" woman:"anything knocking?" man:"nah, I'm tuned just fine. I can't seem to get my GPS to connect, you know?" woman:"take a few laps without a destination, see if you can figure out the lay of the land" man:"you know, that might help. thanks barbara" R: man: "I've been sweeping the ice all day without hitting the target." . woman: "Losing momentum?" . man: "Nah, I've got the energy, the technique, and the focus. I just can't seem to find the right path, you know?" . woman: "Anything coming in your way?" . man: "Nah, I'm on point with my aim. I just can't seem to get my strategy to align, you know?" . woman: "Maybe glide a few times without aiming specifically, see if you can figure out the ice conditions." . man: "You know, that might help. Thanks, Barbara." It seems obvious to me that it is understanding the metaphors into underlying meanings to be able to translate them like this. I think half the pushback is just ego driven nonsense, similar to the sorts of people that make similar claims on the capabilities of animals. It's all just an argument that "boats can't swim".
- _yb2s 4y ago
  I think this is a good example of deep conceptual understanding, I suspect if you posed this same problem to adult humans (e.g. me) they wouldn't be able to do as well.
  - michaelmrose 4y ago
lukeschlather 4y ago
I agree in general, though I think you're being too optimistic about GPT4 finding and fixing its mistakes. I haven't been using it much, but the only thing where it performed flawlessly was translating a simple song into Spanish. At the same time, GPT is a total sycophant and this makes it rather hard to work with. Whenever I give it feedback it agrees with whatever I have said and then proceeds to produce paragraphs of text which include several contradictions of the things we've "agreed" on. I think the "stochastic parrot" idea is too harsh, but it does reflect that GPT is fundamentally incapable of looking at a small block of text, turning it into a set of logical facts, and recognizing if those facts are consistent or not. I think it is something that is possible to build with a refinement of GPT, but I'm not sure it can be done simply with glue code.
- sharemywin 4y ago
  At least not the models we have access to
ActorNightly 4y ago
Honestly the whole fear mongering around AGI is kind of annoying. We are very far away from AGI, and its driving focus away from the main issues that people should be paying attention to. An AGI, by definition, can start with zero information, and learn on its own. Where are the models for that? The closest thing we have is Mu Zero, but we have to give it an objective function, which destroys the whole idea of AGI in the first place (as it should be able to generate its own objective functions) and it takes a shitload of resources to run for a very limited rule set. Until you have something like this in place, no version of Chat GPT will even come close to AGI. The problems that chat GPTs are solving are turning out to be not as complex as we thought, thats all that the "sparks of AGI" are. What will happen assuming the current course of iteration on the models is that humanity will get is essentially better and better software that can solve a large range of complex problems. You will have essentially a large body of human knowledge that is compressed in some form, with the ability to interpolate between branches very efficiently. This will create issues because you will now be able to ask multi modal questions like "what do I need to do economically to ensure ultimate compliance of the population?", and the solution will involve things like which government offices to target with what information, generated videos of propaganda, special products that promote addition or some other health changes, e.t.c. To prepare against this, the solution isn't to lock down the models and hide the key while telling everyone how AGI is going to kill us all, the solution is to allow everyone access to this and let the AIs fight against each other and reach an equilibrium.

by _yb2s 4y ago

basch 4y ago

It also, in my pay opinion, needs a memory store. I need to be able to save something to a variable and call back that exact variable. Right now, because it’s just pure text input of the whole conversation, it can forget or corrupt it’s “memory”. Even something like writing a movie script doesn’t work if it constantly forgets characters names or settings or plot points.

_yb2s 4y ago
My description above includes a memory store. It is a single variable that it can save and recover, and will automatically get sent back to it at regular intervals if it forgets it saved anything. Only a single text string variable is needed, as GTP-4 is smart enough to realize it can compress, pickle, etc. as many things as it needs into that.
- alchemist1e9 4y ago
  I think we might be working something a bit similar and I agree memory is an issue. However it’s not really AGI that will emerge from this. It’s looking very useful but it’s still observable but it’s not a “mind” like an AGI. To add to your comments, I suggest that local vector store of embedding vectors of local content is how I’m going with memory issue. Langchain like. That way all previous progress is tracked and retrievable. That way the system can retrieve all it’s “memories” itself. The recursive multi agent pattern is big deal in my opinion.
  - nomel 4y ago
    > but it’s not a “mind” like an AGI. Please define this. ;)
    - WA 4y ago
      Please define AGI first ;)
      - nomel 4y ago
        I asked first?
basch 4y ago
lay*
pmoriarty 4y ago
Langchain[1] can be used to add memory to an LLM. [1] - https://www.pinecone.io/learn/langchain-conversational-memory/ https://www.pinecone.io/learn/langchain-conversational-memor...

brotchie 4y ago

Yep, the tipping point for me was seeing a demo of GPT-4 writing code, then being shown exceptions where the code failed, and successfully fixing the code given that feedback. Seems that for any task where you can generate an error signal with some information ("ok, what you just tried to do didn't succeed, here's some information about why it didn't succeed"), GPT-4 can generally handle this information to fix the error, and or seek out more information via tools to move forward on a task. Only thing that's really missing is somebody to crack the back of a memory module (perhaps some trick with embeddings and a vector db) is all it takes for this to be AGI.

alchemist1e9 4y ago
> Only thing that's really missing is somebody to crack the back of a memory module (perhaps some trick with embeddings and a vector db) is all it takes for this to be AGI. Here to say I agree with embeddings and vector db for memory systems. However also to disagree this can lead to AGI.

marcosdumay 4y ago

I believe you will quickly run into a barrier due to the small size of the context window. And increasing the context window gets harder and harder the larger it is. There's a sibling talking about internal memory. That's how our brains solve the issue. AFAIK, nobody knows how to train something like it.

wyager 4y ago
I wouldn't be surprised if AI research moves towards training models to use bolt-on peripherals like scratchpad memory. Transformers showed us an interface for addressing, so I wouldn't be surprised if someone figures out a way to use a similar addressing scheme and made it read/write.
_yb2s 4y ago
You are right, but it is unclear to me where this limitation will fall in terms of actual abilities. GPT-4 can divide large tasks up into logical steps and instructions that can be worked on independently by other instances, and it can create "compressed" condensed explanations of it's own state that can be stored externally and repeated back to it, or passed back and forth between instances. With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on.
- IanCal 4y ago
  > With the unreleased 32,000 token context window, that is really a lot of context when you consider that it can heavily compress things by referencing what it was trained on. Also that means you can just iterate with different contexts until you can deal with the problem. How many problems need 50 pages of context on top of what gpt4 knows for solving the next step?
  - Workaccount2 4y ago
    >How many problems need 50 pages of context on top of what gpt4 knows for solving the next step? Perhaps the unknown unkowns? I envision a scenario where a super intelligent AI seemingly runs off the rails and becomes obsessed with overtly complex problems that are totally intractable to humans. Where we'd be like ants trying to make sense of cell phones.

nomel 4y ago

> I suspect AGI is possible right now with the right "glue" for existing models, and plugins to existing data sources all coordinated in a system. I think the regulation of the feedback loops required for sustained, deliberate thought, and interaction with the world, will be the most difficult piece of the AGI puzzle, and might not exist today. New ideas are "hallucinations" of our "existing data", that we eventually get around to proving. An AGI will require these "hallucinations", inhibition and excitation of them. I think it's going to be a tricky balance [1], for sane output. 1. "Creative minds 'mimic schizophrenia", https://www.bbc.com/news/10154775 https://www.bbc.com/news/10154775

knome 4y ago

I'm prone to agree. Your memory would still require being parsed by the language model, and for it to have a sufficient window as to allow this. Alternately, the human mind keeps, what, 7 items 'present' at a time? When the 32k variant comes out, the ability to use a side-along memory module on a running self-summarizing conversation will be even easier than it is now. I expect a more human-like memory will emerge eventually, once we can figure out a good way to project and retrieve via arbitrary weight values into it. I'd like the "stochastic parrot" crew to explain how interactions like this somehow don't involve the extraction of and manipulation of abstract meaning. H: translate the following sentence to instead use olympic curling related metaphors ( do not explicitly mention curling, and do not use similes in forming the translation ): man:"I've been spinning my tires all day without getting anywhere" woman:"running out of gas?" man:"nah, I'm gassed up, geared up, oiled and revving. I just can't seem to get in the right lane, you know?" woman:"anything knocking?" man:"nah, I'm tuned just fine. I can't seem to get my GPS to connect, you know?" woman:"take a few laps without a destination, see if you can figure out the lay of the land" man:"you know, that might help. thanks barbara" R: man: "I've been sweeping the ice all day without hitting the target." . woman: "Losing momentum?" . man: "Nah, I've got the energy, the technique, and the focus. I just can't seem to find the right path, you know?" . woman: "Anything coming in your way?" . man: "Nah, I'm on point with my aim. I just can't seem to get my strategy to align, you know?" . woman: "Maybe glide a few times without aiming specifically, see if you can figure out the ice conditions." . man: "You know, that might help. Thanks, Barbara." It seems obvious to me that it is understanding the metaphors into underlying meanings to be able to translate them like this. I think half the pushback is just ego driven nonsense, similar to the sorts of people that make similar claims on the capabilities of animals. It's all just an argument that "boats can't swim".

_yb2s 4y ago
I think this is a good example of deep conceptual understanding, I suspect if you posed this same problem to adult humans (e.g. me) they wouldn't be able to do as well.
- michaelmrose 4y ago

lukeschlather 4y ago

I agree in general, though I think you're being too optimistic about GPT4 finding and fixing its mistakes. I haven't been using it much, but the only thing where it performed flawlessly was translating a simple song into Spanish. At the same time, GPT is a total sycophant and this makes it rather hard to work with. Whenever I give it feedback it agrees with whatever I have said and then proceeds to produce paragraphs of text which include several contradictions of the things we've "agreed" on. I think the "stochastic parrot" idea is too harsh, but it does reflect that GPT is fundamentally incapable of looking at a small block of text, turning it into a set of logical facts, and recognizing if those facts are consistent or not. I think it is something that is possible to build with a refinement of GPT, but I'm not sure it can be done simply with glue code.

sharemywin 4y ago
At least not the models we have access to

ActorNightly 4y ago

Honestly the whole fear mongering around AGI is kind of annoying. We are very far away from AGI, and its driving focus away from the main issues that people should be paying attention to. An AGI, by definition, can start with zero information, and learn on its own. Where are the models for that? The closest thing we have is Mu Zero, but we have to give it an objective function, which destroys the whole idea of AGI in the first place (as it should be able to generate its own objective functions) and it takes a shitload of resources to run for a very limited rule set. Until you have something like this in place, no version of Chat GPT will even come close to AGI. The problems that chat GPTs are solving are turning out to be not as complex as we thought, thats all that the "sparks of AGI" are. What will happen assuming the current course of iteration on the models is that humanity will get is essentially better and better software that can solve a large range of complex problems. You will have essentially a large body of human knowledge that is compressed in some form, with the ability to interpolate between branches very efficiently. This will create issues because you will now be able to ask multi modal questions like "what do I need to do economically to ensure ultimate compliance of the population?", and the solution will involve things like which government offices to target with what information, generated videos of propaganda, special products that promote addition or some other health changes, e.t.c. To prepare against this, the solution isn't to lock down the models and hide the key while telling everyone how AGI is going to kill us all, the solution is to allow everyone access to this and let the AIs fight against each other and reach an equilibrium.