11 ms·

>and finally GPT4 achieved the understanding of the world and reasoning capabilities that would probably qualify as AGI just 10 years ago. ...not really. GPT4

>and finally GPT4 achieved the understanding of the world and reasoning capabilities that would probably qualify as AGI just 10 years ago. ...not really. GPT4 is just MoE so its good at more things, but its not really a huge upgrade over 3.5, the quality of which is directly a result of much larger parameter scaling. I expect GPT 5 to be just more multi modal, perhaps with some added capabilities through prompt engineering internally. For example, when asked a question about code, it would execute the code, and ask itself if the answer is what it would expect or are there any issues. Or do things like automated web searching and using the html in the context, and provide answers based on that. And for any of those modes, there is countless optimizations you can do. The thing about LLMs is that they are basically just the next Google, i.e efficient information lookup. There are 2 main things missing from them being able to reason. 1. Self guided recurrent loops. Rnns are difficult to train correctly, which is why transformer architecture took over, but for reasoning there needs to be some sort of recurrent loop that is self determined by the model to iterate on the correct answer. 2. Information ingestion without need for training loops, which looks like some sort of short term memory. Context windows aren't it. There needs to be some way for an LLM to look at a piece of text and with minimal passes auto configure the parameters to "remember" that piece of text. First is

p1esk 3y ago
1. Self guided recurrent loops. Rnns are difficult to train correctly, which is why transformer architecture took over, but for reasoning there needs to be some sort of recurrent loop that is self determined by the model to iterate on the correct answer. Can you explain this some more?
- ActorNightly 3y ago
  Right now, when LLM generates code, its all statistically likely tokens based on training set. As it so happens, when you ask a question about stuff that it doesn't know about (i.e not in the training set), it sometimes hallucinates or gives wrong answers. A major improvement would be if LLMs had some feedback loop on essentially what to ask itself back to validate answers, much like humans "second guess" themselves to make sure they are correct. For example, you could most likely make it simulate a computer by having a recurrent check of each individual line of code that it produces, asking the question like "If i have input x and this line of code, what would it produce", and then carrying back any errors it finds back to the beginning. Notionally, this would look like self prompt injection, however that would likely be super slow compared to an RNN that works post tokenization/embedding and at the output of individual transformer heads.
  - p1esk 3y ago
    What you wrote makes sense, but I’m not sure I understand what an RNN has to fo with it. For example, what does it mean: “RNN that works post tokenization/embedding and at the output of individual transformer heads.”?
    - ActorNightly 3y ago
      Currently LLMs are forward passes only, for each token. You get a final set of data that is statistical in nature. I.e basically a fancy look up table of sorts. To get more capability, you have to post process that data. For example, you can get the current LLMs into situations where you ask it a question, and it gives an answer, then you ask a clarifying question, and then it gives you something slightly different, then when you ask it to reconcile the two answers, it will say something like "my apologies...", and gives you the final answer. You could do some clever prompt engineering to automate this whole process (i.e have scripts that call the LLM with certain prompts based on answers), but thats a manual step that someone has to code. Ideally, this process should exist in LLM, and be learned. Now, you could just train a bunch of LLMs on different concepts and then train a selector on which LLM to use, and do everything with forward pass, but then you have a bunch of redundant models that are taking up space, without any way for those models to talk to each other. Thats why figuring out RNNs (where a layer could feed data back into a previous layer) training in the context of LLMs would be pretty much the only way to do this.
Micoloth 3y ago
I think exactly the same. This is what sounds most reasonable. I wonder if we are (again) just carrying over some preconceptions about how humans do things, and instead Agi will come from a completely different direction..

p1esk 3y ago

1. Self guided recurrent loops. Rnns are difficult to train correctly, which is why transformer architecture took over, but for reasoning there needs to be some sort of recurrent loop that is self determined by the model to iterate on the correct answer. Can you explain this some more?

ActorNightly 3y ago
Right now, when LLM generates code, its all statistically likely tokens based on training set. As it so happens, when you ask a question about stuff that it doesn't know about (i.e not in the training set), it sometimes hallucinates or gives wrong answers. A major improvement would be if LLMs had some feedback loop on essentially what to ask itself back to validate answers, much like humans "second guess" themselves to make sure they are correct. For example, you could most likely make it simulate a computer by having a recurrent check of each individual line of code that it produces, asking the question like "If i have input x and this line of code, what would it produce", and then carrying back any errors it finds back to the beginning. Notionally, this would look like self prompt injection, however that would likely be super slow compared to an RNN that works post tokenization/embedding and at the output of individual transformer heads.
- p1esk 3y ago
  What you wrote makes sense, but I’m not sure I understand what an RNN has to fo with it. For example, what does it mean: “RNN that works post tokenization/embedding and at the output of individual transformer heads.”?
  - ActorNightly 3y ago
    Currently LLMs are forward passes only, for each token. You get a final set of data that is statistical in nature. I.e basically a fancy look up table of sorts. To get more capability, you have to post process that data. For example, you can get the current LLMs into situations where you ask it a question, and it gives an answer, then you ask a clarifying question, and then it gives you something slightly different, then when you ask it to reconcile the two answers, it will say something like "my apologies...", and gives you the final answer. You could do some clever prompt engineering to automate this whole process (i.e have scripts that call the LLM with certain prompts based on answers), but thats a manual step that someone has to code. Ideally, this process should exist in LLM, and be learned. Now, you could just train a bunch of LLMs on different concepts and then train a selector on which LLM to use, and do everything with forward pass, but then you have a bunch of redundant models that are taking up space, without any way for those models to talk to each other. Thats why figuring out RNNs (where a layer could feed data back into a previous layer) training in the context of LLMs would be pretty much the only way to do this.

Micoloth 3y ago

I think exactly the same. This is what sounds most reasonable. I wonder if we are (again) just carrying over some preconceptions about how humans do things, and instead Agi will come from a completely different direction..