5 ms·

> Lot's of devs here, but als in the media, believe GPT is the precursor for AGI. Which is ridiculous, it's just an incredible token predictor. Geoffrey Hinton

> Lot's of devs here, but als in the media, believe GPT is the precursor for AGI. Which is ridiculous, it's just an incredible token predictor. Geoffrey Hinton and Ilya Sutskever have said it is impossible to predict the next token without solving reasoning and intelligence. I'm not entirely sure how they justify that. If I have to guess, they would say there's no hacky shortcut that the neural net can cheat with to solve this problem, actual intelligence is the only way, and so that's what we get. My 2c: The thing that LLM critics miss is that LLMs have learned useful representations of complicated abstract concepts. Analogous to edge detectors -> object detectors in vision networks, LLMs have circuits that correspond to human-understandable concepts. I have no idea how we're going to solve hallucinations, but it's obvious to me that we've already figured out a key milestone on the way to AGI: How to encode abstract concepts into neural nets. It's now just a matter of how to use those representations downstream in a way that doesn't hallucinate.

candiodari 3y ago
Ever thought about ... let's say you were God. You wanted to design intelligence, to control insects, animal bodies and eventually humans. What would it need to do? Well, you'll quickly find that bodies require constant control to survive. Whilst an animal's body can survive for months with an animal's mind operating at 1% capacity, if the output of the mind truly stops, the body dies. The heart, respiration, the circulatory system (deciding where the blood goes), ... all need to be controlled with some amount of intelligence for them to function at all. Even just to remain alive, they need intelligence. They don't work on reflexes alone. For mammals, total loss of control leads to fatal injury ... in seconds. Without control, the heart will either stop or (wrong order of muscle firing) be totally ineffective at building pressure in the circulatory system. This will kill oxygen transfer through the blood-brain barrier as soon as the pressure drop reaches the brain, which will be much less than a second, this leads to further loss of control in 2-3 seconds. Irreperable damage sets in before 20 seconds pass, death (permanent and total loss of control) occurs in humans after about 1 minute 30 seconds. The plus side of this seems to be that, when that control is there, mammal bodies are far more efficient than a "passive" robot can be (any robot you can turn off -safely- at any time is a "passive" robot, the big counterexample is a flying drone, which will incur damage and may injure others if turned off or it loses control) So "predict the next token" is really THE most important function of the brain. Second, it is incredibly important to KEEP predicting the next token, to keep going, and keep going, and keep going, no matter how stupid the output, because literally any output will give a result superior to no output at all. No matter the damage, no matter how terribly wrong things are going, it must keep going further.
- omikun 3y ago
  I like where you're going with this! I think a stronger argument is that animals can survive so well by ONLY predicting just a few seconds in to the future for the most part. If you own a cat you'll realize most of them don't have a good understand of physical properties like levers or strings or one thing can knock into another. Of course you'll find on SM cats opening doors with uncanny ability. But I can see from my own cat who can open doors, that he has an incredibly rudimentary understanding of levers. He only knows the lever must be moved but not which direction or how. He can only mash at it with his body until it eventually opens. He doesn't understand why a child proof lock stops him from opening the door so he'll try over and over again. With other cats as well, they only form vague associations over either many repetitions or from an emotional experience. But they don't reason the way we can learn from one experience.
- exe34 3y ago
  It seems to me you're conflating two different things - control systems and intelligence. We already have pretty good engineering tools for control systems and better ones are discovered now and again. LLMs aren't meant to control individual servos, although it appears that transformers can do that too. The human brain doesn't use the same architecture for controlling the heart as it does for processing vision, for example.
  - candiodari 3y ago
    Everyone's always comparing LLM and trasnformers to the human mind. Hence this is a valid comparison, as the human mind definitely is a control system. And it has the active robot limitation.
    - exe34 3y ago
      Everyone's always compared the human mind to the most advanced technology we have. First it was pumps, then transistors, then computers, and now the highest abstraction so far, the transformer. They may or may not be sufficient to replicate the human mind, but they just haven't failed to scale up and show new abilities yet.
deleted 3y ago
[deleted]
caeril 3y ago
> I'm not entirely sure how they justify that. You know that humans hallucinate too, right? All the time, whether intentionally or unintentionally. Our memories are sparse, and we fill in the gaps with things that sound reasonable. We do this constantly. We also lie. Also, constantly. LLMs are definitely not intelligent from a reasoning and "shape rotator" perspective, but the competence of wordcels and the entire concept of "verbal IQ" has been absolutely obliterated in a just few years. Once we integrate human reasoning and "shape rotation" models, it's game over.
- tim333 3y ago
  I was going to say humans do something like hallucinate all the time - we come up with what ifs, fiction and so on. I think the thing is we then hopefully go back and check if it seems real or made up, which would not necessarily be hard to do in LLMs. Having come up with a fact it could google it to check for example - something I do myself. It's interesting with the "shape rotator" or physics engine type stuff OpenAI's Sora video thing seems to have picked up a reasonable amount of that. The current LLMs seem to miss out on a looping thinking aspect as in I could do option A - visualise it - not that wouldn't be good because of some problem - how about option B and so on that we humans call thinking about things. But it seems that could be programmed in.

candiodari 3y ago

Ever thought about ... let's say you were God. You wanted to design intelligence, to control insects, animal bodies and eventually humans. What would it need to do? Well, you'll quickly find that bodies require constant control to survive. Whilst an animal's body can survive for months with an animal's mind operating at 1% capacity, if the output of the mind truly stops, the body dies. The heart, respiration, the circulatory system (deciding where the blood goes), ... all need to be controlled with some amount of intelligence for them to function at all. Even just to remain alive, they need intelligence. They don't work on reflexes alone. For mammals, total loss of control leads to fatal injury ... in seconds. Without control, the heart will either stop or (wrong order of muscle firing) be totally ineffective at building pressure in the circulatory system. This will kill oxygen transfer through the blood-brain barrier as soon as the pressure drop reaches the brain, which will be much less than a second, this leads to further loss of control in 2-3 seconds. Irreperable damage sets in before 20 seconds pass, death (permanent and total loss of control) occurs in humans after about 1 minute 30 seconds. The plus side of this seems to be that, when that control is there, mammal bodies are far more efficient than a "passive" robot can be (any robot you can turn off -safely- at any time is a "passive" robot, the big counterexample is a flying drone, which will incur damage and may injure others if turned off or it loses control) So "predict the next token" is really THE most important function of the brain. Second, it is incredibly important to KEEP predicting the next token, to keep going, and keep going, and keep going, no matter how stupid the output, because literally any output will give a result superior to no output at all. No matter the damage, no matter how terribly wrong things are going, it must keep going further.

omikun 3y ago
I like where you're going with this! I think a stronger argument is that animals can survive so well by ONLY predicting just a few seconds in to the future for the most part. If you own a cat you'll realize most of them don't have a good understand of physical properties like levers or strings or one thing can knock into another. Of course you'll find on SM cats opening doors with uncanny ability. But I can see from my own cat who can open doors, that he has an incredibly rudimentary understanding of levers. He only knows the lever must be moved but not which direction or how. He can only mash at it with his body until it eventually opens. He doesn't understand why a child proof lock stops him from opening the door so he'll try over and over again. With other cats as well, they only form vague associations over either many repetitions or from an emotional experience. But they don't reason the way we can learn from one experience.
exe34 3y ago
It seems to me you're conflating two different things - control systems and intelligence. We already have pretty good engineering tools for control systems and better ones are discovered now and again. LLMs aren't meant to control individual servos, although it appears that transformers can do that too. The human brain doesn't use the same architecture for controlling the heart as it does for processing vision, for example.
- candiodari 3y ago
  Everyone's always comparing LLM and trasnformers to the human mind. Hence this is a valid comparison, as the human mind definitely is a control system. And it has the active robot limitation.
  - exe34 3y ago
    Everyone's always compared the human mind to the most advanced technology we have. First it was pumps, then transistors, then computers, and now the highest abstraction so far, the transformer. They may or may not be sufficient to replicate the human mind, but they just haven't failed to scale up and show new abilities yet.

deleted 3y ago

[deleted]

caeril 3y ago

> I'm not entirely sure how they justify that. You know that humans hallucinate too, right? All the time, whether intentionally or unintentionally. Our memories are sparse, and we fill in the gaps with things that sound reasonable. We do this constantly. We also lie. Also, constantly. LLMs are definitely not intelligent from a reasoning and "shape rotator" perspective, but the competence of wordcels and the entire concept of "verbal IQ" has been absolutely obliterated in a just few years. Once we integrate human reasoning and "shape rotation" models, it's game over.

tim333 3y ago
I was going to say humans do something like hallucinate all the time - we come up with what ifs, fiction and so on. I think the thing is we then hopefully go back and check if it seems real or made up, which would not necessarily be hard to do in LLMs. Having come up with a fact it could google it to check for example - something I do myself. It's interesting with the "shape rotator" or physics engine type stuff OpenAI's Sora video thing seems to have picked up a reasonable amount of that. The current LLMs seem to miss out on a looping thinking aspect as in I could do option A - visualise it - not that wouldn't be good because of some problem - how about option B and so on that we humans call thinking about things. But it seems that could be programmed in.