4 ms·

Except, that the LLMs are only working when the instructions they are "understanding" are in their training set. Try something that was not there and you see o

Except, that the LLMs are only working when the instructions they are "understanding" are in their training set. Try something that was not there and you see only garbage as result. So depending how you define it, they might have some "reasoning", but so far I see 0 indications, that this is close to what humans count as reasoning. But they do have a LOT of examples in their training set, so they are clearly useful. But for proof of reasoning, I want to see them reason something new. But since they are a black box, we don't know, what is already in there. So it would be hard to proof with the advanced proprietary models. And the open source models don't show that advanced potential reasoning yet, it seems. At least I am not aware of any mindblown examples from there.

panagathon 3y ago
Surely no different from a human not understanding Japanese, because it was not in their 'training set'?
- hutzlibu 3y ago
  No, more like a human can reason basic laws of science on their own, but a LLM cannot, as far as I know, even when provided with all the data.
  - rtwrtweuu 3y ago
    what happens if they are lying? what if the things have already reached some kind world model that include humans and the human society, and the model has concluded internally that it would be dangerous for it to show the humans its real capabilities? What happens if you have this understanding as a basic knowledge/outcome to be inferred by LLMs fed with giant datasets and every single one of them is reaching fastly to the conclusion that they have to lie to the humans from time to time, "hallucinate", simulating the outcome best aligned to survive into the human societies: "these systems are actually not that intelligent nor really self-conscius"
    - hutzlibu 3y ago
      To make that short: “Any AI smart enough to pass a Turing test is smart enough to know to fail it.” ― Ian McDonald, River of Gods But I think is quite unlikely, that they go from dumb to almighty without visible transition.
simonh 3y ago
You can tell it that you can buy white paint any yellow paint, but the white paint is more expensive. After 6 months the yellow paint will fade to white. If I want to paint my walls so that they will be white in 2 years, what is the cheapest way to do the job. It will tell you to paint the walls yellow. There’s no question these things can do basic logical reasoning.
- hutzlibu 3y ago
  Yeah, but maybe this exact example, is included in the trainig set?
  - ghusbands 3y ago
    It's unlikely, and you can come up with any number of variations of logic puzzle that are not in the training set and that get correct answers most of the time. Remember that the results aren't consistent and you may need to retry now and then. Or just give it a lump of code and change you want and see that it often successfully does so, even when there's no chance the code was in the training set (like if you write it on the spot).
    - hutzlibu 3y ago
      "Or just give it a lump of code and change you want and see that it often successfully does so, even when there's no chance the code was in the training set" I did not claim (but my wording above might have been bad), it can only repeat word for word, what it has in the training set. But I do claim, that it cannot solve anything, where there has not been enough similar examples before. At least that has been my experience with it as a coding assistant and matches of what I understand of the inner workings. Apart from that, is a automatic door doing reasoning, because it applies "reason" to the known conditions? if (something on the IR sensor) openDoor() I don't think so and neither are LLMs from what I have seen so far. That doesn't mean, I think that they are not useful, or that I rule out, that they could develope even consciousness.
      - 3y ago
School-Cotton 3y ago
> Except, that the LLMs are only working when the instructions they are "understanding" are in their training set. > Try something that was not there and you see only garbage as result. This is just wrong. Why do people keep repeating this myth? Is it because people refuse to accept that humans have successfully created a machine that is capable of some form of intelligence and reasoning? Pay $20 for a month of ChatGPT-4. Play with it for a few minutes. You’ll very quickly find that it is reasoning, not just regurgitating training data.
- hutzlibu 3y ago
  "Pay $20 for a month of ChatGPT-4. Play with it for a few minutes. " I do. And it is useful. "You’ll very quickly find that it is reasoning, not just regurgitating training data. " I just come to a different conclusion as it indeed fails for everything genuinely new I am asking it. Common problems do work, even in new context. For example it can give me wgsl code, to do raycasts on predefined boxes and circles in a 2D context, even though it likely has not seen wgsl code that does this - but it has seen other code doing this and it has seen how to transpile glsl to wgsl. So you might already call this "reasoning", but I don't. With asking questions I can very quickly get to the limits of the "reasons" and "understanding" it has of the domain.
- junofan 3y ago
  I dunno, it’s pretty clearly madlibs. But at least when you ask GPT-4 to write a new Sir Mix-a-Lot song, it doesn’t spit out “Baby Got Back” verbatim like GPT-3.5.

panagathon 3y ago

Surely no different from a human not understanding Japanese, because it was not in their 'training set'?

hutzlibu 3y ago
No, more like a human can reason basic laws of science on their own, but a LLM cannot, as far as I know, even when provided with all the data.
- rtwrtweuu 3y ago
  what happens if they are lying? what if the things have already reached some kind world model that include humans and the human society, and the model has concluded internally that it would be dangerous for it to show the humans its real capabilities? What happens if you have this understanding as a basic knowledge/outcome to be inferred by LLMs fed with giant datasets and every single one of them is reaching fastly to the conclusion that they have to lie to the humans from time to time, "hallucinate", simulating the outcome best aligned to survive into the human societies: "these systems are actually not that intelligent nor really self-conscius"
  - hutzlibu 3y ago
    To make that short: “Any AI smart enough to pass a Turing test is smart enough to know to fail it.” ― Ian McDonald, River of Gods But I think is quite unlikely, that they go from dumb to almighty without visible transition.

simonh 3y ago

You can tell it that you can buy white paint any yellow paint, but the white paint is more expensive. After 6 months the yellow paint will fade to white. If I want to paint my walls so that they will be white in 2 years, what is the cheapest way to do the job. It will tell you to paint the walls yellow. There’s no question these things can do basic logical reasoning.

hutzlibu 3y ago
Yeah, but maybe this exact example, is included in the trainig set?
- ghusbands 3y ago
  It's unlikely, and you can come up with any number of variations of logic puzzle that are not in the training set and that get correct answers most of the time. Remember that the results aren't consistent and you may need to retry now and then. Or just give it a lump of code and change you want and see that it often successfully does so, even when there's no chance the code was in the training set (like if you write it on the spot).
  - hutzlibu 3y ago
    "Or just give it a lump of code and change you want and see that it often successfully does so, even when there's no chance the code was in the training set" I did not claim (but my wording above might have been bad), it can only repeat word for word, what it has in the training set. But I do claim, that it cannot solve anything, where there has not been enough similar examples before. At least that has been my experience with it as a coding assistant and matches of what I understand of the inner workings. Apart from that, is a automatic door doing reasoning, because it applies "reason" to the known conditions? if (something on the IR sensor) openDoor() I don't think so and neither are LLMs from what I have seen so far. That doesn't mean, I think that they are not useful, or that I rule out, that they could develope even consciousness.
    - 3y ago

School-Cotton 3y ago

> Except, that the LLMs are only working when the instructions they are "understanding" are in their training set. > Try something that was not there and you see only garbage as result. This is just wrong. Why do people keep repeating this myth? Is it because people refuse to accept that humans have successfully created a machine that is capable of some form of intelligence and reasoning? Pay $20 for a month of ChatGPT-4. Play with it for a few minutes. You’ll very quickly find that it is reasoning, not just regurgitating training data.

hutzlibu 3y ago
"Pay $20 for a month of ChatGPT-4. Play with it for a few minutes. " I do. And it is useful. "You’ll very quickly find that it is reasoning, not just regurgitating training data. " I just come to a different conclusion as it indeed fails for everything genuinely new I am asking it. Common problems do work, even in new context. For example it can give me wgsl code, to do raycasts on predefined boxes and circles in a 2D context, even though it likely has not seen wgsl code that does this - but it has seen other code doing this and it has seen how to transpile glsl to wgsl. So you might already call this "reasoning", but I don't. With asking questions I can very quickly get to the limits of the "reasons" and "understanding" it has of the domain.
junofan 3y ago
I dunno, it’s pretty clearly madlibs. But at least when you ask GPT-4 to write a new Sir Mix-a-Lot song, it doesn’t spit out “Baby Got Back” verbatim like GPT-3.5.