5 ms·

LLMs can't is such an anti-pattern at this point I'm surprised that anyone still dares to stake it. The piece even has an example of a $10k bet around a can't b

by kromem 2y ago

LLMs can't is such an anti-pattern at this point I'm surprised that anyone still dares to stake it. The piece even has an example of a $10k bet around a can't being proven false in under a day, but somehow doesn't think maybe their own can't examples are on similarly thin ice? In particular, the line about "what models can't do tells us what they don't know" is infuriating. No, that's not the case at all. At least in a number of instances, what they can't do is because of what they do know. As an example, one of thecan'ts I got from HN a year ago for GPT-4 was a variation of a classic logic puzzle. And indeed, the model can't solve it - nor can most major models since. But it's not because the model can't solve the logic - it's because the token similarity to the standard form biases the output towards the standard solution. A hack as simple as changing the nouns to emojis can allow the model to get the correct answer and work through the logic successfully every attempt because it breaks that similarity bias. People are way too confident around a topic where what's 'known' is more mercurial than maybe any field since 1930s particle physics. I'd strongly recommend deleting 'never' or 'can't' from one's vocabularies on the subject unless one enjoys ending up with egg on their faces.

Der_Einzige 2y ago
Amen brother. I had feelings like this which I wanted to share, but you hit the nail on the head.
fragmede 2y ago
interestingly enough, it seems that ChatGPT-4 can now detect the variations of the classic logic puzzle and solve it, so that can't is now a couldn't.
- kromem 2y ago
  We may be talking about different logic puzzles? The only model I've seen that didn't need some rather extreme adjustments to eventually solve it was Mistral large.
  - fragmede 2y ago
    what's your puzzle? mines the river crossing puzzle
    - kromem 2y ago
      Mine is also the river crossing puzzle. What's your variation?
      - fragmede 2y ago
        the boat is big enough to hold all the items I asked it > lion, goat, wolf riddle, but the boat is big enough to carry all of them and it said it could do it in one step. https://chat.openai.com/share/7b7a5462-7649-403d-a4f0-01c387e7eaf0 https://chat.openai.com/share/7b7a5462-7649-403d-a4f0-01c387... ChatGPT-4 today (April 28th, 2024) still fails at it sometimes: https://chat.openai.com/share/1bec923d-d727-42fe-ba9b-9f92b9780bcb https://chat.openai.com/share/1bec923d-d727-42fe-ba9b-9f92b9... This is ChatGPT-4 getting it wrong, months ago: https://chat.openai.com/share/caa37ad6-b7a8-451d-8f39-8a2c04249f4c https://chat.openai.com/share/caa37ad6-b7a8-451d-8f39-8a2c04... This is ChatGPT-4 getting it right, today (April 28th, 2024): https://chat.openai.com/share/d2d9e63e-819e-4681-9f9f-8f77ea12f282 https://chat.openai.com/share/d2d9e63e-819e-4681-9f9f-8f77ea...
        kromem 2y ago
        Ah, ok. My variation is it's a vegetarian wolf, a carnivorous goat, and a cabbage. There's a few different hacks that will get it to work, but one of the more interesting is switching the nouns to emojis. But almost none of the models ever get it on the first try, and every major model since GPT-4 can have the prompt tweaked to get it with the exception of Llama-3, which I just can't get to solve it with anything I've tried so far (and I'm not sure if it's because of extra strong associations to the standard form from the extra training run or if it lacks the core competencies, though I am starting to think it's the latter given how it responds as I point out errors). I particularly like this variation because it requires remapping concepts in unintuitive ways based on broad abstractions, like having a goat potentially eat a wolf because of it being carnivorous.
_akhe 2y ago
An LLM will probably be able to do most of what human minds can do like reason, predict, hypothesize, research, and even get hooked up to other systems to: Visualize, smell, taste, balance, and even direct the movement of limbs, but an LLM can't and won't ever be able to: Feel pain, bliss, anger, sadness, can't feel positive/negative, can't eat/drink, be hungry, feel fatigued, get excited, enjoy things, dislike things, contemplate, meditate, feel warm or cold (though it can detect it), can't feel dizzy (though it can know when it's off balance) - any action where having an experience is a necessary part of what it's doing and the output of it, an LLM is not sufficient to deliver on and never will be. To compare to a brain, the LLM is like the prefrontal cortex or language and decision network in the outermost layer, but we would still need the amygdala in that metaphor - emotional drives, urges, episodic first-person memories, and experiential components that accompany the language and complete it with personhood. For raw sensations and tactiles we might need that innermost brain stem - which is probably more chemistry than computation - for the "lights to be on". For example, some jobs will require not just language intelligence, and not just personhood, but for the light behind the images and feelings in the sensations, so that it feels (and would be) alive.

by kromem 2y ago

Der_Einzige 2y ago

Amen brother. I had feelings like this which I wanted to share, but you hit the nail on the head.

fragmede 2y ago

interestingly enough, it seems that ChatGPT-4 can now detect the variations of the classic logic puzzle and solve it, so that can't is now a couldn't.

kromem 2y ago
We may be talking about different logic puzzles? The only model I've seen that didn't need some rather extreme adjustments to eventually solve it was Mistral large.
- fragmede 2y ago
  what's your puzzle? mines the river crossing puzzle
  - kromem 2y ago
    Mine is also the river crossing puzzle. What's your variation?
    - fragmede 2y ago
      the boat is big enough to hold all the items I asked it > lion, goat, wolf riddle, but the boat is big enough to carry all of them and it said it could do it in one step. https://chat.openai.com/share/7b7a5462-7649-403d-a4f0-01c387e7eaf0 https://chat.openai.com/share/7b7a5462-7649-403d-a4f0-01c387... ChatGPT-4 today (April 28th, 2024) still fails at it sometimes: https://chat.openai.com/share/1bec923d-d727-42fe-ba9b-9f92b9780bcb https://chat.openai.com/share/1bec923d-d727-42fe-ba9b-9f92b9... This is ChatGPT-4 getting it wrong, months ago: https://chat.openai.com/share/caa37ad6-b7a8-451d-8f39-8a2c04249f4c https://chat.openai.com/share/caa37ad6-b7a8-451d-8f39-8a2c04... This is ChatGPT-4 getting it right, today (April 28th, 2024): https://chat.openai.com/share/d2d9e63e-819e-4681-9f9f-8f77ea12f282 https://chat.openai.com/share/d2d9e63e-819e-4681-9f9f-8f77ea...
      - kromem 2y ago
        Ah, ok. My variation is it's a vegetarian wolf, a carnivorous goat, and a cabbage. There's a few different hacks that will get it to work, but one of the more interesting is switching the nouns to emojis. But almost none of the models ever get it on the first try, and every major model since GPT-4 can have the prompt tweaked to get it with the exception of Llama-3, which I just can't get to solve it with anything I've tried so far (and I'm not sure if it's because of extra strong associations to the standard form from the extra training run or if it lacks the core competencies, though I am starting to think it's the latter given how it responds as I point out errors). I particularly like this variation because it requires remapping concepts in unintuitive ways based on broad abstractions, like having a goat potentially eat a wolf because of it being carnivorous.

_akhe 2y ago

An LLM will probably be able to do most of what human minds can do like reason, predict, hypothesize, research, and even get hooked up to other systems to: Visualize, smell, taste, balance, and even direct the movement of limbs, but an LLM can't and won't ever be able to: Feel pain, bliss, anger, sadness, can't feel positive/negative, can't eat/drink, be hungry, feel fatigued, get excited, enjoy things, dislike things, contemplate, meditate, feel warm or cold (though it can detect it), can't feel dizzy (though it can know when it's off balance) - any action where having an experience is a necessary part of what it's doing and the output of it, an LLM is not sufficient to deliver on and never will be. To compare to a brain, the LLM is like the prefrontal cortex or language and decision network in the outermost layer, but we would still need the amygdala in that metaphor - emotional drives, urges, episodic first-person memories, and experiential components that accompany the language and complete it with personhood. For raw sensations and tactiles we might need that innermost brain stem - which is probably more chemistry than computation - for the "lights to be on". For example, some jobs will require not just language intelligence, and not just personhood, but for the light behind the images and feelings in the sensations, so that it feels (and would be) alive.