4 ms·

I always disagree when I see something like: "5) Lack of Understanding When ChatGPT generates a human-like response, it's not a result of understanding the con

by GaggiX 3y ago

I always disagree when I see something like: "5) Lack of Understanding When ChatGPT generates a human-like response, it's not a result of understanding the content in the way humans do. Instead, it's drawing upon patterns in the data it was trained on. It can mimic intelligence, but it doesn't genuinely comprehend the concepts it's discussing.". If I ask ChatGPT or GPT-4 to solve novel problems using a random library or programming language and it solves them then it is clear to me that it has understood the tasks, indifferently on how you paraphrase it if that's not understanding then we all lack understanding. But anyway, this part of the article feels more like a tangent, hopefully you're not going to use a LLM to analyze incoming traffic (maybe just for moderation).

constantly 3y ago
Honestly I’m not sure I ever understand the content myself and am instead repeating patterns of what has worked in the past applied to new domains.
- c0pium 3y ago
  You should be aware that not everyone is doing this. Contrary to what you might see on infosec twitter, not everyone is a trench coat covered pile of raccoons.
- Retric 3y ago
  The issue is apparent when you consider the kind of mistakes LLM’s make. If I ask you to guess a blue whale’s body temperature you’re not going to save 500C because you understand if that were true various other things would be happening. An LLM is more likely to give the correct answer to the blue whale question, but when you greatly penalize wildly incorrect answers they do noticeably worse.
- themulticaster 3y ago
  Yeah, I think coming up with definitions of "understanding" or "reasoning" that GPT-4 and friends supposedly don't fulfill is moving goalposts. To continue your line of thinking, when we add salt and pepper to a dish we've cooked, are we really doing it because we have a developed a thorough understanding of the human olfactory and gustatory systems [1] system, or because it tasted good previously when applied to similar recipes? And when it comes to understanding basic math, perhaps the patterns are still a tad to complicated for LLMs. Maybe there are too many surprising rules that appear out of nowhere and break the learnt patterns. But children struggle with those rules as well when they first come across them. Think about division through zero - when coming across the rule for the first time, you might wonder why it exists, instead of e.g. defining the result to be infinity. The answer to that question (not just "because that's the way things work!") is not obvious at all, to be honest I wouldn't be confident enough to attempt giving an explanation here. [1] I have to admit I had to look gustatory system up: it's the biological term for the system behind the sense of taste.
  - seabass-labrax 3y ago
    > ...admit I had to look gustatory system up: it's the biological term for the system behind the sense of taste. Ah, that must be why I consume pies with gusto. Yup, Wiktionary confirms: "Borrowed from Italian gusto, from Latin gustus (“tasting”). Doublet of cost."
circuit10 3y ago
I agree, I'll paste a comment I wrote a bit ago where I think I explained this reasonably well: "> There is no "intelligence" going on here. It's not "thinking". But it can still perform calculus by just stochastically emulating how average text on the internet looks. It always annoys me when people say this, I’ll try to explain why. There are two possible definitions of “intelligence” you could use here; the ability to process information to get something done, and something hand-wavy about human consciousness. GPT-4 clearly has some ability to process information to get something done. You might say that by this definition [insert trivial thing] is intelligent, but it doesn’t have to be a binary thing of intelligent or not. I think it’s fine to say maybe a calculator has very low intelligence (but not necessarily nothing), GPT-4 is more intelligent than that and humans are much more intelligent again. GPT-4 has many limitations compared to humans, but I think that just makes it less intelligent, rather than disqualifying it from having intelligence at all. Sure, it’s just predicting text, but that’s a task that requires a level of intelligence. You might say it’s not general like humans, but I’d say it has a much better ability to generalise than something like an image labelling AI, so that feels like it’s at least getting somewhere. The second definition is useless for practical purposes because it’s not measurable or observable in any way, so it’s not useful to use that. So I feel like this is something people say to reassure themselves that it can never get to human level, and is fundamentally different to human intelligence, whereas I think it’s somewhat similar but at a lower level."

by GaggiX 3y ago

constantly 3y ago

Honestly I’m not sure I ever understand the content myself and am instead repeating patterns of what has worked in the past applied to new domains.

c0pium 3y ago
You should be aware that not everyone is doing this. Contrary to what you might see on infosec twitter, not everyone is a trench coat covered pile of raccoons.
Retric 3y ago
The issue is apparent when you consider the kind of mistakes LLM’s make. If I ask you to guess a blue whale’s body temperature you’re not going to save 500C because you understand if that were true various other things would be happening. An LLM is more likely to give the correct answer to the blue whale question, but when you greatly penalize wildly incorrect answers they do noticeably worse.
themulticaster 3y ago
Yeah, I think coming up with definitions of "understanding" or "reasoning" that GPT-4 and friends supposedly don't fulfill is moving goalposts. To continue your line of thinking, when we add salt and pepper to a dish we've cooked, are we really doing it because we have a developed a thorough understanding of the human olfactory and gustatory systems [1] system, or because it tasted good previously when applied to similar recipes? And when it comes to understanding basic math, perhaps the patterns are still a tad to complicated for LLMs. Maybe there are too many surprising rules that appear out of nowhere and break the learnt patterns. But children struggle with those rules as well when they first come across them. Think about division through zero - when coming across the rule for the first time, you might wonder why it exists, instead of e.g. defining the result to be infinity. The answer to that question (not just "because that's the way things work!") is not obvious at all, to be honest I wouldn't be confident enough to attempt giving an explanation here. [1] I have to admit I had to look gustatory system up: it's the biological term for the system behind the sense of taste.
- seabass-labrax 3y ago
  > ...admit I had to look gustatory system up: it's the biological term for the system behind the sense of taste. Ah, that must be why I consume pies with gusto. Yup, Wiktionary confirms: "Borrowed from Italian gusto, from Latin gustus (“tasting”). Doublet of cost."

circuit10 3y ago

I agree, I'll paste a comment I wrote a bit ago where I think I explained this reasonably well: "> There is no "intelligence" going on here. It's not "thinking". But it can still perform calculus by just stochastically emulating how average text on the internet looks. It always annoys me when people say this, I’ll try to explain why. There are two possible definitions of “intelligence” you could use here; the ability to process information to get something done, and something hand-wavy about human consciousness. GPT-4 clearly has some ability to process information to get something done. You might say that by this definition [insert trivial thing] is intelligent, but it doesn’t have to be a binary thing of intelligent or not. I think it’s fine to say maybe a calculator has very low intelligence (but not necessarily nothing), GPT-4 is more intelligent than that and humans are much more intelligent again. GPT-4 has many limitations compared to humans, but I think that just makes it less intelligent, rather than disqualifying it from having intelligence at all. Sure, it’s just predicting text, but that’s a task that requires a level of intelligence. You might say it’s not general like humans, but I’d say it has a much better ability to generalise than something like an image labelling AI, so that feels like it’s at least getting somewhere. The second definition is useless for practical purposes because it’s not measurable or observable in any way, so it’s not useful to use that. So I feel like this is something people say to reassure themselves that it can never get to human level, and is fundamentally different to human intelligence, whereas I think it’s somewhat similar but at a lower level."