5 ms·

I'm not sure I buy the author's strict implied definition of "understanding". Yes, ChatGPT sometimes makes mistakes, contradicts itself, and loses track of its

I'm not sure I buy the author's strict implied definition of "understanding". Yes, ChatGPT sometimes makes mistakes, contradicts itself, and loses track of its own reasoning. Those are all things that humans do frequently, although the frequency of those mistakes and in what contexts they're made is different. I have on many occasions gotten minutes into a discussion before realizing that I was simultaneously arguing for and against my own position, and having to step back and reexamine. Yet I'm pretty sure that I still understand things. Certainly, there are certain topics where ChatGPT can be said to utterly and consistently fail to understand, but again, there are many topics where a human will consistently fail to understand as well. I once sat in the middle of an intro philosophy class during which a room full of adults with highschool diplomas failed for thirty minutes straight to understand that "if X then Y" is not the same as "if Y then X" despite the TA's efforts using multiple real-world examples. I would not infer from that that those humans were incapable of understanding in general, but rather that they had not yet developed the cognitive tools to deal with logic abstractly. I've also seen plenty of examples of humans following the format of an explanation or argument while filling in the details with vague jargon and inapplicable nonsense, making their words sound official and superficially convincing, without actually doing the cognitive work to say anything useful. In fact, I've seen that in documents written by people in their capacity as professional engineers and managers. If you've worked in tech for any significant time, you've probably seen it too. But it would be wild for me to claim that the authors of those documents were generally incapable of understanding. Does that mean that ChatGPT definitely understands things? No. I think it means that we don't really know what we mean by words like "understand" yet, because up until now, we've only had to apply words like that to human brains, and we're generally happy to operate in a framework where we assume that human brains mostly work more or less the same way. But I have noticed a trend of people dramatically underestimating the actual practical things that GPT-3 can do based on their assumptions about how it works and what its limitations are.

visarga 4y ago
I think understanding implies the ability to reapply old knowledge in novel situations with success, and to draw new knowledge from experience.
- mistercow 4y ago
  That’s a pretty broad definition, and one that GPT-3 doesn’t have difficulty meeting.
DangitBobby 4y ago
I'll just link you to a comment of of mine where I explain ChatGPT demonstrating a lack of understanding of what it's saying. https://news.ycombinator.com/item?id=34125077 https://news.ycombinator.com/item?id=34125077 Also, for your own amusement you should ask it these two questions and compare the outputs: Which is older, a baby or a teenager? Which is older, a 12 year old baby or a teenager? To the first, it will first explain what a baby is and how old a teenager is (it says 13-19) and say that generally a teenager will be older but that babies and teenagers vary in age and there could be exceptions to this rule. To the second, it will tell you that the question doesn't make sense because there's no such thing as a 12 year old baby because 12 year olds are much too old to be considered babies. Generally it is able to compare quantities correctly but it cannot make indirect comparisons this way. It will incorrectly compare 13km and 10mi 100% of the time, for example, despite knowing that 10mi is 16km. I could keep giving examples of silly or contradictory responses to prompts here. But it's clear that it doesn't really "get" what it's saying in many cases.
- mistercow 4y ago
  Right, I totally agree that there are many cases where it doesn’t really get what it’s saying, but my point is that failure to understand many basic concepts (even concepts which would be surprising for a human to misunderstand) doesn’t imply an inability to understand in general. It only implies that the model has certain deficits. Imagine if we encountered an alien species for whom quantum mechanics was perfectly natural and intuitive. Those aliens ask me a bunch of basic QM questions. I’m more familiar with QM than the average layman, but the best I can do is try to shove their questions into the metaphors I know, and with the aid of the internet, I could do a little bit of the math. Most of what I answer will be confused. The aliens are going to come away with the correct impression that I don’t really understand QM. And because QM is so natural to them, they may also think that this is an indictment of my reasoning skills overall. But the reality is just that they’re testing me on questions for which my mental model is very shoddy. I’m not saying that the situation is analogous for ChatGPT. I’m just saying that this question can’t be answered by collecting up examples of where the model does poorly. For what it’s worth, out of four trials, the model only once gave any consideration beyond the word “typically” to the idea that a baby might be older than a teenager. In that case, it noted “However, it's important to note that age is a relative concept and can vary based on cultural and individual perspectives.” Honestly, I suspect that’s a result of its tuning and prompting to be culturally sensitive. In general, I’ve noticed that ChatGPT is constantly on the lookout for cases where its beliefs may not be culturally universal.

visarga 4y ago

I think understanding implies the ability to reapply old knowledge in novel situations with success, and to draw new knowledge from experience.

mistercow 4y ago
That’s a pretty broad definition, and one that GPT-3 doesn’t have difficulty meeting.

DangitBobby 4y ago

I'll just link you to a comment of of mine where I explain ChatGPT demonstrating a lack of understanding of what it's saying. https://news.ycombinator.com/item?id=34125077 https://news.ycombinator.com/item?id=34125077 Also, for your own amusement you should ask it these two questions and compare the outputs: Which is older, a baby or a teenager? Which is older, a 12 year old baby or a teenager? To the first, it will first explain what a baby is and how old a teenager is (it says 13-19) and say that generally a teenager will be older but that babies and teenagers vary in age and there could be exceptions to this rule. To the second, it will tell you that the question doesn't make sense because there's no such thing as a 12 year old baby because 12 year olds are much too old to be considered babies. Generally it is able to compare quantities correctly but it cannot make indirect comparisons this way. It will incorrectly compare 13km and 10mi 100% of the time, for example, despite knowing that 10mi is 16km. I could keep giving examples of silly or contradictory responses to prompts here. But it's clear that it doesn't really "get" what it's saying in many cases.

mistercow 4y ago
Right, I totally agree that there are many cases where it doesn’t really get what it’s saying, but my point is that failure to understand many basic concepts (even concepts which would be surprising for a human to misunderstand) doesn’t imply an inability to understand in general. It only implies that the model has certain deficits. Imagine if we encountered an alien species for whom quantum mechanics was perfectly natural and intuitive. Those aliens ask me a bunch of basic QM questions. I’m more familiar with QM than the average layman, but the best I can do is try to shove their questions into the metaphors I know, and with the aid of the internet, I could do a little bit of the math. Most of what I answer will be confused. The aliens are going to come away with the correct impression that I don’t really understand QM. And because QM is so natural to them, they may also think that this is an indictment of my reasoning skills overall. But the reality is just that they’re testing me on questions for which my mental model is very shoddy. I’m not saying that the situation is analogous for ChatGPT. I’m just saying that this question can’t be answered by collecting up examples of where the model does poorly. For what it’s worth, out of four trials, the model only once gave any consideration beyond the word “typically” to the idea that a baby might be older than a teenager. In that case, it noted “However, it's important to note that age is a relative concept and can vary based on cultural and individual perspectives.” Honestly, I suspect that’s a result of its tuning and prompting to be culturally sensitive. In general, I’ve noticed that ChatGPT is constantly on the lookout for cases where its beliefs may not be culturally universal.