2 ms·

I'll just link you to a comment of of mine where I explain ChatGPT demonstrating a lack of understanding of what it's saying. https://news.ycombinator.com/item

I'll just link you to a comment of of mine where I explain ChatGPT demonstrating a lack of understanding of what it's saying. https://news.ycombinator.com/item?id=34125077 https://news.ycombinator.com/item?id=34125077 Also, for your own amusement you should ask it these two questions and compare the outputs: Which is older, a baby or a teenager? Which is older, a 12 year old baby or a teenager? To the first, it will first explain what a baby is and how old a teenager is (it says 13-19) and say that generally a teenager will be older but that babies and teenagers vary in age and there could be exceptions to this rule. To the second, it will tell you that the question doesn't make sense because there's no such thing as a 12 year old baby because 12 year olds are much too old to be considered babies. Generally it is able to compare quantities correctly but it cannot make indirect comparisons this way. It will incorrectly compare 13km and 10mi 100% of the time, for example, despite knowing that 10mi is 16km. I could keep giving examples of silly or contradictory responses to prompts here. But it's clear that it doesn't really "get" what it's saying in many cases.

mistercow 4y ago
Right, I totally agree that there are many cases where it doesn’t really get what it’s saying, but my point is that failure to understand many basic concepts (even concepts which would be surprising for a human to misunderstand) doesn’t imply an inability to understand in general. It only implies that the model has certain deficits. Imagine if we encountered an alien species for whom quantum mechanics was perfectly natural and intuitive. Those aliens ask me a bunch of basic QM questions. I’m more familiar with QM than the average layman, but the best I can do is try to shove their questions into the metaphors I know, and with the aid of the internet, I could do a little bit of the math. Most of what I answer will be confused. The aliens are going to come away with the correct impression that I don’t really understand QM. And because QM is so natural to them, they may also think that this is an indictment of my reasoning skills overall. But the reality is just that they’re testing me on questions for which my mental model is very shoddy. I’m not saying that the situation is analogous for ChatGPT. I’m just saying that this question can’t be answered by collecting up examples of where the model does poorly. For what it’s worth, out of four trials, the model only once gave any consideration beyond the word “typically” to the idea that a baby might be older than a teenager. In that case, it noted “However, it's important to note that age is a relative concept and can vary based on cultural and individual perspectives.” Honestly, I suspect that’s a result of its tuning and prompting to be culturally sensitive. In general, I’ve noticed that ChatGPT is constantly on the lookout for cases where its beliefs may not be culturally universal.
cultureswitch 4y ago
The same can be said of any human being. And there's a reason for that. Ultimately, our brain processes just as automatic and mindless as that of ChatGPT, just much larger, specialized through evolution and trained for longer.
- DangitBobby 4y ago
  The brain process itself being automatic doesn't mean we don't understand.
  - mistercow 4y ago
    Right, just like ChatGPT operating on matching and predicting patterns doesn't mean that it doesn't understand.
    - DangitBobby 4y ago
      I never said that. I said that it has demonstrated a lack of understanding, not that it is fundamentally incapable of understanding.

mistercow 4y ago

Right, I totally agree that there are many cases where it doesn’t really get what it’s saying, but my point is that failure to understand many basic concepts (even concepts which would be surprising for a human to misunderstand) doesn’t imply an inability to understand in general. It only implies that the model has certain deficits. Imagine if we encountered an alien species for whom quantum mechanics was perfectly natural and intuitive. Those aliens ask me a bunch of basic QM questions. I’m more familiar with QM than the average layman, but the best I can do is try to shove their questions into the metaphors I know, and with the aid of the internet, I could do a little bit of the math. Most of what I answer will be confused. The aliens are going to come away with the correct impression that I don’t really understand QM. And because QM is so natural to them, they may also think that this is an indictment of my reasoning skills overall. But the reality is just that they’re testing me on questions for which my mental model is very shoddy. I’m not saying that the situation is analogous for ChatGPT. I’m just saying that this question can’t be answered by collecting up examples of where the model does poorly. For what it’s worth, out of four trials, the model only once gave any consideration beyond the word “typically” to the idea that a baby might be older than a teenager. In that case, it noted “However, it's important to note that age is a relative concept and can vary based on cultural and individual perspectives.” Honestly, I suspect that’s a result of its tuning and prompting to be culturally sensitive. In general, I’ve noticed that ChatGPT is constantly on the lookout for cases where its beliefs may not be culturally universal.

cultureswitch 4y ago

The same can be said of any human being. And there's a reason for that. Ultimately, our brain processes just as automatic and mindless as that of ChatGPT, just much larger, specialized through evolution and trained for longer.

DangitBobby 4y ago
The brain process itself being automatic doesn't mean we don't understand.
- mistercow 4y ago
  Right, just like ChatGPT operating on matching and predicting patterns doesn't mean that it doesn't understand.
  - DangitBobby 4y ago
    I never said that. I said that it has demonstrated a lack of understanding, not that it is fundamentally incapable of understanding.