3 ms·

I think a lot of people get the impression current NLP models like GPT-3 lack something - "understanding" or something. But they can't say exactly what it is. I

I think a lot of people get the impression current NLP models like GPT-3 lack something - "understanding" or something. But they can't say exactly what it is. If they could define exactly what it was, they could indeed describe a way to train for it or program it. So a theoretical argument could useful if gave us an idea what are our mysterious something it. However, I'm not sure if this article makes a contribution here. Proofs of impossibility generally aren't useful, as you say.

schoen 5y ago
Also, when people have tried to define exactly what it is, the NLP models have then usually turned out to be able to have or acquire it!
- joe_the_user 5y ago
  Also, when people have tried to define exactly what it is, the NLP models have then usually turned out to be able to have or acquire it! That can be true without being demonstration that we're seeing unbounded progress in language use and understanding. A theory of the limit of NLP is still a theory of NLP and by no means better than other theories.
trabant00 5y ago
> "understanding" or something. But they can't say exactly what it is. Umm, you just said it yourself, understanding. The model has no clue whatsoever what any of the output means, just that it scores good. I don't understand Japanese. But given enough time and feedback I can try all the possible combinations of sounds I can come up with and answer a question in Japanese till I get a satisfying feedback from the Japanese speaker (ex: his facial expression) that asked the question. Is this satisfactory as for speaking Japanese in your opinion?
- ivalm 5y ago
  That’s a red room style argument, but I’m guessing if you have time to get feedback/etc on all possible combinations you might just learn Japanese (because it is informationally compact to get high scores through semantic understanding rather that wrote memorization).
CharlesW 5y ago
> I think a lot of people get the impression current NLP models like GPT-3 lack something - "understanding" or something. But they can't say exactly what it is. It's always seemed obvious to me (as an outsider) that it's missing reason and explainability. GPT-3 is a neat tool, but it seems like anthropomorphism to suggest that it's more than the best mimic humanity has been able to create so far.

schoen 5y ago

Also, when people have tried to define exactly what it is, the NLP models have then usually turned out to be able to have or acquire it!

joe_the_user 5y ago
Also, when people have tried to define exactly what it is, the NLP models have then usually turned out to be able to have or acquire it! That can be true without being demonstration that we're seeing unbounded progress in language use and understanding. A theory of the limit of NLP is still a theory of NLP and by no means better than other theories.

trabant00 5y ago

> "understanding" or something. But they can't say exactly what it is. Umm, you just said it yourself, understanding. The model has no clue whatsoever what any of the output means, just that it scores good. I don't understand Japanese. But given enough time and feedback I can try all the possible combinations of sounds I can come up with and answer a question in Japanese till I get a satisfying feedback from the Japanese speaker (ex: his facial expression) that asked the question. Is this satisfactory as for speaking Japanese in your opinion?

ivalm 5y ago
That’s a red room style argument, but I’m guessing if you have time to get feedback/etc on all possible combinations you might just learn Japanese (because it is informationally compact to get high scores through semantic understanding rather that wrote memorization).

CharlesW 5y ago

> I think a lot of people get the impression current NLP models like GPT-3 lack something - "understanding" or something. But they can't say exactly what it is. It's always seemed obvious to me (as an outsider) that it's missing reason and explainability. GPT-3 is a neat tool, but it seems like anthropomorphism to suggest that it's more than the best mimic humanity has been able to create so far.