4 ms·
Add someone “in the know”, GPT-3 is more than a dictionary. I understand how language models mechanically work and have built and trained them, but the zero-sho
by moconnor 5y ago
Add someone “in the know”, GPT-3 is more than a dictionary. I understand how language models mechanically work and have built and trained them, but the zero-shot capabilities of these large models were really unexpected.
Interact with one. There it’s clearly an understanding of language there, and of much more. Compared to everything else we have made, large language models are clearly a step closer to general AI.
- azinman2 5y agoYou can call it a step, sure. But that would imply if you just continue on the same path you’ll get there. But these models don’t do anything for planning or reasoning, nor have a point of view, only proxy embodiment to the extent that humans have written about what that means, nor handle incongruities within its own model due to the many voices it’s trained on. Just adding parameters or more data doesn’t help with any of these fundamental problems. I think it also can be very difficult to assess what they “know” because of the Chinese dictionary problem… when you get back things that sound like we’ll formulated thought, it’d easy to project a deeper level of understanding on it than may actually be occurring.