3 ms·
> No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is v
by balfirevic 5y ago
> No, if you study how GPT-3 works, it becomes pretty clear that that model can't really produce "understanding", even if we can't clearly define what that is very well.
But the quote doesn't say that GPT-3 can't produce understanding because of the specifics of how it works, but "because understanding cannot occur in an isolated computation or behavior, no matter how clever", which is a more general claim.
- soco 5y agoAnd what is "understanding" anyway? Lots of "it can't be this it can't be that" but it doesn't tell what it actually is. Which makes it all easier to slap labels on things which don't actually need labels to just work. Does GPT-3 work? Yes it does and it works without bothering with whatever that "understanding" was. I bet Skynet wouldn't bother either to have an understanding or a soul, it would just... work.
- jerf 5y agoI think it would be fair to say that "understanding" requires some sort of internal state change. We can definitely talk about how we did not understand in the past, but now we do. GPT-3 can't experience a state change, by its design. I don't have to be able nail down exactly what kind of state change "understanding" is to point out that the way we use the word clearly entails one, and GPT-3 clearly can't have one, so therefore it is quite reasonably excluded from "understanding". (In theory GPT-3 could be extended by continuous online training, but my impression is that convergence is too slow for that to be practical, and it wouldn't look much like "learning".)