4 ms·
> It does not know anything. I would argue that it "knows" an awful lot, but it can't actually reason with it. However impressive GPT3 type models are, I am n
by ur-whale 5y ago
> It does not know anything.
I would argue that it "knows" an awful lot, but it can't actually reason with it.
However impressive GPT3 type models are, I am not particularly convinced that they're much more than glorified hashtables.
If the hash table is large enough, it can produce lot of answers to a lot of questions, or approximately imitate a lot of stuff it's seen before.
Whether it can actually combine "knowledge" it has stored in its weights into a pattern it's never seen before ... I'm not convinced.
- Der_Einzige 5y agoRe: "Glorified Hastable" There is a 1-1 correspondence between data compression and generative models. GPT-2 is a highly effective loseless data compression tool: https://bellard.org/textsynth/sms.html https://bellard.org/textsynth/sms.html Always wondered why this insight is not taught as much, especially in the context of things like dimensionality reduction...
- sheikheddy 5y agoThe Hutter prize for improved compression algorithms is explicitly about the relationship between compression and intelligence. http://prize.hutter1.net/ http://prize.hutter1.net/