3 ms·
Does it really provide the statistically most common next token? I don’t think that’s even true. And we’ve seen what looks like emergent behavior as parameter
by kenjackson 4y ago
Does it really provide the statistically most common next token? I don’t think that’s even true. And we’ve seen what looks like emergent behavior as parameters increase. I suspect it is actually creating abstractions at a deeper level than standard DNNs.
- graypegg 4y agoI’d be curious to know! I get skeptical when it starts moving into the “emergent properties of language” but language was built by fleshy neurons, it could be that some inherent pattern emerges in the language we speak that reflects the inner workings of our brain, which by trying to emulate makes a similar brain on the other end. However, that feels a bit magical to me at the moment. Still interesting to be along for the ride humanity is going on!