2 ms·
That's a bit unclear at this point. Around GPT-3 it became apparent that LLMs are no longer glorified Markov chains - the training sets grew large enough, and t
by TeMPOraL 4y ago
That's a bit unclear at this point. Around GPT-3 it became apparent that LLMs are no longer glorified Markov chains - the training sets grew large enough, and the latent space high dimensional enough, that the model starts to capture the kind of fuzzy associations humans make. That is, it seems to capture an important subset of what we consider thinking.
So I guess GPT-4 may still be most useful at destroying the modern information economy, thanks to being an infinite generator of garbage indistinguishable from content marketing. But GPT-5 might be just good enough to enable some qualitative breakthroughs in robotics, spaceships and underwater cities...
(NOTE: this is not be believing that OpenAI solved consciousness. This is me recognizing the surprising ability for generative language models to navigate abstract concept space, just by virtue of being exposed to enough association implicitly encoded in everything we say or write...)