We already had Markov chains over words, ChatGPT is just Markov chains over ideas (or even ideas about ideas about ideas). The astonishing bit is that huge amou

We already had Markov chains over words, ChatGPT is just Markov chains over ideas (or even ideas about ideas about ideas). The astonishing bit is that huge amounts of knowledge that we previously thought was purely human can be reduced down to this level. But no amount of work around the edges of these systems is going to fundamentally transform them. You can make it quote its sources, you can make it more reluctant to be racist, but we still haven’t cracked the self-critical, experimental pondering and exploration that takes a bunch of received ideas and turns them into reasoning and insight. On the one hand I’m hopeful we don’t pursue this local maxima too far, because it seems commercially interesting but fundamentally wasteful. On the other hand I think we’re closer than ever to unleashing destructive forces we can’t contain (even extremely dumb ones!)