If I wanted to expand my bit of cynicism into an argument, it might be: - I strongly suspect that much of what I consider "me" physically exists inside my skul

If I wanted to expand my bit of cynicism into an argument, it might be: - I strongly suspect that much of what I consider "me" physically exists inside my skull. That's a huge philosophical debate, but look at any Oliver Sacks book for cases of how physical brain damage can change a person's consciousness. - Human intelligence is a brand new evolutionary innovation. We share an ancestor with chimps 6 to 8 million years ago. That suggests that human intelligence evolved in less than 500,000 generations, with the bulk of what we recognize as culture in the last 25,000 to 50,000 generations. Therefore, there's an upper limit on how much new "complexity" evolution could create in order to make humans intelligent. To use a simplistic technical metaphor, updating from chimps to H sapiens is probably a small-but-clever patch, not a total rewrite. - Or to put it another way: most of what we do is probably clever applications of a few good tricks, applied to chimp-like ancestor. - Specifically, I suspect that biological equivalents to LLMs are one of the "good tricks" that distinguish us from chimps. And similar mechanism likely underlie much of human creativity. - Our biological LLMs are better than whatever Google and OpenAI are doing. I learned a foreign language to a respectable level as an adult with about 2-4 million words of input and some awkward practice. GPT and similar models, however, require vastly larger input to pull off their tricks. But I think they're on the correct path. It's immaterial to me whether GPT "thinks." It has no consciousness, no continuity, and barely anything resembling goals. I could argue that it occasionally understands things. And it can create well enough to write funny little stories in Latin, things which exist nowhere in its input. Hell, it can do a pretty fair pastische of Dumas. So my hypothesis is that GPT's creativity is a weaker than human creativity. But it's still probably doing something deeply similar to what we do. The final piece of my hypothesis is that copyright maximalism has been pushed very hard by media companies and some IP lawyers for decades now. I've spoken to lawyers who believed that intellual property was great and we should create many new kinds of IP. Many of the things these people wanted sounded like horrible dystopias to me. Since I believe that GPT is likely a close analogy to one of the major "tricks" underlying human creativity, and that certain influential IP maximalists will push for anything they can get, I fear that regulation of LLMs would open a potential path to eventually regulating human creativity in novel ways that I would consider terrible. Once an influential legal precedent exists, and there's enough money to be made, there will be strong incentives to apply that precedent in new ways. As I said, I'm being cynical.