Well, to some degree I think we are considering this approach to social media to have been a mistake, and are looking not to make that mistake again. I mean, a

Well, to some degree I think we are considering this approach to social media to have been a mistake, and are looking not to make that mistake again. I mean, at the very least, AI is going to amplify all the same stuff that social media did and push it a lot further. > I've yet to see any AI catastrophist address the fact that the training set for current state of the art LLMs is already effectively "the whole internet". It's difficult to see how an LLM which sets out to improve itself overcomes this limitation. Honestly, I think that even with AI at GPT-4 level, we could be in for a bad time. This tech is going to accelerate everything -- and it's a lot harder for a society to steer itself when things are changing so quickly. But I think you may be basing your viewpoint on some shaky assumptions here: * that LLMs are the smartest model architecture we can come up with * that the size of the training set is the limiting factor I think it's very likely that there will be more breakthroughs, even if it's just combining LLMs with other types of existing AI in novel ways, at first. As for the second, humans clearly learn much more than AI without reading the entire internet, so it seems likely that there's something else limiting LLMs. And that a different design could far surpass what LLMs can do. > We haven't even seen the early stages of models able to reason or develop and retain new knowledge independently. Maybe not with a pure LLM, but people have hooked up LLMs to a sort of notebook, where they can note down stuff they want to remember for later, just like a person with a not-so-great memory. And that's just kinda duct-taped on - I can imagine it being much better integrated and faster to access in future. > Maybe at that point I would agree it's time to apply the brakes, but at the moment regulation would be tantamount to strangling AI in the crib. I guess the question is, are we developing something that's going to be really bad for humanity? Because maybe that's the right thing to do. If at some point we research and figure out how to make AI that doesn't screw things up a lot, it's not too hard to allow it again. But it's hard to stop now, and it's going to be even harder to stop once companies start using AI lobbyists to influence policy. *edit -- and given how many people involved in AI have expressed concerns about where things are going, I wouldn't say that this would be out of ignorance. I see Moloch's (<https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/>) dirty fingerprints -- slowing down or stopping is a nasty coordination problem, where individual incentives are all to keep going as fast as you can.