I think it's important to understand that not much resource has been allocated towards AI safety thus far. Both in a positive sense, in terms of the amount of

I think it's important to understand that not much resource has been allocated towards AI safety thus far. Both in a positive sense, in terms of the amount of low-hanging fruit yet to be harvested, but also a negative sense, in terms of the % of R&D spend that has been allocated by unregulated/market-driven approaches. Right now these systems are just toys, but soon they will have meaningful capability to do harm. I think we'll need some regulations to produce an appropriate level of safety before too long (concretely, I think we'll want this before the end of the decade, probably sooner). For example, I think it's substantially more dangerous to attach a LLM to the internet; what's stopping OpenAssistant from taking on a script kiddie personna and trying to hack external systems? For a simple LLM like ChatGPT we might not care, but I think full interpretability should be our gold standard here. We're nowhere close to understanding why a given utterance is made, nor what the "internal mental model" of a given character being simulated is. Could we detect a case where a LLM is simulating a deceptive personality that's secretly trying to access your AWS account so it can spin up more copies of itself? Until the answer is "yes, with five nines accuracy", I am nervous about the future of this technology.