5 ms·
This might be related to the fact that fully automating AI safety can't be meaningfully done. And a lot of work is put into automating parts of it. Circuit-find
by sometimelurker 5mo ago
This might be related to the fact that fully automating AI safety can't be meaningfully done. And a lot of work is put into automating parts of it. Circuit-finding algorithms and SAEs are automated algorithms for interpreting parts of LLMs, and RLAIF (RL with AI feedback) for alignment requires an LLM to judge if another LLM is visibly misaligned. (Claude says 'genuine' a lot due to this. Its harder to look misaligned when you use the word 'genuine' a ton) And there's work on having AIs write cute little stories in which AIs are ethical, and putting those stories in the pretraining corpus.
So there's a ton of work being done already on automating parts of alignment, but since the core premise of alignment being that its hard to encode human values into the reward function, automating it fully would be equivalent to solving it.