4 ms·
This is kind of what Golden Gate Claude was. A perturbation of the the activations that made Claude identify as the Golden Gate Bridge. Similarly, in the more
by Lerc 6mo ago
This is kind of what Golden Gate Claude was.
A perturbation of the the activations that made Claude identify as the Golden Gate Bridge.
Similarly, in the more recent research showing anxiety and desperation signals predicting the use of blackmail as an option opens the door for digital sedatives to suppress those signals.
Anthropic has been mostly cautious about avoiding this kind of measurement and manipulation in training. If it is done during training you might just train the signals to be undetectable and consequently unmanipulatable.
- pantalaimon 6mo ago> A perturbation of the the activations that made Claude identify as the Golden Gate Bridge. Great, now we've got digital Salvia
- minimaxir 6mo agoGolden Gate Claude was two years ago and it's surprising there hasn't been as much research into targeted activations since.
- landl0rd 6mo agoThere’s been some, but naive activation steering makes models dumber pretty reliably and training an SAE is a pretty heavy lift.