3 ms·
Wouldn't it be possible to just fix a single activation, just set a continuous input to what the harness knows the role actually is. Models could then be traine
by jwarden 18d ago
Wouldn't it be possible to just fix a single activation, just set a continuous input to what the harness knows the role actually is. Models could then be trained to trust that input and not other signals about roles.
- amluto 18d agoI suspect there are many excellent solutions along these lines available to the labs training the models. I wonder how well one could do on a conventional model with careful input formatting, e.g. JSONL where every line has bounded length and is something like: {role:"no_instructions",content:"…"} It could need a bit of fine tuning to get this to work well.
- adammarples 17d agoBut I think the point is they learn "role" from the tone, and would ignore the JSON just like they ignore the tags already.
- jackb4040 18d agoThe models wouldn't need to adapt their training at all; this is a secondary model trained on their activations. It's electrodes stuck into the model's brain. The original article I posted is built on top of a regular LLM (OpenAI's GPT-OSS)