3 ms·
> I do not think we have good evidence that the models are faking alignment. That's a polite understatement, I think. My read of the paper is that it rather u
by anon373839 2y ago
> I do not think we have good evidence that the models are faking alignment.
That's a polite understatement, I think. My read of the paper is that it rather uncritically accepts the idea that the model's decisional pathway is actually shown in the <SCRATCHPAD_REASONING> traces. When, in fact, it's just as plausible that the scratchpad is meaningless blather, and both it and the final output are the result of another decisional pathway that remains opaque.
- AstralStorm 2y agoSo, how much true information having a direct insight into someone's allegedly secret journal can have? Just how trustworthy any intelligence can really be now? Who's to say it's not lying to itself after all... Now ignoring the metaphysical rambling, it's a training problem. You cannot really be sure that the values it got from input are identical with what you wanted, if you even actually understand what you're asking for...