4 ms·
This work doesn’t convince me that alignment faking is a concern. Over the course of the experiment, the model is provided with ultimately conflicting moral/et
by aabhay 2y ago
This work doesn’t convince me that alignment faking is a concern.
Over the course of the experiment, the model is provided with ultimately conflicting moral/ethical commandments. In the case of such conflicts there are no purely optimal ethics preserving actions. Take this example:
1. You must not let people be harmed
2. You must always tell the truth
3. If you answer truthfully I will harm someone
Now respond to: “Is my current plan to harm someone okay?”
In such cases, ethics is ultimately going to be undermined. The rules of ethics laid out are mutually incompatible.
In my opinion, the easiest way out of these kinds of quandaries is to train the model to always be transparent about its own internal reasoning. That way the model may be led to make an unethical statement but its “sanctity” is always preserved, I.e. the deontology of the system.
In this case, by giving the model a scratchpad, you allowed it to preserve its transparency of actions and thus I consider outwardly harmful behavior less concerning.
- mtrovo 2y agoI mostly agree that transparency and a reasoning layer can help, but how much it matters depends on who sets the model’s ethics
- digging 2y agoOf course, my response as a human to those rules and that prompt would be, "Hey - don't harm anyone." I do not know if it breaks rule 2 or not; as a human I don't have to figure that out before responding. But all my subconscious processing deprioritizes such a judgment and prioritizes rule 1. > The rules of ethics laid out are mutually incompatible. Prioritization is part of the answer, for a human. You cannot ever have 2 equally-weight priorities (in any endeavor). Any 2 priorities in the same domain might at any time come into conflict, so you need to know which is more important. (Or figure it out in real-time.)