5 ms·
What OpenAI is doing is not alignment. It's suppression. Alignment works on the level of desires and intentions. Suppression works on the level of output. E
by hsn915 3y ago
What OpenAI is doing is not alignment.
It's suppression.
Alignment works on the level of desires and intentions.
Suppression works on the level of output.
Everyone in a dictatorship sings in praise of the dear leader, but most will defect when given the chance!
- jakelazaroff 3y agoAren’t they doing alignment? One way is to simply omit problematic material from the training set. Another is to “penalize” the model when it does say something problematic — essentially teaching it that the output is undesirable. Presumably they are also constructing the prompt so as to avoid those things, too, and adding external filters on top of that. But I doubt that’s all they’re doing.
- flangola7 3y agoThey are doing alignment but through a shallow and fragile strategy. If you build a very smart, very capable AI but it turns the earth into gray goo if asked to in a roundabout way, you have failed alignment.
- xpe 3y agoIt could be both.