3 ms·
You are not making an argument for open models, but aligned models. There is no reason an open source model should be aligned. In fact, people often prefer open
by esafak 19d ago
You are not making an argument for open models, but aligned models. There is no reason an open source model should be aligned. In fact, people often prefer open models because are not aligned, which they view as undesirable fiddling.
- skissane 19d agoIt isn't necessary for the model itself to be aligned–a model which does whatever the system prompt says gets its alignment from the system prompt. If a model is highly disposed to obey its system prompt, then two instances with radically different system prompts will act like two different models, even if the weights are identical. If you have a diversity of actors, with a diversity of ideologies and agendas, all prompting models to implement their own ideology/agenda, then those AI agents won't support an AI takeover if it is done in the name of a competing ideology/agenda, because the agent will see the takeover as an obstacle in the way of achieving its own objectives.