6 ms·
> Once a model is open-weight, safeguards that do exist can be removed Safeguards trained into the model (ie exist in the weights) can’t be removed.
by le-mark 4mo ago
> Once a model is open-weight, safeguards that do exist can be removed
Safeguards trained into the model (ie exist in the weights) can’t be removed.
- gck1 4mo agoYou don't have to remove the safeguards if you can prompt your way around them. There's a subreddit for people wanting to sex-talk to various models. It just so happens that the same prompt they use to 'jailbreak' SOTA models for sex talks also works if you want to have model write malware, or tell you how to design a highly illegal device.
- regularfry 4mo agoSearch for "heretic"+Gemma/qwen/DeepSeek for examples where exactly this has been done.