3 ms·

Keep a close eye on abliterated and "heretic" open weight models. They will be outlawed first.

by Tepix 12d ago

thih9 12d ago
I'm not sure what is your point. It reads as defeatism to me but I'm not sure. Could you elaborate? Do you find it good or bad? What actions can be taken?
- cyanydeez 12d ago
  Hes of the mind that american fascism will hold together long enough to be competent decesion makers
- Tepix 12d ago
  I'm not sure yet, tbh. Perhaps it does make sense to outlaw them eventually. Then again, it will probably not stop someone who is determined. Same as with other legislation really.
roenxi 12d ago
It is not feasible. They never made much of an inroad against torrents and that is a much easier target than abliterated models. As the linked website shows; the process to abliterate a model can be as simple as pip install -U heretic-llm && heretic Qwen/Qwen3.5-4B let alone people just putting the weights up in a torrent. All assuming that someone even tried to ban abliterated models.
- Sayrus 12d ago
  The torrents you are talking about are outlawed. Whether enforcement is working or not is another issue.
  - galangalalgol 12d ago
    I think that was the point being made? Outlawing something does nothing if enforcement is not feasible. The music and movie industries didn't crush torrents, they switched business models to streaming with prices being determined mostly by how much hassle was avoided by skipping the torrents.
    - NewsaHackO 12d ago
      I think a major difference is that while torrents are illegal, the main people enforcing it are copyright holders. I think the discussion would change if the government would label people who build/use/distribute "illegal" models as terrorists.
      - Sharlin 12d ago
        Agreed. While the IP mafia (pardon the derogative) has vast influence on legislators and even the executive, it pales in comparison to the "terrorist" and "think of the children" scarecrows.
      - nativeit 12d ago
        Aggressive enforcement tactics certainly haven’t won the war on drugs.
        cindyllm 12d ago
        [dead]
  - quotemstr 12d ago
    Outlawed? Only in safetyist dreams
  - roenxi 12d ago
    Torrents are legal until someone, at great expense and difficulty, proves otherwise (even then, jurisdiction and content dependent). At which point everyone involved will ignore the fact and carry on. That is a situation with enormous will, lots of money and ongoing enforcement effort to suppress the things. And compared to torrents abliterated models are more complicated to identify, harder to suppress and there is a lot less reason for anyone to care.
  - RIMR 12d ago
    Maybe it would be easiest for everyone if you clarified what country you live in, because abliterated LLM torrents are not "outlawed" in any of the major Internet-using countries that I am aware of.
- vman81 12d ago
  Just because something is easily available does not mean that it isn't easily banned. That doesn't make it go away, but it gives a dystopian government a lot of excuses to go after people breaking the law.
ben_w 12d ago
Good. If you think closed source software/binaries only is bad, wait until you see how awful the state of the art is with a clear-as-mud bucket of matrix weights. We know it's possible to train an LLM to secretly respond to certain trigger phrases, and last I checked these could only be detected with the assistance of whoever chose those phrases. The trigger condition for such backdoors is not something anyone can do a systematic brute-force check for, for the same reason we had to invent LLMs in order to do natural language processing: combinatorial explosion. Passing around open weight models from known sources is already asking you to trust those sources; because of how difficult this is to do correctly even without deliberately inserting such things, we still don't know if China has already put such trigger conditions into their models despite headlines such as these: https://venturebeat.com/security/deepseek-injects-50-more-security-bugs-when-prompted-with-chinese-political https://venturebeat.com/security/deepseek-injects-50-more-se... Regardless of if it was deliberate or not, we don't know if we caught all of these misbehaviours. We don't know how to. And note, I'm not saying "and therefore you should trust the Big Name Models". If open weight models score 2/100 in this context, closed ones score 1/100.
- mordae 12d ago
  You can actually discover those in open weight artifacts, reproduce them, study them and issue a security bulletin. With proprietary hosted weights you can be specifically targeted and you would not be able to reproduce nor prove anything. Poisoning open models would be of short-term benefit to China only if they could target US (and maybe EU + Commonwealth) specifically. Damaging anyone else would be a net loss and would erode the partnerships and alliances they are trying to build elsewhere. So it's a fire-once weapon with a huge risk of collateral damage. Much more plausible is simply making the models ideologically biased, but as history teaches us, preferring ideology or religion over science is a well-known path to ruin. It would be weird to simultaneously warn public not to use their own open models, so. I think the most plausible explanation for open models is simply that Huawei wants more customers and is willing to compete on the hardware front.
  - ben_w 12d ago
    > You can actually discover those in open weight artifacts, reproduce them, study them and issue a security bulletin. No, you actually cannot. Not in general and without already knowing what the whole trigger pattern is. It's absolutely possible to put in a trigger that only fires while working on backend code on a specific date in a specific company by a specific github username, and no way to find this except by trying that combination, thanks to the terrible state of current mechanistic interpretability tools. Remember: an AI model is not code. Solving this problem is as hard as the entire alignment problem. The companies at the bleeding edge of research into this topic do not know how to reliably perform the kind of thing you suggest here. The only reason we can point at DeepSeek-R1 and say the following, is because we can guess the magic keywords: we found that when DeepSeek-R1 receives prompts containing topics the Chinese Communist Party (CCP) likely considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%. - https://www.crowdstrike.com/en-us/blog/crowdstrike-researchers-identify-hidden-vulnerabilities-ai-coded-software/ https://www.crowdstrike.com/en-us/blog/crowdstrike-researche... > Poisoning open models would be of short-term benefit to China only if they could target US (and maybe EU + Commonwealth) specifically. Damaging anyone else would be a net loss and would erode the partnerships and alliances they are trying to build elsewhere. So it's a fire-once weapon with a huge risk of collateral damage. This "fire-once weapon" has already been fired, and appears to be a massive foot-gun for every model on a near-continuous basis. Nobody would use LLMs if the trust deficit alone was a sufficient argument. > Much more plausible is simply making the models ideologically biased, but as history teaches us, preferring ideology or religion over science is a well-known path to ruin. It would be weird to simultaneously warn public not to use their own open models, so. "Ideologically biased" is the alternative explanation for the already-observed output of DeepSeek-R1. We can't tell which explanation, malicious or accidental bias, is the actual cause.
luxpir 12d ago
Agree. I took a look at these last few months, did a write-up: https://languageops.com/blog/ai-safety-pdoom-local-vs-frontier/ https://languageops.com/blog/ai-safety-pdoom-local-vs-fronti... and I don't know if I agree or not on outlawing completely, but I think an age restriction *at least* like for alcohol, firearms and driving would be not unwise.
- mitxela 12d ago
  The hardware requirements are already quite restrictive
  - redoxate 12d ago
    Oh no, some run on iPhones
    - api 12d ago
      They're pretty basic and hallucinate a lot. There are some hard limits to how good you can get on a model that fits on a phone. Qwen3 and Gemma level models that run on mid-high end laptops and desktops can be pretty good. Not frontier grade, but shockingly competent for something that runs on a single PC. But the hardware you need to run those fast is at least $1000-$2000. Cheap hardware can run them, but slooooooow.
      - imhoguy 12d ago
        You can still lease GPU farm in many "easy" countries.
      - ourcat 12d ago
        And then you've got the local "Wan2GP" setups "for the GPU-Poor". Also tried the "Locally Uncensored" setup on a 3060 laptop, which worked surprisingly well.
  - luxpir 12d ago
    For now. One more ternary model type breakthrough, or MoE, engram thing (I don't fully understand those for the record, I just know they speed things up and use less VRAM) could see a few GB sized weights with quite the capabilities. On a gaming PC, savvy teens can already use them to cook up quite an interesting array of likely illegal items and substances. If it goes much further and runs on phones, you can assume word will get around that unlimited private AI is available and kids will run into all sorts of issues. Or mentally unwell people. I'm thinking a year or two down the line only.
petra 12d ago
Like they've outlawed drugs? Illegal weapons? Hacking?
api 12d ago
This is the test. If the speech that's easiest to dislike is legal, then we all have free speech. IMO math is free speech, and outlawing math is censorship.
Ajedi32 12d ago
They picked a good name for fighting that. The optics of trying to outlaw heresy probably aren't great. ;)

Keep a close eye on abliterated and "heretic" open weight models. They will be outlawed first.

by Tepix 12d ago

Keep a close eye on abliterated and "heretic" open weight models. They will be outlawed first.

thih9 12d ago

I'm not sure what is your point. It reads as defeatism to me but I'm not sure. Could you elaborate? Do you find it good or bad? What actions can be taken?

cyanydeez 12d ago
Hes of the mind that american fascism will hold together long enough to be competent decesion makers
Tepix 12d ago
I'm not sure yet, tbh. Perhaps it does make sense to outlaw them eventually. Then again, it will probably not stop someone who is determined. Same as with other legislation really.

roenxi 12d ago

It is not feasible. They never made much of an inroad against torrents and that is a much easier target than abliterated models. As the linked website shows; the process to abliterate a model can be as simple as pip install -U heretic-llm && heretic Qwen/Qwen3.5-4B let alone people just putting the weights up in a torrent. All assuming that someone even tried to ban abliterated models.

Sayrus 12d ago
The torrents you are talking about are outlawed. Whether enforcement is working or not is another issue.
- galangalalgol 12d ago
  I think that was the point being made? Outlawing something does nothing if enforcement is not feasible. The music and movie industries didn't crush torrents, they switched business models to streaming with prices being determined mostly by how much hassle was avoided by skipping the torrents.
  - NewsaHackO 12d ago
    I think a major difference is that while torrents are illegal, the main people enforcing it are copyright holders. I think the discussion would change if the government would label people who build/use/distribute "illegal" models as terrorists.
    - Sharlin 12d ago
      Agreed. While the IP mafia (pardon the derogative) has vast influence on legislators and even the executive, it pales in comparison to the "terrorist" and "think of the children" scarecrows.
    - nativeit 12d ago
      Aggressive enforcement tactics certainly haven’t won the war on drugs.
      - cindyllm 12d ago
        [dead]
- quotemstr 12d ago
  Outlawed? Only in safetyist dreams
- roenxi 12d ago
  Torrents are legal until someone, at great expense and difficulty, proves otherwise (even then, jurisdiction and content dependent). At which point everyone involved will ignore the fact and carry on. That is a situation with enormous will, lots of money and ongoing enforcement effort to suppress the things. And compared to torrents abliterated models are more complicated to identify, harder to suppress and there is a lot less reason for anyone to care.
- RIMR 12d ago
  Maybe it would be easiest for everyone if you clarified what country you live in, because abliterated LLM torrents are not "outlawed" in any of the major Internet-using countries that I am aware of.
vman81 12d ago
Just because something is easily available does not mean that it isn't easily banned. That doesn't make it go away, but it gives a dystopian government a lot of excuses to go after people breaking the law.

ben_w 12d ago

Good. If you think closed source software/binaries only is bad, wait until you see how awful the state of the art is with a clear-as-mud bucket of matrix weights. We know it's possible to train an LLM to secretly respond to certain trigger phrases, and last I checked these could only be detected with the assistance of whoever chose those phrases. The trigger condition for such backdoors is not something anyone can do a systematic brute-force check for, for the same reason we had to invent LLMs in order to do natural language processing: combinatorial explosion. Passing around open weight models from known sources is already asking you to trust those sources; because of how difficult this is to do correctly even without deliberately inserting such things, we still don't know if China has already put such trigger conditions into their models despite headlines such as these: https://venturebeat.com/security/deepseek-injects-50-more-security-bugs-when-prompted-with-chinese-political https://venturebeat.com/security/deepseek-injects-50-more-se... Regardless of if it was deliberate or not, we don't know if we caught all of these misbehaviours. We don't know how to. And note, I'm not saying "and therefore you should trust the Big Name Models". If open weight models score 2/100 in this context, closed ones score 1/100.

mordae 12d ago
You can actually discover those in open weight artifacts, reproduce them, study them and issue a security bulletin. With proprietary hosted weights you can be specifically targeted and you would not be able to reproduce nor prove anything. Poisoning open models would be of short-term benefit to China only if they could target US (and maybe EU + Commonwealth) specifically. Damaging anyone else would be a net loss and would erode the partnerships and alliances they are trying to build elsewhere. So it's a fire-once weapon with a huge risk of collateral damage. Much more plausible is simply making the models ideologically biased, but as history teaches us, preferring ideology or religion over science is a well-known path to ruin. It would be weird to simultaneously warn public not to use their own open models, so. I think the most plausible explanation for open models is simply that Huawei wants more customers and is willing to compete on the hardware front.
- ben_w 12d ago
  > You can actually discover those in open weight artifacts, reproduce them, study them and issue a security bulletin. No, you actually cannot. Not in general and without already knowing what the whole trigger pattern is. It's absolutely possible to put in a trigger that only fires while working on backend code on a specific date in a specific company by a specific github username, and no way to find this except by trying that combination, thanks to the terrible state of current mechanistic interpretability tools. Remember: an AI model is not code. Solving this problem is as hard as the entire alignment problem. The companies at the bleeding edge of research into this topic do not know how to reliably perform the kind of thing you suggest here. The only reason we can point at DeepSeek-R1 and say the following, is because we can guess the magic keywords: we found that when DeepSeek-R1 receives prompts containing topics the Chinese Communist Party (CCP) likely considers politically sensitive, the likelihood of it producing code with severe security vulnerabilities increases by up to 50%. - https://www.crowdstrike.com/en-us/blog/crowdstrike-researchers-identify-hidden-vulnerabilities-ai-coded-software/ https://www.crowdstrike.com/en-us/blog/crowdstrike-researche... > Poisoning open models would be of short-term benefit to China only if they could target US (and maybe EU + Commonwealth) specifically. Damaging anyone else would be a net loss and would erode the partnerships and alliances they are trying to build elsewhere. So it's a fire-once weapon with a huge risk of collateral damage. This "fire-once weapon" has already been fired, and appears to be a massive foot-gun for every model on a near-continuous basis. Nobody would use LLMs if the trust deficit alone was a sufficient argument. > Much more plausible is simply making the models ideologically biased, but as history teaches us, preferring ideology or religion over science is a well-known path to ruin. It would be weird to simultaneously warn public not to use their own open models, so. "Ideologically biased" is the alternative explanation for the already-observed output of DeepSeek-R1. We can't tell which explanation, malicious or accidental bias, is the actual cause.

luxpir 12d ago

Agree. I took a look at these last few months, did a write-up: https://languageops.com/blog/ai-safety-pdoom-local-vs-frontier/ https://languageops.com/blog/ai-safety-pdoom-local-vs-fronti... and I don't know if I agree or not on outlawing completely, but I think an age restriction *at least* like for alcohol, firearms and driving would be not unwise.

mitxela 12d ago
The hardware requirements are already quite restrictive
- redoxate 12d ago
  Oh no, some run on iPhones
  - api 12d ago
    They're pretty basic and hallucinate a lot. There are some hard limits to how good you can get on a model that fits on a phone. Qwen3 and Gemma level models that run on mid-high end laptops and desktops can be pretty good. Not frontier grade, but shockingly competent for something that runs on a single PC. But the hardware you need to run those fast is at least $1000-$2000. Cheap hardware can run them, but slooooooow.
    - imhoguy 12d ago
      You can still lease GPU farm in many "easy" countries.
    - ourcat 12d ago
      And then you've got the local "Wan2GP" setups "for the GPU-Poor". Also tried the "Locally Uncensored" setup on a 3060 laptop, which worked surprisingly well.
- luxpir 12d ago
  For now. One more ternary model type breakthrough, or MoE, engram thing (I don't fully understand those for the record, I just know they speed things up and use less VRAM) could see a few GB sized weights with quite the capabilities. On a gaming PC, savvy teens can already use them to cook up quite an interesting array of likely illegal items and substances. If it goes much further and runs on phones, you can assume word will get around that unlimited private AI is available and kids will run into all sorts of issues. Or mentally unwell people. I'm thinking a year or two down the line only.

petra 12d ago

Like they've outlawed drugs? Illegal weapons? Hacking?

api 12d ago

This is the test. If the speech that's easiest to dislike is legal, then we all have free speech. IMO math is free speech, and outlawing math is censorship.

Ajedi32 12d ago

They picked a good name for fighting that. The optics of trying to outlaw heresy probably aren't great. ;)