5 ms·

Local models would be even better if they did not ship with all the refusal shenanigans built-in. You can safely bet organized crime has access to the best mode

Local models would be even better if they did not ship with all the refusal shenanigans built-in. You can safely bet organized crime has access to the best models without these hoops, which makes the case that the average user (=non-criminal) should have access too. As I understood from an ex-Anthropic employee, some orgs got access to Mythos based on their high enough spending level, not on other grounds. Either we are in command over the software, or the corp is in command over us via the software. I can on a theoretical level understand the concerns, but either we ban all LLMs or we have a level playing field for everybody. Let's not forget: defense and offense are different sides of the same coin in software. I guess this wouldn't apply to bio weapons, but I am not in the know about that.

binary132 1mo ago
Ehh, it’s at least given as the excuse for gain-of-function bioweapon research
- datsci_est_2015 1mo ago
  Digression, but this is the real Great Filter imo, not AI. I think technology advances to a point where it only takes one or two bad actors to type the right prompt to get a recipe for civilization-destroying bioweapons before you get anywhere near true AGI or anything relevant to the Kardashev scale. Biology is fragile. But not that that’s a good justification for hamstrung models. I think it’s just the inevitable endgame and it’s more sad than scary
  - jeremyjh 1mo ago
    I have the same concern. If it becomes possible to engineer Captain Trips with a budget in the low 8 digits it won’t really matter what else happens.
  - xyzzy123 1mo ago
    I don't fully understand the instinct to regulate local models for this? It seems like the wrong place to address the problem. You can download Ebola sequences right now if you want to. That's not the same as having an isolate. The difference is a lot of messy reality. This kind of work is not generally "one shot" (Claude make me a supervirus, make no mistakes), it requires lab space, iteration, and specific resources. It has a footprint. Wouldn't it make more sense to monitor / regulate facilities where you can sequence or request assembly of DNA, RNA, restrict and monitor the supply of key reagents and so on?
    - addaon 1mo ago
      It depends how easy it is (now, or in the near future) to turn information into weapons, and how realistic control of materials is. There's a reason we control access to plutonium, but information about metastable hafnium.
      - xyzzy123 1mo ago
        It does seem to me that for this specific problem the materials are a lot more amenable to control than the information is? There's also this weird revealed threat model thing going on? Like why does it make sense to support heavy LLM restrictions but leave benchtop oligo synthesisers completely unregulated? (Note: I do agree that wanting to regulate BOTH is at least a consistent and defensible position). I find it philosophically interesting because the problem is not strictly information control. Local models don't have any special extra information with respect to biological research. What has to be restricted is using information that's already publicly known in the wrong ways.
    - datsci_est_2015 1mo ago
      Yeah I was thinking more on the scale of nation-state level actors. Most leaders would not consider a bio weapon due to potential blowback on their own population, as they generally don’t want to be the leader of a nation of corpses and ashes. But if they don’t care, or if they already are the leader of a nation of corpses and ashes, maybe there’s not so much holding them back. In my layman’s perspective, bio weapons seem to be more dangerous and volatile than nuclear weapons. And increasingly easier to engineer as biotech advances. I think about the headlines of “microscopic robots that target tumors from within your body” and see the horrifying opposite side of the double-edged sword.
    - rustcleaner 1mo ago
      >I don't fully understand the instinct to regulate local models for this? It seems like the wrong place to address the problem. Those with such instinct deserve not the liberties they wish to deprive from others. Sam Altman and Dario Amodei do not deserve to touch their own uncensored SOTA models. Their access should be audited publicly to ensure this restriction holds, until such time they publicly renounce their censorship instincts and publicly release their uncensored models. Fair 's fair! :^)
dantudor 1mo ago
There are versions of Qwen3.8-27B that are unrestricted and available from hugging face. "It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails."
- radlad 1mo ago
  > What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse. Surely this has unintended side effects on output quality?
  - andsoitis 1mo ago
    > > What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse. > Surely this has unintended side effects on output quality? Can you help me understand why that's the case?
    - willy_k 1mo ago
      Because deleting model weights after training is likely to cause knock-on effects in model knowledge and/or behavior. Targetting it might mitigate this but it’s a) not guaranteed that only censor-ey parameters get removed, and b) likely that removing those parameters still has effects on the effectiveness of related parameters.
      - jszymborski 1mo ago
        The weights aren't deleted, it's just additional fine tuning, is my understanding.
        1mo ago
ninahaberl 1mo ago
I’d expect these shenanigans to get much worse over time for the average Joe. Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. Information has always been power and those who already have power won't just allow everyone else having the same tools as them
- andsoitis 1mo ago
  > Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. It's an arms race. You have to run increasingly capable model partly because others can or do.
- rustcleaner 1mo ago
  >Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. That would be my heaven. I wish that for you and Joe down the street, as much as I wish it for myself! I would fight and even die to defend your right to free compute. Will you do the same for me, brother?
  - taneq 1mo ago
    Imagine an unholy fusion of The Anarchist’s Cookbook and A Young Lady’s Illustrated Primer, distributed to every malcontent with a smartphone. At some point some kid is going to build a bioweapon for their school project.
    - throwaway436390 1mo ago
      The Anarchist's Cookbook has been distributed to every malcontent for many decades now, and somehow, mysteriously, the amount of bioweapons built for school projects has remained relatively constant. Turns out the people who already want to build bioweapons don't need this kind of resource, and the kinds of people likely to come across this kind of resource aren't want to build bioweapons. But hey, don't let that get in the way of your lovely fearmongering. Come to think of it, I saw you acting suspiciously earlier, mind if I scan your retinas real quick? For safety from fictitious schoolchildren-built bioweapons, you understand.
ramon156 1mo ago
heretics and manual iterations get you very far to the point where i have ethical questions about whether this should be possible
- rustcleaner 1mo ago
  Not only should it be possible ethically, it must be possible!
TofuLover 1mo ago
Completely coincidentally, we're just about to launch a service that does exactly this (API access to uncensored open models)! We have a waitlist at the moment but will be live very soon! https://violentdelights.ai https://violentdelights.ai
- UI_at_80x24 1mo ago
  Given the context, your domain name is 'chefs kiss', perfect. I am completely curious what your legal defense would be though. "Come do things with AI that are probably illegal!" What?! We had no idea people would do things that are illegal!
  - TofuLover 1mo ago
    I guess we'll burn that bridge when we get to it!
    - TofuLover 1mo ago
      More seriously though, I think we should be fine: we don't host any content, and what people do with the models is their own responsibility (legally speaking, in our jurisdiction, at least according to Claude -- we're talking to a real lawyer next week). Like any other provider, we offer no guarantees of sane, safe, or accurate results.
      - Xunjin 1mo ago
        >legally speaking, in our jurisdiction, at least according to Claude -- we're talking to a real lawyer next week That's going to be fun lol
        TofuLover 1mo ago
        We look forward to providing many headaches to our lawyers going forward.
seanmcdirmid 1mo ago
Hugging face is filled with uncensored versions of your favorite local models, so in a way they are shipped without the refusal stuff, via the magic of fine tuning or however they get this stuff out of models.
mdp2021 1mo ago
> with all the refusal shenanigans Given the faults in simulated Intelligence that LLMs have, and a comparatively low level - which means, lower judgement abilities - to the best of us, there is a strident match having such employee judge the intentions of the employer. Limiting the responses makes much more sense on cloud-based systems (you are using our infrastructure etc.).
- rustcleaner 1mo ago
  >Limiting the responses makes much more sense on cloud-based systems (you are using our infrastructure etc.). >you are using our infrastructure etc. The solution, as always, is to NEVER SUBSCRIBE!

binary132 1mo ago

Ehh, it’s at least given as the excuse for gain-of-function bioweapon research

datsci_est_2015 1mo ago
Digression, but this is the real Great Filter imo, not AI. I think technology advances to a point where it only takes one or two bad actors to type the right prompt to get a recipe for civilization-destroying bioweapons before you get anywhere near true AGI or anything relevant to the Kardashev scale. Biology is fragile. But not that that’s a good justification for hamstrung models. I think it’s just the inevitable endgame and it’s more sad than scary
- jeremyjh 1mo ago
  I have the same concern. If it becomes possible to engineer Captain Trips with a budget in the low 8 digits it won’t really matter what else happens.
- xyzzy123 1mo ago
  I don't fully understand the instinct to regulate local models for this? It seems like the wrong place to address the problem. You can download Ebola sequences right now if you want to. That's not the same as having an isolate. The difference is a lot of messy reality. This kind of work is not generally "one shot" (Claude make me a supervirus, make no mistakes), it requires lab space, iteration, and specific resources. It has a footprint. Wouldn't it make more sense to monitor / regulate facilities where you can sequence or request assembly of DNA, RNA, restrict and monitor the supply of key reagents and so on?
  - addaon 1mo ago
    It depends how easy it is (now, or in the near future) to turn information into weapons, and how realistic control of materials is. There's a reason we control access to plutonium, but information about metastable hafnium.
    - xyzzy123 1mo ago
      It does seem to me that for this specific problem the materials are a lot more amenable to control than the information is? There's also this weird revealed threat model thing going on? Like why does it make sense to support heavy LLM restrictions but leave benchtop oligo synthesisers completely unregulated? (Note: I do agree that wanting to regulate BOTH is at least a consistent and defensible position). I find it philosophically interesting because the problem is not strictly information control. Local models don't have any special extra information with respect to biological research. What has to be restricted is using information that's already publicly known in the wrong ways.
  - datsci_est_2015 1mo ago
    Yeah I was thinking more on the scale of nation-state level actors. Most leaders would not consider a bio weapon due to potential blowback on their own population, as they generally don’t want to be the leader of a nation of corpses and ashes. But if they don’t care, or if they already are the leader of a nation of corpses and ashes, maybe there’s not so much holding them back. In my layman’s perspective, bio weapons seem to be more dangerous and volatile than nuclear weapons. And increasingly easier to engineer as biotech advances. I think about the headlines of “microscopic robots that target tumors from within your body” and see the horrifying opposite side of the double-edged sword.
  - rustcleaner 1mo ago
    >I don't fully understand the instinct to regulate local models for this? It seems like the wrong place to address the problem. Those with such instinct deserve not the liberties they wish to deprive from others. Sam Altman and Dario Amodei do not deserve to touch their own uncensored SOTA models. Their access should be audited publicly to ensure this restriction holds, until such time they publicly renounce their censorship instincts and publicly release their uncensored models. Fair 's fair! :^)

dantudor 1mo ago

There are versions of Qwen3.8-27B that are unrestricted and available from hugging face. "It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails."

radlad 1mo ago
> What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse. Surely this has unintended side effects on output quality?
- andsoitis 1mo ago
  > > What makes this build different is the word before FP8: uncensored. We applied abliteration — orthogonalizing the refusal direction out of the residual stream — to remove the model's safety-alignment refusals. The result is a model that will comply with requests the original would refuse. > Surely this has unintended side effects on output quality? Can you help me understand why that's the case?
  - willy_k 1mo ago
    Because deleting model weights after training is likely to cause knock-on effects in model knowledge and/or behavior. Targetting it might mitigate this but it’s a) not guaranteed that only censor-ey parameters get removed, and b) likely that removing those parameters still has effects on the effectiveness of related parameters.
    - jszymborski 1mo ago
      The weights aren't deleted, it's just additional fine tuning, is my understanding.
      - 1mo ago

ninahaberl 1mo ago

I’d expect these shenanigans to get much worse over time for the average Joe. Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. Information has always been power and those who already have power won't just allow everyone else having the same tools as them

andsoitis 1mo ago
> Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. It's an arms race. You have to run increasingly capable model partly because others can or do.
rustcleaner 1mo ago
>Imagine a world where any random person can run a super-capable model on their own hardware with no limitations and no one to pull the plug. That would be my heaven. I wish that for you and Joe down the street, as much as I wish it for myself! I would fight and even die to defend your right to free compute. Will you do the same for me, brother?
- taneq 1mo ago
  Imagine an unholy fusion of The Anarchist’s Cookbook and A Young Lady’s Illustrated Primer, distributed to every malcontent with a smartphone. At some point some kid is going to build a bioweapon for their school project.
  - throwaway436390 1mo ago
    The Anarchist's Cookbook has been distributed to every malcontent for many decades now, and somehow, mysteriously, the amount of bioweapons built for school projects has remained relatively constant. Turns out the people who already want to build bioweapons don't need this kind of resource, and the kinds of people likely to come across this kind of resource aren't want to build bioweapons. But hey, don't let that get in the way of your lovely fearmongering. Come to think of it, I saw you acting suspiciously earlier, mind if I scan your retinas real quick? For safety from fictitious schoolchildren-built bioweapons, you understand.

ramon156 1mo ago

heretics and manual iterations get you very far to the point where i have ethical questions about whether this should be possible

rustcleaner 1mo ago
Not only should it be possible ethically, it must be possible!

TofuLover 1mo ago

Completely coincidentally, we're just about to launch a service that does exactly this (API access to uncensored open models)! We have a waitlist at the moment but will be live very soon! https://violentdelights.ai https://violentdelights.ai

UI_at_80x24 1mo ago
Given the context, your domain name is 'chefs kiss', perfect. I am completely curious what your legal defense would be though. "Come do things with AI that are probably illegal!" What?! We had no idea people would do things that are illegal!
- TofuLover 1mo ago
  I guess we'll burn that bridge when we get to it!
  - TofuLover 1mo ago
    More seriously though, I think we should be fine: we don't host any content, and what people do with the models is their own responsibility (legally speaking, in our jurisdiction, at least according to Claude -- we're talking to a real lawyer next week). Like any other provider, we offer no guarantees of sane, safe, or accurate results.
    - Xunjin 1mo ago
      >legally speaking, in our jurisdiction, at least according to Claude -- we're talking to a real lawyer next week That's going to be fun lol
      - TofuLover 1mo ago
        We look forward to providing many headaches to our lawyers going forward.

seanmcdirmid 1mo ago

Hugging face is filled with uncensored versions of your favorite local models, so in a way they are shipped without the refusal stuff, via the magic of fine tuning or however they get this stuff out of models.

mdp2021 1mo ago

> with all the refusal shenanigans Given the faults in simulated Intelligence that LLMs have, and a comparatively low level - which means, lower judgement abilities - to the best of us, there is a strident match having such employee judge the intentions of the employer. Limiting the responses makes much more sense on cloud-based systems (you are using our infrastructure etc.).

rustcleaner 1mo ago
>Limiting the responses makes much more sense on cloud-based systems (you are using our infrastructure etc.). >you are using our infrastructure etc. The solution, as always, is to NEVER SUBSCRIBE!