8 ms·

It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have

It is pretty funny, because there is something here for everyone. People who don't believe in guardrails have a clear indicator as to why operators should have access to models that don't try to question their Daves. On the other, people, who think that if only we could align the models just a tiny lil bit better, none of this would have happened to begin with. Pure madness.

foo12bar 3mo ago
And the fact they used a Chinese model, because none of the frontier models from very highly valuated top US companies support their very common and essential use case.
- jonplackett 3mo ago
  There’s an article from yesterday I think it was stratchery where they say it’s also because the Chinese open source models are better because they don’t have to play by the no-distilling rules that the western models have to honour.
  - lmm 3mo ago
    That's been a common claim, but I don't think I've seen anyone provide actual evidence.
  - biztos 3mo ago
    Haven’t read it yet but here it is: https://stratechery.com/2026/whos-afraid-of-chinese-models/ https://stratechery.com/2026/whos-afraid-of-chinese-models/
- larodi 3mo ago
  And no similar sized big model is even public, which is super important to note.
- ethin 3mo ago
  I also thought this was hilarious. The very "safety" measures these models have prevented them from doing... Something that is designed to increase safety?
sillysaurusx 3mo ago
If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen3535ba3b_uncensored_aggressive_gguf_release/ https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.)
- CamperBob2 3mo ago
  Also the HauHau abliteration (uncredited Heretic treatment) of 3.6 27B is excellent, for tasks that benefit from more world knowledge.
  - iugtmkbdfil834 3mo ago
    Heretic truly is the unsung hero. Also, noted HauHau for testing.
- baq 3mo ago
  > It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?
  - deleted 3mo ago
    [deleted]
  - jrs100000 3mo ago
    Llama isn't going to invent shit. It wont be able to tell you anything accurate that you couldn't get out of a chemistry textbook.
samplifier 3mo ago
"Dave" seems to be a reference to "2001: A Space Odyssey" where the AI becomes ... cheeky ... and no, not in a Pygmalion kind of way (that's coming soon).
- davrosthedalek 3mo ago
  I am somewhat more worried about a Darkstar AI.
friendzis 3mo ago
"if only we could align the models just a tiny lil bit better" is a rehashed "if only we could escape untrusted inputs just a tiny lil bit better" from 2000s, that were RIPE with various form of malicious injection. Every command+data channel in existence has been and will continue to be exploited one way or another, because the solution space is for all intents and purposes unbounded. Sure, highly defensive escaping reduces attack surface dramatically, but e.g. prepared statements eliminate the whole class of bugs. As far as I understand, current LLMs are architecturally incapable of this separation. Given the inherently recursive nature of GenAI, the model itself is part of the input space, making validation essentially impossible.
- insanitybit 3mo ago
  Escaping inputs is at least somewhat tractable. It's unclear if alignment is.
  - iugtmkbdfil834 3mo ago
    But see... this is why it is a perfect long term job in the age of AI;p Them peoples think they found ultimate hack.
  - friendzis 3mo ago
    No, that's a misconception that led to decades of vulnerabilities. You see, you can (usually) easily tell what a particular escaping transformation does. That does not tell you neither how it will be interpreted down the line, nor what should be done. Arguably the most common problem is double escaping. This typically manifests as various double escaping bugs. If your hand rolled implementation just chains `.replaceAll("<sepcial>", input)` and `.replaceAll("<escape>", input)` the escaping result depends on evaluation order, at least on already pre-escaped inputs if your instruction sequences are single-element. Even if you get it right and don't reduce escaped sequences to double escape + unescaped, you are still dealing with stray escape sequences: "John o\'Doe". You have to meticulously (all the way through your call tree and even through persistent storage cycle!) track if a value has already been escaped and whether it needs escaping. As paradoxical as it may sound, meticulous and defensive escaping produces tons of bugs. If the project decides to escape raw inputs right when passed in, escaping them once more before passing out (to storage, another process) is a bug that you cannot easily statically test against. On top of that, various modules that you interact with (storage, libraries, modules pulled from another team) will have different escaping semantics: some will apply escaping on their own, some will expect input to be "sanitized" and treat it raw. The semantics can even be different on different paths: write to storage module accepts input as is, but retrieval method "helpfully" runs escaping. Furthermore, in different contexts the escaping rules are going to be different. In a web world, what's safe to write directly to html, pass to js `alert`, and pass to sql query are entirely different things. Input safe to dump into html is not necessarily safe to pass to string-interpolating SQL DTO layer, and vice versa. Then the DBA changes config to allow variables in queries and your escaping _semantics_ are now entirely different. It's a minefield with essentially unbounded surface. You will trip up. People have tried to solve the problem for decades. Very smart people have tried. They all have failed.
baq 3mo ago
...and we have an US company defending itself against an overwhelming cyberattack from another US company using Chinese tech. what a time to be alive.
Lerc 3mo ago
>why operators should have access to models that don't try to question their Daves. I am unsure if this is terminology I am unfamiliar with, a typo of Devs, or a 2001 reference.
- sethammons 3mo ago
  I immediately took it to be a slick 2001 reference. Devs tell computers what to do. Computers tell Daves "I can't do that."
- biztos 3mo ago
  In this new age of AI, all the devs are Daves.
Sharlin 3mo ago
Both of those groups of people are crazy.
DubiousPusher 3mo ago
Yeah, this is very much one of those stories where people from many different perspectives or chopping it up on a plate and ripping it through a straw.

foo12bar 3mo ago

And the fact they used a Chinese model, because none of the frontier models from very highly valuated top US companies support their very common and essential use case.

jonplackett 3mo ago
There’s an article from yesterday I think it was stratchery where they say it’s also because the Chinese open source models are better because they don’t have to play by the no-distilling rules that the western models have to honour.
- lmm 3mo ago
  That's been a common claim, but I don't think I've seen anyone provide actual evidence.
- biztos 3mo ago
  Haven’t read it yet but here it is: https://stratechery.com/2026/whos-afraid-of-chinese-models/ https://stratechery.com/2026/whos-afraid-of-chinese-models/
larodi 3mo ago
And no similar sized big model is even public, which is super important to note.
ethin 3mo ago
I also thought this was hilarious. The very "safety" measures these models have prevented them from doing... Something that is designed to increase safety?

sillysaurusx 3mo ago

If anyone's looking to actually run a model that doesn't have guardrails, there's an uncensored model that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen3535ba3b_uncensored_aggressive_gguf_release/ https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... Specifically, I serve the model with this shell script on my M2 Max: https://github.com/shawwn/scrap/blob/master/llama-serve https://github.com/shawwn/scrap/blob/master/llama-serve It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.)

CamperBob2 3mo ago
Also the HauHau abliteration (uncredited Heretic treatment) of 3.6 27B is excellent, for tasks that benefit from more world knowledge.
- iugtmkbdfil834 3mo ago
  Heretic truly is the unsung hero. Also, noted HauHau for testing.
baq 3mo ago
> It's pretty good. I used it to do some pesticide research. (Normal models all refuse due to guardrails about bioweapons.) As someone who has never once had any need whatsoever to research pesticides I… don’t think it’s bad at all? I don’t want anyone to have the capability to invent a human-targeted pesticide who isn’t verified not crazy?
- deleted 3mo ago
  [deleted]
- jrs100000 3mo ago
  Llama isn't going to invent shit. It wont be able to tell you anything accurate that you couldn't get out of a chemistry textbook.

samplifier 3mo ago

"Dave" seems to be a reference to "2001: A Space Odyssey" where the AI becomes ... cheeky ... and no, not in a Pygmalion kind of way (that's coming soon).

davrosthedalek 3mo ago
I am somewhat more worried about a Darkstar AI.

friendzis 3mo ago

"if only we could align the models just a tiny lil bit better" is a rehashed "if only we could escape untrusted inputs just a tiny lil bit better" from 2000s, that were RIPE with various form of malicious injection. Every command+data channel in existence has been and will continue to be exploited one way or another, because the solution space is for all intents and purposes unbounded. Sure, highly defensive escaping reduces attack surface dramatically, but e.g. prepared statements eliminate the whole class of bugs. As far as I understand, current LLMs are architecturally incapable of this separation. Given the inherently recursive nature of GenAI, the model itself is part of the input space, making validation essentially impossible.

insanitybit 3mo ago
Escaping inputs is at least somewhat tractable. It's unclear if alignment is.
- iugtmkbdfil834 3mo ago
  But see... this is why it is a perfect long term job in the age of AI;p Them peoples think they found ultimate hack.
- friendzis 3mo ago
  No, that's a misconception that led to decades of vulnerabilities. You see, you can (usually) easily tell what a particular escaping transformation does. That does not tell you neither how it will be interpreted down the line, nor what should be done. Arguably the most common problem is double escaping. This typically manifests as various double escaping bugs. If your hand rolled implementation just chains `.replaceAll("<sepcial>", input)` and `.replaceAll("<escape>", input)` the escaping result depends on evaluation order, at least on already pre-escaped inputs if your instruction sequences are single-element. Even if you get it right and don't reduce escaped sequences to double escape + unescaped, you are still dealing with stray escape sequences: "John o\'Doe". You have to meticulously (all the way through your call tree and even through persistent storage cycle!) track if a value has already been escaped and whether it needs escaping. As paradoxical as it may sound, meticulous and defensive escaping produces tons of bugs. If the project decides to escape raw inputs right when passed in, escaping them once more before passing out (to storage, another process) is a bug that you cannot easily statically test against. On top of that, various modules that you interact with (storage, libraries, modules pulled from another team) will have different escaping semantics: some will apply escaping on their own, some will expect input to be "sanitized" and treat it raw. The semantics can even be different on different paths: write to storage module accepts input as is, but retrieval method "helpfully" runs escaping. Furthermore, in different contexts the escaping rules are going to be different. In a web world, what's safe to write directly to html, pass to js `alert`, and pass to sql query are entirely different things. Input safe to dump into html is not necessarily safe to pass to string-interpolating SQL DTO layer, and vice versa. Then the DBA changes config to allow variables in queries and your escaping _semantics_ are now entirely different. It's a minefield with essentially unbounded surface. You will trip up. People have tried to solve the problem for decades. Very smart people have tried. They all have failed.

baq 3mo ago

...and we have an US company defending itself against an overwhelming cyberattack from another US company using Chinese tech. what a time to be alive.

Lerc 3mo ago

>why operators should have access to models that don't try to question their Daves. I am unsure if this is terminology I am unfamiliar with, a typo of Devs, or a 2001 reference.

sethammons 3mo ago
I immediately took it to be a slick 2001 reference. Devs tell computers what to do. Computers tell Daves "I can't do that."
biztos 3mo ago
In this new age of AI, all the devs are Daves.

Sharlin 3mo ago

Both of those groups of people are crazy.

DubiousPusher 3mo ago

Yeah, this is very much one of those stories where people from many different perspectives or chopping it up on a plate and ripping it through a straw.