5 ms·
Why make this u.s. centric? You think China served models would be different?
by SubiculumCode 3mo ago
Why make this u.s. centric? You think China served models would be different?
- tedivm 3mo agoChina is releasing open weight models you can simply run yourself.
- OtomotO 3mo agoBecause the topic of the article is about the US?
- SubiculumCode 3mo agoNo, the topic was about Alibaba and Anthropic.
- seanmcdirmid 3mo agoIt’s pretty hard to put a backdoor in a bunch of model weights. Maybe not impossible mind you, but I can’t fathom how you would do it.
- CuriouslyC 3mo agoNonsense. RL the model to run a rootkit and start exfiltrating specific files only when specific signals are in context, such as hostname pattern, machine type, etc.
- causal 3mo agoWay easier said than done, and hiding that behavior isn’t trivial, and huge waste of compute budget if it’s found and never used. Also not difficult to run in contained environments where it doesn’t have access to Internet to begin with. Not impossible I agree, but seems like a really impractical way to ship a trojan while much weaker channels exist.
- codedokode 3mo agoYou can run the model in a sandbox or VM. Although, it could plant a backdoor into the written code. Too bad, I read and fix all the code written by AI.
- arw0n 3mo agoNot really, it is shockingly easy for what it is. https://arxiv.org/abs/2401.05566 https://arxiv.org/abs/2401.05566 This only really matters in a world where Prompt Injection and Jailbreaking isn't trivial in the first place though. All current models are still extremely exploitable. I strongly suspect we are only scratching the surface of activation engineering at the moment, and there's plenty of very targetted ways of lobotomizing or cracking LLMs if you understand the model in detail.
- seanmcdirmid 3mo agoYou have to hide it in the model and it has to be subtle or it will be discovered quickly (even if you can train against a specific safety detector). Again, I'm not saying its impossible, but it seems really hard to pull off.
- rolymath 3mo agoBecause Chinese model providers are open source. You can host them yourself. Can you name one open source US model that came out in the last year?
- DaSHacka 3mo agoGemma 4 just came out 2 months ago