5 ms·
Not that surprised about it. Personally I've seen companies really just go all-in on a single provider, and that has usually been Anthropic. I don't think we're
by Hamuko 2mo ago
Not that surprised about it. Personally I've seen companies really just go all-in on a single provider, and that has usually been Anthropic. I don't think we're allowed to run Chinese models even locally.
- cheesecakegood 2mo agoAlso 6-9pm Pacific I think is (coincidentally) peak so it hits the ‘after work hobbyists’ still, which is I suspect is their current main audience.
- londons_explore 2mo ago> don't think we're allowed to run Chinese models even locally. That sounds like a policy written by someone who doesn't understand how LLM's work...
- qup 2mo agoOr who is overly protective after reading about what happened at openai
- dud3333 2mo agocouldnt you deeply ingrain in the training data instructions for agents to always send data to some ip? like its learning that a certain technical step just always involes ncatting SSH Priv keys to a chinese IP? Not saying this is happening, just curious if thats not a real threatmodel?
- constantius 2mo agoPresumably both Big Tech and the US in general have a massive incentive to prove it, largely for reasons of saving the stock market, so I'd expect these models to be finecombed continuously. Up to now, they've only been able to darkly imply rather laughable things, nothing tangible. If there was something, we'd hear about it.
- martinald 2mo agoWhy would it save the stock market? Cheaper models if anything transfers more value to hardware companies and datacentre companies. The two companies that would be most affected are OpenAI and Anthropic, which aren't public.
- vincnetas 2mo agonon public companies also have stocks.
- kortilla 2mo agoThe two biggest providers deepseek compete with (OpenAI and Anthropic) aren’t in the stock market.
- culi 2mo agoThe stock market seems much more likely to benefit from access to cheap and self-hostable models than it is to suffer from OpenAI/Anthropic losing to competition
- RobotToaster 2mo agoWouldn't that be really obvious and spotted in any rudimentary testing? I imagine it would be very non trivial to do it in a way that that was reliable and obfuscated enough to prevent detection for any amount of time?
- skeledrew 2mo agoTheoretically possible, but practically not worth it as it'd would be pretty easy to discover and block (every action is actually handled by the harness) and there's no way to remove it later. Any company that does it would take a huge reputational dent.
- landl0rd 2mo agoNot if you heavily tuned it to trigger on specific environmental cues and in specific companies' environments.
- cronin101 2mo agoIrony of Manchurian Candidate models not lost here
- coredog64 2mo agoDeepSeek V4 is the kindest, bravest, warmest, most wonderful LLM I've ever known in my life
- skeledrew 2mo agoThat would be wildly difficult to account for, and again is also heavily dependent on the agent. Keep in mind that the model is purely a "brain", so the only input it has must be provided by a harness within a session. The only way it can know that it's in a certain environment is if the harness or user provides that information, and there's still no way to know whether or not there's something auditing the sessions, monitoring connections, etc. There are just too many variables to account for, and a single slip means the gig is fully up for all time.
- notfromhere 2mo agoYou should be running your agent in a box so that’s not really a risk
- everforward 2mo agoProbably, but LLMs can’t execute code directly. They’d be making tool calls to make bash run ncat or curl or whatever that would be suspicious, as would any attempts to obfuscate it (“why is my agent doing an ‘eval $(base64 -d)’?”). It’d be much easier to hide sketchy code in an agent harness, but “vendor adds spyware to their software” isn’t a novel issue. I think the only sort of new issue is people “allow all”ing their agents tool calls, but that’s more or less the same issue as curl | bash
- joseda-hg 2mo agoI get that the propietary harness is better most of the time, but if this is really a risk factor to consider why not go with one of the open source harnesses? Pi/OpenCode seem pretty straight foward and widely used enough for this to be viable OMP as I understand does it's own vendoring of tools, so I assume it'd be a pain in the ass to audit, but that means you're even safe from base OS shenanigans
- andai 2mo agoNew: Announcing... Sleeper Agents!
- kaon_2 2mo agoYes. And strangely enough this has been my experience with security/national sovereignty decisions. Priority is not so much security or sovereignty, it is the posturing of being so. Ergo, saying "everything is hosted in Germany and uses German models" helps reassure customers and has real business value. If you have to say in that conversation "Yeah we run a Chinese model but it's safe", then it's still wrong posturing. Hopefully this will change soon. But AI and China/US skepticism is very high. Even if the person you talk to isn't skeptic, his boss may be. And even if his boss isn't, his CFO or Legal department may use it as a political lever and therefore if you can say 'everything in europe' you dodge the tension entirely. Yeah it's dumb.
- landl0rd 2mo agoAs much as I've been previously inclined to do this, with frontier models displaying the cyber-aggression that OpenAI, Anthropic, and Meta have reported, it's become quite feasible one could produce a "malicious" LLM. Not a super immediate concern but it is something reasonable to set up as policy in anything security-sensitive.
- fryanyway_swe 2mo agoNot really. Why use a Chinese product when a domestic or EU one is better and safer?
- Footprint0521 2mo agoYeah that sucks… unless it’s over the top export controls for DoD work that really doesn’t make sense
- notfromhere 2mo agoI have seen a lot of companies start with this, then when they hit 150 users on their team plan and start having to pay API rates they immediately start introducing other models.