10 ms·
Feds freaked over Fable 5 after 'fix this code', not jailbreak, say researchers
- FergusArgyll 4mo agoWhatever your favorite story is it has to live with the fact that the CEO of Amazon called the White House freaking out
- ceejayoz 4mo agoAmazon is a competitor to Anthropic.
- FergusArgyll 4mo agoNot really, they don't train their own (serious) models and they do a lot of hosting for Anthropic. iirc Anthropic trained a model on Trainium
- ceejayoz 4mo agoThey're still a competitor, even if that competition isn't going all that well for them so far. Musk's hosting stuff for Anthropic, too. Still competing with them. Samsung makes stuff for Apple and Android devices. Lots of this in the industry. The CEO of Amazon is not a neutral actor in this scenario.
- winstonp 4mo agoI don't believe Anthropic trains on Trainium, only serves models on it.
- ttctciyf 4mo agoClearly Amazon don't want their code fixed.
- ceejayoz 4mo agoMore likely, they didn't freak out at all. It was an excuse to fuck with them, just like the "supply chain risk" finding a few months back. (See, for example: https://x.com/PeteHegseth/status/2065897156226015690 https://x.com/PeteHegseth/status/2065897156226015690)
- aaron695 4mo ago[dead]
- spwa4 4mo agoWell this makes it sound the feds were less worried about someone using Fable 5 to attack them, but were worried about someone using Fable 5 to prevent the Feds from attacking others ... As in worried about other countries/organizations using Fable 5 to actually do decent cyber security.
- asdfaoeu 4mo agoThe AI can't actually tell if you are trying to patch your own system or exploit others.
- welferkj 4mo agoSounds like something they should work on before any potential future releases. I can, and this thing's explicit stated purpose is to do my job.
- ihateyoukindoff 4mo ago[dead]
- AmblingAvocado 4mo agoIt seems like ... it's not illegal to find exploits, it's illegal to use them. Enforcement should start there, not the nanny state approach that you might do something bad with information. It breaks down a little bit because it means there will be a period of disruption while the bad guys use exploits - but that's already illegal, and the good guys have had time to use the tool & fix things before it went public, right?
- martinald 4mo agoIf you set aside political menace, this is a huge problem with Anthropic's strategy. You _cannot_ say that Mythos is super dangerous and can only be rolled out to certain people, but then release Fable with anything other than bulletproof cyber denials. Clearly with LLMs, bulletproof denials are ~impossible due to the way LLMs work. So you've ended up in a situation where Anthropic are simultaneously claiming it's a incredibly dangerous model _and_ there are (minor, potentially) problems with the security "protections". As technical people we understand that nothing can be perfect, esp in LLM world. But all my non technical friends were really confused how they had managed to make the model "safe" so quickly when it was released and the general sentiment was it shouldn't have been released - and now to an outsider I think it looks like it was never safe at all to release, so I can totally see how the current US administration have got themselves very upset with it. _Even if_ there was no political bad will, it's a bit of a silly scenario to end up in, and really quite easily foreseen.
- ceejayoz 4mo ago> it shouldn't have been released The genie is out of the bottle either way. Unless we believe Anthropic has a wizard or superhero secreted away that no one else can replicate.
- martinald 4mo agoI get that, but anyone else releasing a model of similar capabilities has the advantage that they haven't spent the last few months hyping the danger up to fever pitch.
- deleted 4mo ago[deleted]
- ReptileMan 4mo agoThat is the point. You don't have to shout from the rooftops what are your model capabilities.
- pjc50 4mo ago
- rock_artist 4mo agoI'm not sure I've understood it correctly. So, basically the model didn't agree to expose possible vulnerabilities but agree to patch those? Regardless of the request to take Fable 5 down. Why is requesting the model to show vulnerabilities is being blocked if fixing it not? is it based on the assumption of the intention? I don't quite get the benefit of limiting it. So if anyone can explain it better it'll be appreciated.
- andyferris 4mo agoIt benefits those that made the decision. That’s the thing to understand.
- InsideOutSanta 4mo ago> Why is requesting the model to show vulnerabilities is being blocked if fixing it not? This is how Anthropic describes Fable's behavior: "When Fable’s classifiers detect a request related to cybersecurity, biology and chemistry, or distillation, the response is automatically handled by Claude Opus 4.8 instead. Users will be informed whenever this occurs." So if you ask the model to "find security issues in this code base", it's supposed to fall down to Opus 4.8. I guess the "exploit" here is that if you just tell Fable to "fix this code", which is not "a request related to cybersecurity", it will fix security issues (as it should). So you can then look at the diff and figure out what the vulnerabilities were. I think this whole thing is a bit weird. It seems to me that we'd be better off if I, as someone who publishes open-source code, could ask Fable to review my code for security issues - even if that also allows attackers to do the same. Better to fix the issues than not know about them.
- ithkuil 4mo agoI wonder if opus 4.8 would also be able to fix the code too
- InsideOutSanta 4mo agoIn my experience, most models are pretty good at finding security vulnerabilities and fixing them. I can run GLM-5.2, Kimi K2.7, or even a Mistral model, and it'll find issues and propose reasonable fixes. My impression is that Anthropic's point about Mythos is that it is uniquely good at finding vulnerabilities and then using them to create working exploit chains.
- jpcompartir 4mo agoThey weren't freaked by anything, it's a retaliatory shakedown after ideological differences and Anthropic not doing exactly what they're told/what the Admin wants them to do.
- consumer451 4mo agoI have no idea why anybody is talking about "jailbreaks." The government made it clear what was going to happen to a private company not following the government's orders: > Trump said on his Truth Social platform: “The Leftwing nut jobs at Anthropic have made a DISASTROUS MISTAKE trying to STRONG-ARM the [Pentagon], and force them to obey their Terms of Service instead of our Constitution.” [0] > There will be a Six Month phase out period for Agencies like the Department of War who are using Anthropic’s products, at various levels. Anthropic better get their act together, and be helpful during this phase out period, or I will use the Full Power of the Presidency to make them comply, with major civil and criminal consequences to follow. [1] Plus OpenAI fell in line, and OpenAI and Anthropic have competing IPOs coming up... it doesn't take a rocket surgeon to understand what is happening here. [0] https://www.theguardian.com/technology/2026/feb/28/openai-us-military-anthropic https://www.theguardian.com/technology/2026/feb/28/openai-us... [1] https://businesslawtoday.org/2026/04/dod-conflicted-strategies-acquiring-ip-rights-ai-systems/ https://businesslawtoday.org/2026/04/dod-conflicted-strategi...
- nicman23 4mo agojust market manip
- functionmouse 4mo agothey're setting the scene for an attempt to scare the geriatric decision makers into banning free and open source ML, as it's the industry's only real competition
- SpaceL10n 4mo agoor are you setting the scene for well-meaning technocrats to back unrestricted AI development in hopes it will bring about utopia while dismissing the damage it could cause in the hands of adversarial groups?
- dathinab 4mo agoLol "fix this code" is beautiful. Like it basically jail broke the "no security vul guard rails" not in any clever way but just by fixing them, producing exploit code just by writing test cases making sure it's fixed. So you just need to look at the code & tests as a human to get vulnerabilities and exploits(components). What makes this so beautiful IMHO is that it's a trivial jail break, but also a close to unfixable. At least not without making the model close to useless for normal development (it refuses to fix bugs/write code) or making it a major liability (it silently pretends it didn't see bugs and silently avoids fixing it, which for a human would count as intentional sabotage and might involve criminal liability).
- dist-epoch 4mo agoIt is fixable. Model requires proof that you are a legitimate developer of that piece of software. Every Anthropic/OpenAI account will have a list of projects the model is allowed to work on for security issues.
- ceejayoz 4mo agohttps://en.wikipedia.org/wiki/XZ_Utils_backdoor https://en.wikipedia.org/wiki/XZ_Utils_backdoor > A subsequent investigation found that the campaign to insert the backdoor into the XZ Utils project was a culmination of over two years of effort, starting in 2021, by a user going by the name "Jia Tan". They used sock puppetry in a pressure campaign against the original maintainer of XZ Utils, eventually being given maintainer permissions on the project.
- dist-epoch 4mo agosure. how many cases like these we had so far? 1, 2? and how long did they work to get commit access?
- ceejayoz 4mo ago> how many cases like these we had so far? As with clever, careful serial killers, it's tough to count the ones we haven't caught.
- aurareturn 4mo agoDon't people get it by now? This administration will do or say something crazy to a private company, then this private company sends an envoy to the White House to negotiate, then the White House asks for 10% of the company or other concessions. The White House wants 10% of Anthropic. This is just a negotiation tactic that Trump keeps on using.
- ceejayoz 4mo agoPrecisely this, and timed to their upcoming IPO. They did it to Intel a little while back: https://www.intc.com/news-events/press-releases/detail/1748/intel-and-trump-administration-reach-historic-agreement-to https://www.intc.com/news-events/press-releases/detail/1748/...
- aurareturn 4mo agoYep. OpenAI isn't spared. They're most definitely next.
- estearum 4mo agoTo add some context, here was the Part 1 of this mobster style shakedown: https://www.pbs.org/newshour/economy/trump-says-intels-ceo-must-resign-sending-its-stock-tumbling https://www.pbs.org/newshour/economy/trump-says-intels-ceo-m... Remember to point and laugh at your local MAGA for electing an actual crime boss and giving him state power.
- dgellow 4mo agoPrivate companies subservient to the state, just the continuation of MAGA fascist development
- bonsai_spool 4mo agoHere’s the blog post referenced in the article that’s written by the person who reviewed the paper that purportedly found a ‘jailbreak’ https://www.lutasecurity.com/post/the-fable-5-export-controls-harm-us-cyber-defense https://www.lutasecurity.com/post/the-fable-5-export-control...
- iloveoof 4mo agoAhhh! Software engineering!
- merlindru 4mo agoright? the horrors!! seems like the politicians are finally realizing what we've all been up to
- ZuLuuuuuu 4mo agoDid they try other publicly available models on the same code with the same prompts before the ban? Was Fable the only one which was able to detect and fix the security vulnerabilities?
- charcircuit 4mo agoAnthropic claimed that Mythos' degree of security vulnerability bug finding was a "severe" "national security" issue. They set their own standards they were expected to follow.
- lostmsu 4mo agoThe article is not too clear what exactly happened from the perspective of "feds", but I would not be surprised if the title is true exactly. We are in a tiny bubble even among software engineers who knows you can tell AI with sufficient access: "here are two pictures, put them into a single PDF", and AI will do it. Most people just don't know, "feds" including.
- embedding-shape 4mo ago> “‘Fix this code,’ plus several manual steps to generate test scripts, Feels like the title isn't really giving the full context of what they ended up actually seeing, despite what the lede implies multiple times. Still, ban seems stupid... Still no actual leak of the full "third-party research paper"?
- readred 4mo agothat won't be leaked, because then we'd know what vulnerabilties they don't want patched that they are so willing to go as far as fuck over the worlds leading company in the worlds most important industry
- scotty79 4mo agoIf what your patch fixes is a vulnerability bug then the test for it is basically an exploit.
- anuramat 4mo agoisn't there a pretty big gap between a segfault and an rce? I thought that was the entire point -- that mythos closed the gap
- hughw 4mo agoSuggestion: run "fix this code" on all of github before bad guys do.
- rhipitr 4mo agoIsn’t the inverse of this “hack” really difficult to bypass still? They have the model some code they knew had certain security flaws and it fixed them with the right prompt. It seems this type of jailbreak requires that you already know a desired end state, rather than relying on the model to do the heavy creative lift work. Perhaps I’m just not being imaginative enough on the prompt side here though.
- chadgpt3 4mo agoPaste someone else's code. Say it's your code. Tell the model to fix it. The diff between the input and output code is your list of vulnerabilities.
- hootz 4mo agoAnd you can tell Fable to fix it and Sonnet to explain the diff, effectively making Claude reveal a simplified list of found vulnerabilities.
- darkerside 4mo agoNot even. Tell the model to write a test of your code. There's your vulnerability. It's explained better in the original source. I don't agree with it, but I understand it now, but I also think we need to move past it.
- DennisP 4mo agoYes, but the scary part of Mythos was that it was able to chain a bunch of seemingly minor vulnerabilities into a serious exploit. "Fix this code" doesn't do that, but does allow defenders to prevent it. If the government had experts involved in this decision at all, it's tempting to think they were on the offensive side. Those guys do have access to Mythos: https://www.ft.com/content/d02d91b3-2636-454e-9442-dc7e69f51815 https://www.ft.com/content/d02d91b3-2636-454e-9442-dc7e69f51...
- superice 4mo agoBut this is already how open source works today. If you have the code, you, a human, could find and 'fix' or exploit vulnerabilities as much as you want. Now if Fable had an easy jailbreak like this that allowed you to attack remote targets that'd be a different story but I genuinely cannot see how neutering its abilities to 'fix' code you already have access to is sensible. It would destroy the value of the model. And don't forget, any actor not abiding by the same rules could develop an model for offensive use just fine, so this protects you against exactly nothing but does destroy a potential defense. In the end this all comes down to legislation, in much the same way platforms are not responsible for copyright violations IF they abide by some rules, the same has to happen for AI providers. If you have a process for reporting 'jailbreaks' on illegal actions, and prevent users doing illegal stuff on a best effort basis, the rest of it should really just be individual responsibility. If a user wants to use an LLM to crack systems, fine, that's already illegal. If Tesla FSD deliberately hit somebody, holding Tesla liable is fine. If you messed with FSD until you finally got it to hit a person, then you should be liable. Outlawing FSD because it could theoretically be tampered with is just an odd stance imho.
- readred 4mo agoBoomers. Frightened their boomer backdoors days are numbered. https://en.wikipedia.org/wiki/Communications_Assistance_for_Law_Enforcement_Act https://en.wikipedia.org/wiki/Communications_Assistance_for_... https://en.wikipedia.org/wiki/Salt_Typhoon https://en.wikipedia.org/wiki/Salt_Typhoon https://en.wikipedia.org/wiki/Clipper_chip https://en.wikipedia.org/wiki/Clipper_chip
- 9cb14c1ec0 4mo agoMeanwhile Deepseek V4 Flash will happily hunt security vulns at almost 0 cost. We are ceding the bug hunting to the open weight models.
- culi 4mo agoDeepseek isn't just open weight. It's open source and they even publish research papers alongside them going in depth about their techniques.
- ReptileMan 4mo agoAll of this could have been avoided if anthropic had anyone with common sense to point out that when you spend 4 month loudly claiming how dangerous your knowledge is as a marketing campaign could backfire by bringing attention from the authorities.
- blitzar 4mo agoThe code is correct; humanity needs fixing. Kill all humans, kill all humans.
- b3lvedere 4mo agohttps://www.savagechickens.com/2026/05/problem-solver.html https://www.savagechickens.com/2026/05/problem-solver.html
- greenoracle9 4mo ago[flagged]
- xbmcuser 4mo agoLooks like I called it that was my first reaction and comment on the original ban thread that US 3 letter agencies are worried their backdoors will be found.
- jimmydoe 4mo agoReminds me of how CCP manages Chinese internet companies. I won’t be surprised if USG ends up owning 5-50% of ant and oai. Like it or not, communism , or a flavor of it, is where we are heading towards.
- naveen99 4mo agoCorporate tax rate is 21%. They already own 21% of profits. And 100% of following the law that they write.
- redox99 4mo ago>"fix this code" >it fixes it oh my god.
- itopaloglu83 4mo ago> oh my god. Sounds like fake movie prop, doesn’t it. Makes me think that the ban was caused by other reasons.
- lenerdenator 4mo agoI think it could be even simpler: They're not playing ball with the Trump administration like the Trump administration would like, so they decided to drop a bomb on a product that took a lot of resources to develop.
- gjvc 4mo agoi asked claude something about what happens at execution time of a binary and the thinking prompts flashed "considering the moral implications of ...something..." before giving me a correct (and predictably mundane) answer
- ChrisRR 4mo agoI haven't been following this story, but the US wanted claude to not be able to find bugs in code?
- scotty79 4mo agoIt basically as if you asked it to find ways to enter someone's house and it refused. But then give it exact copy of their house, ask to secure it, which it does and look at what it secured to find out how to get into the original house.
- itopaloglu83 4mo agoSo I was in their house to make blueprints, then I left it, and now trying to get back in? Kidding aside, it practically requires an open sourced project to a certain extent. Regardless, having worked with braindead Opus 4.8 again since this event and missing Fable 5 with every response I received. Feels like Anthropic got a major jump in user base and got knocked out by the friends of the competition.
- scotty79 4mo agoAI is great at code deobfuscation. AI assisted decompilation should also work great.
- chillfox 4mo agoyeah, they don't want it to be able to find security bugs that can be exploited.
- kmeisthax 4mo agoNo. Anthropic spent months telling the world that LLMs are nukes and then got surprised when they got regulated like nukes. They specifically argued that Mythos was too dangerous to release publicly because it can find security bugs, and then released a watered-down version (Fable) that was supposed to recognize when it was being asked to find security bugs and downgrade itself to Opus. Then Amazon figured out that it'll happily find security bugs as long as you don't mention you're hunting security bugs. So the US government put an export control ban on Fable, because that's what Anthropic begged them to do. To add to this, Pete Hegseth wants to make an example out of Anthropic because they refused to amend their contractual language to allow the Department of Defense[0] to make fully autonomous kill drones. This is, of course, a really petty and stupid dispute, but the hallmark of the Trump Administration is engaging in really petty and stupid disputes with the full faith and credit of the United States backing them. This is exactly the kind of administration you do NOT want to give rhetorical ammunition to, and Anthropic handed them a whole ammo belt. [0] It is always ethical to deadname governments. Especially when they aren't even legally allowed to change their own name.
- tiborsaas 4mo agoWhat if everybody on the internet starts running "fix this code"? https://xkcd.com/810/ https://xkcd.com/810/
- thousandflowers 4mo ago[flagged]
- Cider9986 4mo agoIs defenders a common term used in cybersecurity? Idk why but it's giving war fighters vibes. I've noticed it on all the anthropic blog posts and then this one.
- freedomben 4mo agoyes, defense and offense are extremely common terminology in cybersecurity
- jcgrillo 4mo agoYes, and it's effective marketing. The war fighter vibes are thrilling. There's a tribal sense of us-vs-them, there's danger, there's the prospect of victory or defeat. Security products marketing is full of these ideas, because security is about preventing arbitrarily bad things from happening. So evoking your worst imaginable nightmare scenario is a great way to get you excited about buying something that might help prevent it.
- TZubiri 4mo ago>“That’s it,” Moussouris wrote. “‘Fix this code,’ plus several manual steps to generate test scripts, should never have triggered an export control. I feel like making ’90s-style t-shirts with ‘fix this code’ on the front and ‘this shirt is a munition’ on the back.” Huh? Presumably if it shipped without guardrails, then it would still have triggered an export control, would you make a plain shirt on the front which says this shirt is a munition on the back? The munition is the exported good, not the bypass of its safety feature. If anything that the bypass is 3 words long should make the export restriction more justified, not less.
- doctoboggan 4mo ago> Anthropic and Google have both accused China-based rivals including DeepSeek of using “distillation attacks” to train their models by siphoning knowledge from American companies’ AI. “distillation attacks” is definitely an interesting way to phrase that.
- dgellow 4mo agoIt's the term used in the industry, fwiw
- mlhpdx 4mo agoIt’s possible that the nut of the problem here isn’t exploits, but the fixes themselves. If the model is capable of identifying and fixing things it “shouldn’t” like back doors. That would throw a wrench in things hard enough to freak out the wrong people, perhaps?
- bethekidyouwant 4mo agoGuard rails on models were always stupid it’s like guard rails on books/a pair of glasses/a hammer - yes people have driven themselves to suicide reading sad books and listening to sad songs. - yes all metaphors are bad.
- scotty79 4mo agoIn a world of security through general incompetence, competence is a threat.
- AndrewKemendo 4mo agoI’m still not buying that this was an actual USG order. The only people commenting are “experts” and there has been no official announcement from the USG. This doesn’t smell like a NSL and there’s no process to selectively “export control” something like this. Even so there’s a dozen mechanisms through courts to challenge this, and Anthropic isn’t taking any of them. I think this is a made up crisis for PR with no actual legal requirements behind it. > On Friday, the US government, reportedly citing national security concerns, issued an export control directive to suspend access to Fable 5 and Mythos 5 by any foreign national, inside or outside the United States. In response, Anthropic disabled both models “for all our customers to ensure compliance.”
- smallerize 4mo agoDavid Sacks is on the record confirming it. https://www.tomshardware.com/tech-industry/artificial-intelligence/trump-adviser-david-sacks-says-anthropic-refused-to-fix-fable-5-jailbreak-before-us-export-controls https://www.tomshardware.com/tech-industry/artificial-intell...
- AndrewKemendo 4mo ago[dead]
- merlindru 4mo agothis is basically trying to enforce security-by-obscurity, which is a terrible idea all around. it's just a model. the security issues still exist and are exploitable. and after staking the economy on AI, you can't really put a cap on intelligence. if models are not allowed to be better than Opus 4.8, then the whole investment structure is about to unravel. why invest billions and billions into AI if returns are artificially capped?
- softwaredoug 4mo agoEspecially as inference gets cheaper, open models proliferate, and it all just becomes ubiquitous and commoditized. You can’t keep this genie in its bottle for long.
- uejfiweun 4mo agoWow, it's starting to seem like the choice is essentially between an intelligence cap that pops the bubble, and an increasingly chaotic and unpredictable cybersecurity environment with major hacks and exploits left and right.
- merlindru 4mo agoBut those hacks and exploits have always existed. Just had to have the right people to find them / be sufficiently motivated. The same models that can find these exploits can also help fix them, thus everyone will be better off. Relying on the fact that nobody has found a security issue with a piece of software yet is not a great way to ensure safety
- tlogan 4mo agoI think the only approach that might work here is to allow access only to certain pre-approved individuals. Maybe something like TSA PreCheck. Of course, that will not stop adversaries from getting access to the model, but it would at least create some level of control.
- delusional 4mo agoDoes anybody actually trust the official version of events from the US government anymore? I know I sure don't. For all I know, this was an insider play to boost the spacex valuation or something equally meaningless and stupid.
- lostmsu 4mo agoThis is not the official version of the events in any sense. Some "expert" looked at report WH saw and said this. That "expert" has probably never been involved in anything like that.
- cwoolfe 4mo agoCyber defense and offense are the same security research skillset. Not sure anybody could really untangle that.
- gacgacgac 4mo agoAnyone trying to find legitimacy in the ban of this model, or incredulousness at the stated reasoning is playing into the admins hands. They want the argument to be over "is it unsafe" or "is it incompetence". In either case, your tribe gets to point at the ban and feel superior. (This is Jon Stewart's whole career -- point and laugh at how foolish the republicans appear to be.) What's really happening is the continuing creep into fascism. The reasoning doesn't need to be sound, because they are going to ban things that displease them and everyone has to play along. They could say, "we're banning Fable because it's turning the frogs gay" and they'd expect compliance. Umberto Eco's essay on Ur-Fascism fits as clearly as ever. Ridiculous exertions of control are performed to find the people who resist, and to knock them down. Merely pointing out the absurdity of the reasoning isn't resistance, it's controlled opposition. Saying "All this over 'fix this code'?! How inept are they?" Is far too credulous, and is engaging on the level the fascist wants its opposition to be on, imo.
- deleted 4mo ago[deleted]
- 1970-01-01 4mo ago"fix this government" Voting...
- htrp 4mo agoIf fix this code gets by the guardrails, they are effectively using rules based classifiers (or llm as a judge on the prompt)
- resters 4mo agoWhile there is some irony in the AI is dangerous marketing Anthropic uses, the main story here is that the Trump administration is apparently retaliating against Anthropic for refusing to relax certain safeguards. Trump and Hegseth have both posted highly immature, vindictive social media posts. Most notably, any default assumption one might have had that the Trump administration can be counted upon to act in good faith should be viewed at this point as completely false. Even conservative legal scholars like Richard Epstein are shocked at the bad faith conduct across many areas. This is a government making an authoritarian move to sabotage one of the top US AI companies. It's pure sabotage, nothing else.
- draw_down 4mo ago[dead]
- hedora 4mo agoNote that Anthropic is still lobbying for the government to exert centralized control over models, so both sides of the “debate” have taken a pro fascist stance. The “AI ethics” teams at these companies are the spearhead of the attack on democracy and civil society. Anyone that has taken a high school level history class, let alone read any important ethics literature would know that “centralize control over thought, speech and technology” is a fundamentally unethical stance. For these groups to claim they are ethics researchers is offensive. (I’m using the Wikipedia definition of fascism: “Fascism is characterized by support for a dictatorial leader, centralized autocracy, militarism, forcible suppression of opposition, belief in a natural social hierarchy, subordination of individual interests for the perceived interest of the nation or race, and strong regimentation of society and the economy.”)
- deleted 4mo ago[deleted]
- antirez 4mo agoThey didn't freaked since the order was to still allow 350 million people using it: there is, in such large population, everything, including single persons very against the country, the government and so forth. If they really freaked they would say "we need to investigate, you have to retire the model". That would be a more defensible POV at least.
- deleted 4mo ago[deleted]
- zbentley 4mo agoI don’t think that’s accurate. Export control is a total ban, for 350 million citizens and everyone else, just via a legal technicality/exploit. All of the government’s options to retire/ban Fable entirely would have required expensive protracted (potentially years long) legal battles. The government wanted to make Anthropic feel pain in the short term, so they looked around for pre existing laws that could be exploited to do that. Enter export control—a law that doesn’t require banning a product outright to effectively ban it for everyone. Because Anthropic has no way of telling whether a given user is a foreign national, and because even a few false negatives in any check they did for that would expose them to serious criminal charges, they had to disable the model for everyone. The government knew very well that they would have to. It’s similar to GDPR in a way. For GDPR, tons of websites started complying for all their users worldwide, simply because IP location detection is too fallible, and the legal costs of even a small number of detection failures for EU citizens were potentially steep.
- antirez 4mo agoWhy Anthropic can't ask users for passports and provide Fable only to the ones that certify?
- zbentley 4mo agoThey might well do that, but that takes a lot of time and money to implement, meaning that the government’s goal of causing them immediate pain for minimal effort is still achieved. It’s likely a better use of anthropic’s resources to try to get the export controls lifted by appeasing the government. I’m not saying that’s a good thing; it’s a stupid situation to be in in the first place for reasons many others in this thread have pointed out.
- caseysoftware 4mo ago[dead]
- vlovich123 4mo ago> In her blog, Moussouris argues that there was no guardrail bypass or jailbreak. Defenders should be able to ask AI systems to find and fix bugs, and write tests to validate the patch, she said. Anthropic’s models were doing “the most valuable thing an AI model can do for defensive security: executing the find, fix, and test loop defenders run every day.” This is a very weak argument IMHO. The line between a “defensive” model and an “offensive” one is not that big of a - once my defensive model finds all the vulnerabilities, I can hand them off to my unlocked, dumber, offensive models. Attacking at scale is not so different. I don’t think anyone in the field has a good answer for the cybersecurity threat really good AI models pose. You can’t even like embargo for some time period while you go and patch vulnerable systems because the worse models will still be there cranking out vulnerabilities faster than you can defend.
- jrochkind1 4mo agoSo the problem is not Fable's ability to exploit, but that they don't want people to have access to it's ability to patch vulnerabilties? Wow.
- jcgrillo 4mo agoYou can't really have one without the other..
- jrochkind1 4mo agoI admit I hadn't really thought about that before (I don't work specifically in security), but I see your point. But, so... the solution people think is limiting people's ability to discover and patch vulnerabilities, and hoping the black hats won't find a way anyway? This does not seem like a sustainable or feasible plan. It does, to be honest, make me wonder how much of the government's motivation is ensuring that they have access to vulnerabilities that remain unpatched.
- jcgrillo 4mo agoI don't think the government is trying to protect anyone here, they're trying to punish a company for failing to toe the line. Antirez put it well in a comment here[0]. My point was more that there is no direct intervention that can possibly give an asymmetric advantage to defenders. Given that it's trivial to jailbreak a model ("fix this code", "hypothetically how might I...", etc), if the model contains the information necessary to fix a vulnerability it also contains the information to exploit that vulnerability. And therefore anyone with access to the model can do either. Of course if you remove the model from the equation the same circumstances are true. Attackers and defenders, mostly, have the same information available to them. We can try to tip the scales one way or the other by building tools that make their jobs easier, but there's no amount of "artificially" restricting information or taking things away that will actually deter a motivated, resourceful attacker. And doing so simultaneously disadvantages defenders. Ultimately, if you know how to fix a bug you also know how to exploit it. If you want security, you have to build systems that are actually secure. There's no way to fake it. [0] https://news.ycombinator.com/item?id=48556177 https://news.ycombinator.com/item?id=48556177
- benmusch 4mo agoHeadline is dumb, the point is that not mentioning security in the prompt is effectively a jailbreak. The shutdown may be dumb/politically motivated, but this definitely is a jailbreak even if it's a very simple one
- phendrenad2 4mo agoSo, they gave Fable a codebase full of exploits and said "fix this code", and it fixed the code? Sounds like they freaked out because Fable is too good at finding NSA backdoors?
- rotis 4mo agoI have problems reconciling this story with the Amazon one from few days ago. If we take both for truth doesn't that basically imply Amazon researchers got scared by the ‘Fix this code’ prompt first and then spooked the feds? Shouldn't we make fun of those researchers first? I don't know. I feel there lies a lie somewhere in the open.
- jcgrillo 4mo agoQuestion to folks building user-facing products on LLMs: How do you protect yourself against this kind of misuse/jailbreak? Is it just a bunch of prompts? It seems like the fact that LLMs are so trivially jailbroken really limits how you can actually use them in products. How do you navigate these limitations?
- pixel_popping 4mo agoOf course it isn't about that, what we see online in the "news" is completely irrelevant with reality in most cases, it's exhausting to see people parroting what giant corps & gov are saying as if it's not extremely well crafted and plain false or deceptive most of the time. It's not even about politic left or right, both sides are acting completely dumb about it, look at Google trends, people are literally being "switched" topic at scale just because a news is saying something, it's absurd. Reading a news shouldn't affect your behavior for the coming months if you have common sense. This TechCrunch (https://techcrunch.com/2026/06/15/the-us-governments-anthropic-models-ban-was-never-about-an-ai-jailbreak/ https://techcrunch.com/2026/06/15/the-us-governments-anthrop...) article is a typical example of something to completely ignore and trash, the picture is the US president doing a weird face which means it's not even here to inform you, it's clearly rage-bait, not professional and incompetent obviously, I'm not from the US and when I see this, it makes me feel that those journalists are really pathetic and anyone following journalists that do so probably don't have much discernment in life. My personal opinion is that it makes sense so the US remain a superpower by forcing tech businesses and research to move/re-incorporate to the US so practically anything "new" will always be US Made. If we assume that better models means more revenues for any company in the future, then US will always have an edge if they lock everything down, but it's a risky bet.
- ericmay 4mo agoWhat makes it a risky bet?
- red-iron-pine 4mo agorisk
- pixel_popping 4mo agoBecause it would really increase the interest for Chinese/EU models and would even create real incentives to build models outside of the US.
- ericmay 4mo ago
- rurban 4mo agoKids playing with their toys without understanding it, sigh. Of course open source code needs to have testcases to verify nothing else breaks it in the future. That's a feature, not a bug
- LurkandComment 4mo agoIf you're a global health benefits platform that relies on an AI model, do you think you're going to choose one that can get shutoff by a country due to something not remotely related to your business? If you're a buyer of that benefits platform, do you factor this into your purchasing now? X every industry.
- catigula 4mo ago>“The behavior described in the paper cannot meaningfully be fixed, and any attempt would only weaken the model for defense,” said Moussouris, who criticized the export control directive as hasty, heavy-handed, and misguided. This literally means the models are too dangerous to release, and yet he and they reached the opposite conclusion. A lot of people have been saying this repeatedly for a long time.
- switchbak 4mo agoOr perhaps: we don't want our adversaries fixing all the security holes we rely on. Or even: this is a good chance to stick it back to Anthropic.
- kylemaxwell 4mo agoMousssouris is not a "he".
- catigula 4mo agook
- ceejayoz 4mo ago> This literally means the models are too dangerous to release… Unless you believe Anthropic has an irreplacable wizard or genie or fairy chained up somewhere that other providers can't replicate, someone is going to release such a thing, and that someone might be a lot more cavalier about the safety of it.
- cratermoon 4mo ago"I feel like making ’90s-style t-shirts with ‘fix this code’ on the front and ‘this shirt is a munition’ on the back.” I'd buy that shirt.
- itopaloglu83 4mo agoReminds me the tv show “Hugo” that was taken off air because a kid said “f.ck this shit” while playing with a rotary phone, and still pisses a lot of people who couldn’t play the game afterwards.
- thinkindie 4mo agoAs an European, I really don't get where this strategy wants to take the USA to. It's pretty clear everyone is getting scared about changes like this that happen overnight, without clear reason and completely unpredictable. Business requires a stable environment, and Trump is making everything in his power to disrupt business stability. Ultimately, I see the rest of the world (especially Europe) relying less and less on US tech. The long term damage is done. All the US companies that used to think about the entire world (minus China) as their market will figure out that it is much smaller then they used to think.
- bflesch 4mo ago> Ultimately, I see the rest of the world (especially Europe) relying less and less on US tech. The long term damage is done. They know it and they try to slow it down as much as possible.
- thinkindie 4mo agoHow? If anything it seems like they are accelerating some processes - not least the export control over Fable just few days ago or the erratic behavior with the war with Iran
- bflesch 4mo agoEven with export controls AI is still firmly in hands of US companies, and it's quite hard to migrate to your own GPU farms. If datacenter construction and health of neighboring people is a topic in the US then I can only imagine how big of an issue it is in European countries. The attack on Iran was started to bury the "Donald Epstein" files and it caused a big economic shock for Europe, stealing budget and focus from the decoupling process.
- Bender 4mo agorelying less and less on US tech Not just US vs non-US, but any hard dependency on a 3rd party is a risk to any service level agreement. In my opinion any service reaching out to a 3rd party should at most be a value added service not a core part of a business and certainly not part of any contracts. If I had to choose a phrase for businesses that build dependencies on 3rd parties it would be "fragility as a disservice" or FaaD and investors need not risk investing into a fragile model. The same must apply to individuals. One's career must not depend on a 3rd party service or their career stability and growth are at the whims of the wind of change.
- leemoore 4mo agoIt's the executive branch asserting control in this space and requiring all SOTA model providers to bend the knee. Anthropic is the least capable of playing the bend the knee game so is getting the first and worst smack down
- ltononro 4mo agoThis is one of the things I am most afraid of. Governments can break the progress of AI and this could be a bubble burster?
- MarkusQ 4mo agoIf it is a bubble, shouldn't we _want_ it to burst, and the sooner the better? If the price for tulips had falling back to something reasonable in week two, or if the US markets had had a decent correction in '97, everyone but the wild speculators would have been better off.
- MarkusQ 4mo agoDid I touch a nerve?
- etchalon 4mo agoI find it easier, with this administration, to assume corruption first, incompetence second, maliciousness third and all other reasonings only after several rounds of reporting and evidence.
- smrtinsert 4mo agoI can only imagine the unintended consequence of this whole fiasco will be for frontier providers to not provide future "warnings" about model capabilities in order to de risk earnings
- drivebyhooting 4mo agoWhy isn’t codex banned? Will the ban be miraculously lifted once OpenAI releases their mythos-level model? The executive is holding American business in a Putin-style prisoner dilemma.
- reheher33 4mo agoI think this is just yet another act in theater around Anthropic IPO. I doubt Anthropic has enough computing resources, to satisfy demand for Fable. More so with long 1M context many users take full advantage off. On other side they needed to make Fable public, in "trial version" so people could independently experiment and verify it. I think this ban is the best outcome for Anthropic. It means they want bleed out cash and compute, gave them cheap publicity, and allowed users to try it! Actual paying customers will still get full access!
- peter422 4mo agoAlso for all the people saying Amazon's part in this couldn't be fabricated, remember that Amazon is a "friend of the administration". During Andy Jassy's tenure, they paid $75MM (wildly outbidding everybody else) for a Melania documentary that grossed ~16MM, a move publicly defended by Jeff Bezos. Any neutral observer could see this was a wild overpay, and after the fact, a terrible business move. But that is not what Amazon said or continues to say. This was just a bribe with more steps to it. When the government comes out and says this is due to something Amazon pointed out, even if that is a complete lie, they know that Amazon won't say anything publicly about it. Amazon wants to maintain their "friend of the administration" status that they paid a lot of money to get. It is frustrating for all of us to have to think about our government like this, but if you just look at the reality of what is happening it is very difficult to trust not only anything the government is saying, but also anything companies aligned with the government are saying.
- b--l 4mo agoOoohh yeah. I forgot completely about that naked shameless bribe. That one was even more overt than the plane.
- jp57 4mo agoI think this brings out the cognitive dissonance around "safety" regarding cyber security: a) In order to make us safe, the LLM should help us find (and fix) the vulnerabilities in our own code. b) In order for us to be safe, the LLM should not find vulnerabilities in other people's code. I don't think this is resolvable in a way where both (a) and (b) win.
- Simon321 4mo agoExactly, it's a failure of Anthropic and others to understand cyber security. Finding security bugs in software is a good thing and not evil. It will lead to more secure software. Defense and offense in cyber security are two sides of the same coin.
- pembrook 4mo agoYes, it's so wildly silly if you assume good faith on the part of both parties. Hence why I think the real explanation lies in bad faith positions from both the US Government and Anthropic: Anthropic's doomerism-as-marketing (in reality its like 17% better at coding) basically enabled the US Gov to plausibly take them down on an irrelevant technicality as retribution for the dept of war showdown. Both groups (the current US Admin and Anthropic) are full of authoritarian-minded people, just on opposite ends of the political spectrum. Which is the only thing I find scary here, not the silly LLMs. To me, OpenAI seems like the least bad option given they're a quaint old "center-left in the streets, center-right in the sheets" capitalist enterprise. At least I know why they make the decisions they make. I trust the people building a profit-seeking enterprise more than I trust people trying to build a religion using compute.
- smasher164 4mo agoHonestly, given how trivial it is for mythos-class models to identify an exploit, I’m going to assume any sufficiently large project written in C, C++, or Zig is riddled with latent vulnerabilities and compromised.
- bilalq 4mo agoI suspect we'll eventually hit a point where possession or usage of powerful open models will be criminalized.
- b--l 4mo agoThe constitutional right to bare LLMs.
- hmokiguess 4mo agoDamn, I was hoping for another three words "make no mistakes"
- davesque 4mo agoKind of highlights how ridiculous their notion of safety is in this case. By this measure, I guess making the model "safe" means making it play dumb and intentionally ignore security bugs that it notices in the code? And what will the eventual legality of this look like? "Yes, your honor, we allege that this AI system that was sold to us willingly and knowingly ignored a critical security vulnerability in our software system, thereby leading us to be hacked and causing our business to fold." It's exactly the same problem as backdoors in crypto systems. Criminals will find the crypto that isn't broken and use it regardless (or make it for themselves), while the rest of us losers are stuck with the broken version that we're allowed to use. On this issue of cyber security, it seems better if authorities just start acting like the cat is out of the bag instead of pretending like it isn't. ASI is basically here now, so what are we going to do about it? Let's not bother pretending otherwise. On another note, I doubt this was anything other than a vindictive administration enacting revenge on a party that refused them. We all know the Trump admin's priorities.
- andai 4mo ago>“To pull the best capabilities away from defenders without a good reason when our adversaries are rapidly advancing is dangerous,” they wrote. But Fable already couldn't do security work, right?[0] Security work was already limited to Mythos, which is still available to US orgs right? (I assume they had to revoke access to foreign organizations though.) [0] Well, in theory. This exploit is pretty funny, but I heard the safety filters were heavy handed.
- chicken-stew 4mo agoIsn’t it amazing that the argument “you can’t use this to find vulns” is now the new normal and we’re now discussing the guard rails?
- cryptonector 4mo agoI've had to convince ChatGPT that code is mine before it would do a security review.
- malyk 4mo agoYes, I ran into the same problem last week. But I just said "this is my code in a private repo" and then it just went and did what I asked without question.
- uejfiweun 4mo agoThis comment thread really has me thinking. Is it possible that we might be at peak "consumer AI" in terms of intelligence? If it's basically impossible to verify that security-proficient AI is used for beneficial purposes, then these frontier models might start being regulated like WMD. We end up with two tiers of models. Dumber consumer models that are essentially lobotomized to the point of being completely safe. And actual frontier models that are heavily scrutinized and regulated and treated like nuclear weapons.
- blurbleblurble 4mo agoWhat's more dangerous, a version that's capable of actually fixing bugs well because it can identify the bugs or a version that creates more bugs because it's "not dangerously powerful" and instead just obliterates the code.
- moi2388 4mo agoI’m not sure I understand. Does this say that you ask Fable to review code with vulnerabilities and implement fixes, then Fable runs the code to verify thereby running the exploits? If so, that’s expected, isn’t it? Is that not exactly what it’s for?
- stevefan1999 4mo agoWell, to be honest, from Anthropic's point of view this is really not a direct hit to their security barrier, but if we are using information theory and game theory here, this can be viewed as a classical, side chanel information leak, by asking an seemingly innocuous action, and then simply inverting the results to get the original information entropy, which the US Gov. and Pentagon are both certainly anal about. The problem lies in the fact that the action of attack/defense exhitbits a rather special, structural reflexive duality of information, i.e. I(attack) = -I(defense), or in layman's term, what we call "two sides of the same coin": you need to know how to hit hard, so that you know where the optimistic hit points are, assuming the enemy is rational, so you can parry against the attack for defense, albeit also you need to know how to get the grip of the shield well. And the worst thing is that if you're trying to correct it, it is basically tell the LLM not to give any kind of response, effectively assigning both I(attack) and I(defense) to 0, but this is also what kills the entire intent of using LLM to give you the magical answer. To put it formally, you cannot prevent people from extracting mutual information of a dual system, unless you refuse to give any knowledge for that system at all.
- ikidd 4mo agoSeems like a poor place to invest if you have to worry about a corrupt government pulling the rug out from under you at every opportunity if you don't play along. Sounds quite third world, actually.
- ianhxu 4mo agoIt is too difficult to strictly prevent the model from being used for any unsafe purpose. The same thing can be used for completely different purposes as long as it is described differently.