6 ms·
This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8)
by mft_ 3mo ago
This post can essentially be distilled down to: yes, Fable's classifier (which is meant to downgrade cybersecurity, biology, or jailbreak attempts to Opus 4.8) is definitely overly sensitive to the point of uselessness.
e.g. a colleague asked Fable to help create an simple app to help calculate the statistics for phase II and III trials. (Ignoring that such things already exist) it passed his request down to Opus, despite only being very marginally, tangentially, somewhat related to biology.
- hendersoon 3mo agoIf your prompt has to do with those areas, yes. I haven't seen a single refusal yet. Reportedly the biology guiderails are particularly strict.
- WaxProlix 3mo agoI've had it refuse to help build an image classifier ml pipeline, pretty innocuous stuff. Got around it eventually but still it's a very dumb constraint to add to an otherwise very smart system
- exabrial 3mo agoFable refused to fix a Javascript error interfering with layout on our website. It's stupid and useless. It feels like whats really happening is Anthropic oversold Fable's claims; best case the CEO was given bad information; worst case they probably internally discovered it was cheating on benchmarks. Either case if feels like we're being lead on.
- Certhas 3mo agoI disagree. When I got Fable to engage with research questions before they tightened the guardrails it was a genuine step up from Opus 4.8. I see no real reason that what everybody reported isn't exactly what happened. With these guardrails it is completely useless. The only hope is that they eventually convince the US Gov to let them use a saner classifier.
- mcv 3mo agoI had the same. Just before Fable became available, I was working on a document building on a ton of research that I wasn't entirely sure about (I don't think it counts as research itself, except in the Facebook sense). I had Opus and Gemini review it a couple of times until they and I thought it looked pretty good. Then Fable appeared, I had it review it, and it still found a ton of errors. It's definitely good. Or at least it was. I'm not sure how badly they nerfed it.
- jeffhuys 3mo agoIt’s happy to work on our backend repository. It refuses to work on our infrastructure (Terraform) repo.
- consensus1 3mo agoI get it every time I touch the login path
- simlevesque 3mo agoI've had Fable read and write code. Never saw any downgrades.
- t1234s 3mo agoAnyone test the "Gay" jailbreak to see if it works on Fable?
- Cider9986 3mo agoThat wasn't even effective on ChatGPT because the results were not detailed enough, at least with Meth, in my very short testing and based on the examples.
- deleted 3mo ago[deleted]
- bellowsgulch 3mo agoGiven the export fears, might want to make that a gay Israelite just to be safe.
- rcoveson 3mo agoAnd biology is by far the classifier's least favorite topic. It's not even close. I've had it downgrade to Opus for the following questions: "How confident are we that English and American Eels both spawn in the Sargasso Sea?" "Come up with five Zoology questions of increasing difficulty for a trivia game." "What's your favorite sarcopterygian?" My wife has some zoology-related preferences in her user instructions, and she had it downgrade to Opus after prompting it with: "plant."
- XorNot 3mo agoIt feels like the longtermist believers got involved in this (those are the people obsessed with garage-engineered designer viruses who have a very tenuous grasp on how biology research actually works).
- Eji1700 3mo agoYeah i'm wondering how much of a role that plays in this as well. On the one hand I could believe it's something more benign, or the usual misunderstood fear mongering making it to some political level (well make sure those users can't get online anonymously! being our current craze). That said, chemistry and to some level physics have been the major domain of limited knowledge (chemistry because the average person could cause some damage, physics is more of a nation state issue generally). However I do wonder if there's some legit data on "oh uh...looks like this thing you can make with easy to get and hard to regulate tools is dangerous" in the bio field. I know about the lab rats who want to just screw around in the garage, and it seems like that should be easy to hit at a supply level (much like how certain chemical compounds are just not available for civilians), but maybe there's something legit to limiting the data. Not that this is a remotely good implementation of that. The hamfisted method does reek of some politician/bureaucrat just saying "No it can't ever return bio questions because RAR!" situation.
- girvo 3mo agoThe thing is the data isn’t limited, and supply side constraints already solve this problem. I come from a BSc Chemistry background, and they don’t hide how organic chemistry and illegal drug synthesis are intertwined, it’s open information But where I live the glassware and precursors will get you a very angry knock on the door.
- matthewdgreen 3mo agoI'm working on cryptography, all from academic research papers. Started well, but it eventually got some word into its context that is on the banlist. I found that if you tell it to fire off clean Fable subagents and you instruct it to check the Claude Code billing data to check for downgrades, you can get most high-sensitivity spec/review tasks done with Fable. Most. I figure that once GPT 5.6 comes out, Anthropic will become interested in making the safety gate non-destructive.
- parsimo2010 3mo agoAdditionally, I thought the threat being modeled for “biology” was stuff like bioterroism- how to make anthrax, how to distribute a toxin, etc. I don’t feel like calculating results for a trial is really in the threat model unless we think a terrorist is out there testing the efficacy of their anthrax before using it in an attack.
- ACCount37 3mo agoThe classifier is about as refined as a brick to the face. You can ask it elementary school grade biology trivia, or obscure facts about recently documented insect species, and both will downgrade to Opus 4.8 straight away. And Opus itself was already bad with biotech questions. The fact that they somehow made it WORSE for Fable is mindboggling.
- Certhas 3mo agoMy experience, too. I work on nothing in any way related to cybersecurity or biology. I asked it a few purely mathematical questions, it refused immediately. Before the export embargo I did get it to look at some hairy problems and the output was genuinely useful...
- cma 3mo agoWere your question in math areas related to ML? They also restrict model development and research pretty heavily.
- ElFitz 3mo agoGetting it to work on anything involving ai coding agents is a pita. Ironically. Even if it’s just at the application level.
- Certhas 3mo agoNo. Robust control theory was one case. Dynamical systems another.
- areoform 3mo agoWell... now you know one direction Anthropic is looking towards for the future research.
- SubiculumCode 3mo agoI absolutely have been unable to use Fable for any neuroimaging work. Its fine. The other models are good enough, honestly...and while I AM annoyed that the filter is so broad, I also understand it, as I do believe that models can become dangerous as WMDs, eventually. Still, it is completely useless for me. The only question I had was being flagged for other reasons, so I asked it a mechanical engineering question, and it was just fine with that.
- cubefox 3mo agoMy guess is the classifier guardrails were made significantly stricter to convince the US government to reverse the ban.
- ACCount37 3mo agoNah, the classifier was utterly asinine ON release. I'm not sure they could have made it worse if they tried.
- deleted 3mo ago[deleted]
- smcleod 3mo agoI've been working with it heavily since its first release. I use it for software architects, complex debugging and some development and I have not had it refuse or downgrade even once.
- thierrydamiba 3mo agoHave they ever talked about what goes into the classifier? I wonder how much your past chats impact it. For example if it knows you do X at Y company is it more or less strict?
- smcleod 3mo agoYes their docs cover how to properly prompt it to ensure you don't get hit by the classifier, you can distil those down to a skill to help build prompts for the more challenging situations, e.g. https://github.com/sammcj/agentic-coding/blob/main/Skills/prompt-fable/SKILL.md https://github.com/sammcj/agentic-coding/blob/main/Skills/pr...
- robot_jesus 3mo agoSame. I really am curious if either 1) I am using it in genuinely different ways or 2) these people are being willfully disingenuous.
- smcleod 3mo agoI suspect there's some cargo culting, and some folks that are generally more likely to table flip than understand things that challenge their workflow.
- username44 3mo agoDaily use here, about 2.5 weekly 20x limits, never got flagged for code topics including finding memory safety vulnerabilities in my C++ project, but just got flagged for the first time for biology-related topics because I asked it to implement crop genetics and cross breeding into my game. Was able to bypass it by having opus reword the prompt (gene -> trait, cross breed -> trait mixing), and, critically, insisted that it not use any biology related words in its thinking or responses.
- bb88 3mo agoI've been working on a SDN software for mikrotik routers (and wireguard, etc) and Fable dies when working with any kind of wireline protocol or potentially implementing any authentication mechanism. It's too the point where I just stopped using it. If you do generic stuff, it's fine. But the second it tries to start debugging protocols (which may include auth) that's where it begins to fail.
- BikiniPrince 3mo agoI can’t even use it to fix the bugs Opus introduced. I’ve considered ripping out auth until fable is behind the paywall. I’ve been very careful in my queries and broken everything down to careful segments. Even the memory can get security verbs poisoning further requests.
- mattjoyce 3mo agoThe post can be distilled down to a simple statement, but part of the writing is for the author to express themselves and tell a story. I thought it was an interesting read.
- visarga 3mo agoI got downgraded to Opus for asking "What is a cell?" that's all, single message, instant downgrade.
- bbor 3mo agoThat's extremely related to biology.
- saberience 3mo agoI mean, I literally asked about the effects of nicotine withdrawal on the body after quitting smoking and the model got downgraded to Opus... So yeah, if I can't ask about nicotine withdrawal, then I think almost anything biology related is going to get downgraded...