6 ms·

News just broke in this Wired story: "Anthropic Walks Back Policy That Could Have ‘Sabotaged’ AI Researchers Using Claude" https://www.wired.com/story/anthropic

• hedgehog 5mo ago
  The "tradeoff" warning implies they stand by their thinking and don't think there was anything qualitatively wrong with it which, if nothing else, is helpful so potential customers can know how they think. I think the core lesson is if you want reliable infrastructure to build into an application you should use a different provider. (edit: I'm not specifically an Anthropic hater, but having just spent some time adding complexity to an app to deal with the existing refusal behavior in Sonnet... I understand why they might want this in an end user chatbot but for an API it's really not acceptable)
  • brookst 5mo ago
    Is it not a trade off? I think they made the wrong choice, but it seems reductive so say there was no choice at all and should never have been consideration of trade offs of silent versus not. Even wide open, uncensored models are often the product of a deliberate choice. I have a hard time faulting people for intentionality (even when they get it wrong).
    • hedgehog 5mo ago
      They have a lot of choices, why would that specifically be a tradeoff? It's common for people to construct a tradeoff under which their preferred action is the more virtuous option, and thus they can be "the good guys", but that doesn't mean their framing makes any sense at all. Silently downgrading requests to a weaker model and billing the customer at full price, then framing the debate as how much (not if) this behavior is correct, that's an expression of values. People make mistakes all the time, if they thought it was actually wrong they could well have said so and explained what corrective action they've taken. One of the most famous examples of doing this right was the Pentium FDIV bug. Intel stood behind the product by recalling the affected units at great expense, and that (rightly) earned a lot of trust for decades.
      • brookst 5mo ago
        You seem to be focused on the decision making, but I still don’t understand how it’s not a tradeoff. All binary decisions (silent or not) are tradeoffs because there are upsides and downsides to each, the question is which is better on the whole. If I’m deciding whether or not to eat ice cream, there are trade offs involved because I can’t simultaneously have it both ways. And Anthropic did apologize, explain reasoning, and what they learned. They got it wrong; they picked the wrong trade offs and got a net worse decision than they should have. I’m with you on everything except this idea that it was an obvious decision with no upsides to silent and no downsides to loud.
        • hedgehog 5mo ago
          Sure, ad absurdum anything can be called a tradeoff, but when you describe something as a tradeoff in communication you're also making an argument that there are a range of reasonable answers (otherwise why would you bring it up). Whether or not to eat ice cream, it's very reasonable to consider lots of situations where both, or some compromise (a small ice cream), are fine choices. If I came to you on the street while you were holding your ice cream, told you I was going to take your ice cream and eat it, and after your protest changed my mind and said that was the "wrong balance" in how much I took, you could very fairly conclude that I was the sort of person who thinks it's ok to steal people's ice cream (maybe just not today). Now maybe there was some great miscommunication, and I thought you were done with it, but then I wouldn't say I made the wrong tradeoff, I would just tell you I misunderstood the situation and hope you'd invite me to ice cream another time. Anthropic in this case is saying they think there's a balance in how much of the service customers pay for that they should deliver, as clarified in their Wired statement. That's a choice.
          • hedgehog 5mo ago
            (they also reset usage for many accounts after the Mythos/Fable rollback, which is great)
• h6d_100c 5mo ago
  To late. I canceled my Max subscription. The idea they would even do this is so destroyed any remaining trust. Why would I pay them 1000s of dollars in extra usage per month for something they could still be doing behind the scenes? Any errors previously chalked up to thinking effort or other backend changes? Maybe it was intentional prompt injection the entire time.
  • z3ratul163071 5mo ago
    that you still had max after all their deceptions is amazing
    • h6d_100c 5mo ago
      Yeah; not my smartest decision given their ongoing “issues”
  • trhway 5mo ago
    You've been Stuxnet-ed by Anthropic :)
  • musebox35 5mo ago
    I work on open source text-to-image finetuning of open source models like zimage/flux2 klein 4b and inference time latency optimization. The moment I read the silent treatment, I went ahead and cancelled my subscription too since I would never know whether the models they launch will silently corrupt my output. This is totally unacceptable. There is a big difference between silent / flagged if you are doing ml research but not at frontier capability. This goes on to show that - All that interpretability / safety research they are doing can also be weaponized against customers (steering vectors, intent classification, ...) in the name of safety from malicious actors. - If they deem profitable, they might nerf to original model and its training data for ml research at a bulk scale and then they won't even have to announce it so long as the overall benchmark score stays high enough. As the IPOs get closer, they can do whatever they want to assure the investors that they have a moat that can not be crossed over by their own products. Considering this affects all ML researchers/students at universities, smaller scale research labs, this is just "cutting the branch you are sitting on".
• rafram 5mo ago
  The mitigations against distillation are separate, and not what the OP is about at all.
• bostik 5mo ago
  They need to walk back a lot more. Unilaterally revoking zero-data retention, even for enterprise contracts that explicitly require that? Nope. Fable is utterly unusable for any kind of security work. I tripped the safeguards yesterday - using Fable to dig into a complex (& annoying) security bug that has so far resisted both human and Opus 4.8 level investigation. "Sorry Dave, I can't let you do that." For the time being we are requesting Anthropic disable Fable for our enterprise and turn ZDR back on. The two may be interlinked so that one will always get neither or both. ZDR is a contractual obligation. Fable in its current form is useless. Might as well flip the old behaviour on and avoid burning money for no reason while this mess is being sorted out.
  • rurban 5mo ago
    Not just security work. Normal bug finding was impossible, because the model suddenly called triaging and verifying a possible fix a cyber security threat.
    • insanitybit 5mo ago
      I was just building a library to use file capabilities (ie: open_at) and it refused. This thing won't even help you write safe software.
      • rurban 5mo ago
        Whow, same for me. Insane context bugs in flake 5
  • dmurray 5mo ago
    The announcement I saw was that your enterprise would have to turn off ZDR to get Fable, not that users could accidentally opt out of ZDR by selecting the wrong model. Unilaterally disabling ZDR seems like a step too far in the enterprise market, even for a company trying to figure out what its users will let it get away with.
    • bostik 5mo ago
• pseudosavant 5mo ago
  They are still downgrading. They just aren't doing it silently. I don't know how big of a win that is? They still trained on everyone else's data without license or attribution but want to prevent someone else from doing the same thing to them. Some pretty audacious hypocrisy from Anthropic this week.
  • KeplerBoy 5mo ago
    Imo that's a big win. The LLM just gaslighting you into suboptimal approaches was insane.
    • pseudosavant 5mo ago
      I guess, but yesterday Anthropic had their version of Google removing the "Don't be evil" from their motto. They destroyed a metric ton of goodwill they'll never regain.
      • gr_norm 5mo ago
        Yeah, they showed their true colors there. This, compounded with the fact that they're the only frontier lab with no open models, tells you all you need to know. Tired of the insanely patronizing (+ conveniently and overwhelmingly self-serving) attitude out of them. My goal is to own my computing and be able to choose what to do with it.
      • monegator 5mo ago
        And just a few days ago i was being called out because i considered anthropic "evil" I mean, did nobody ever get the vibes, never see a pattern emerging? (well they don't or they wouldn't be so amazed by pattern recognition machines on steroids)
  • musebox35 5mo ago
    It is much more reasonable to do it in a visible / flagged way. At least you have visibility over the quality of service you get as a customer. Silent treatment is a breach of trust, what you buy changes depending on the context based on the goals of the producer. It is like your computer silently blocking ads from competitors at the hardware level, which is crazy. I think they erred on the wrong side of things due to IPO pressure. At least there is competition from multiple companies. Still it is best to have personal benchmarks for the domain you are working on to have a real evaluation of the value you get for the money/time you spent on these products. Without trust, that might be the only way forward to keep the companies honest. This happens eventually in all sectors, a good magazine/website that does independent product evaluation is priceless. Sadly, the new ad-driven internet decimated those that worked great in the 90/00s. Still there are independent blogs that does some evaluation and that is better than nothing.
• Aperocky 5mo ago
  I don't think it's the widespread condemnation, I think it's some high paying customer and potential investor telling them to stick it.
• nl 5mo ago
  This is different to the cyber limitations though. To be precise - it makes the "won't work on frontier machine learning" refusal the same as the "won't work on cyber security" refusal (instead of the way it previously would work on frontier machine learning problems but give sub-optimal answers without informing the user)
  • dannyw 5mo ago
    Some anecdotal social reports seem to suggest it wasn’t just giving suboptimal answers, but rather mucking around and sabotaging your codebase and training (like editing hyperparameters in project files despite not being requested). Of course, it’s impossible to know if that was deliberate sabotage, or model misbehaviour. Which is exactly the problem. That may be considered malware / a criminal act tbh.
• AussieWog93 5mo ago
  Non-paywalled: https://archive.md/yxYhU https://archive.md/yxYhU
• selicos 5mo ago
  If any work is blocked/etc, refund all credits from that session/last X minutes. Minimum.
• deleted 5mo ago
  [deleted]
• Grimblewald 5mo ago
  Corporate America never backs down. It simply rallies and tries again later until people are too fatigued to care. The only solution is to abandon ship, which I am doing. MS walked back in OS ads the first few times, but ultimately we still ended up on the exact trajectory everyone was outraged at. OpenAI still ended up on its path to closed AI despite initial walk backs. The story repeats itself over and over again, so, once the bad behavior starts, you leave. Their apologies are as hollow as their moral posturing.
  • mettamage 5mo ago
    I hope this has some answers [1]. It’s on the front page right now, but your frustration clearly seems to have some implicit answers that [1] is trying to answer. [1] https://news.ycombinator.com/item?id=48477135 https://news.ycombinator.com/item?id=48477135
  • n6242 5mo ago
    Same with VISA/Mastercard deciding what we can/cannot buy. The only solution is to stop using their credit cards at all.
    • Cider9986 5mo ago
      Yes, Monero is a lot better than credit cards for privacy and freedom. I hope to see it accepted more.
      • j16sdiz 5mo ago
        Its not only the corporate america. Those crypto scammers do the same simply rallies and tries again later until people are too fatigued to care.
      • fatata123 5mo ago
        [dead]
    • abustamam 5mo ago
      Easy to say, but every bank I've had the (dis)pleasure of doing business with only ever issued a Visa or Mastercard so it's not really feasible to just "stop using them"
• consumer451 5mo ago
  The other major thing is almost as bad, and actually maybe even worse for trust of AI features in b2b apps: > Anthropic requires 30 day data retention for Fable and Mythos https://news.ycombinator.com/item?id=48464258 https://news.ycombinator.com/item?id=48464258 I used to be able to tell my enterprise customers something simple, that I really believe: "We use Anthropic models via Bedrock/Azure, therefore we are guaranteed that your data will not be used for training models." That simple blanket statement is no longer true. Also, most normal people/customers only read headlines, and this is a huge story. From my point of view, as someone deploying LLMs in my apps, trust comms with my clients just got set back two years.
  • Spooky23 5mo ago
    I’m very cautious with using these tools with certain clients, as I’m often contractually obligated to do things that my downstream supplier can rug pull at any time. You should never use any of the frontier models with operational workloads manipulating or interpreting customer data.
    • consumer451 5mo ago
      I appreciate the reply. Could you please help me understand what you mean by "You should never use any of the frontier models?" Does that mean the latest model, hosted by the lab, Bedrock, or Azure Foundry? Or, do you mean only use self-hosted models, or what did you mean by that? I would really love to learn what others are doing. I felt like my trust story was solid enough, prior to all this. I have been deploying and integrating Claude and Sonnet (latest 4.x-2), on Azure, as my client base has MS contract trust, for better or worse, and Anthropic models have been making my products amazing. To see my other thoughts on this cluster f, please see: https://news.ycombinator.com/item?id=48488781 https://news.ycombinator.com/item?id=48488781
      • Spooky23 5mo ago
        Sure. It's really about informed consent and acceptance of risk. I'm very conservative about that due to my background and business. Say you have some flow that is processing/handling regulated, sensitive or other customer data with the LLM as part of an operational process. An example that I'm thinking of is for a customer who wants to more efficiently resolve or route IT incidents to the right place. The incident data may contain user-provided data has strings attached from a compliance perspective. If you're using a third party API, your T&Cs are the only protection that you have. Microsoft/Google/Amazon are pretty decent by default. When I worked for the government, we had the leverage to extract much favorable terms from the big vendors like Google, Amazon, Microsoft as well. With Anthropic, and OpenAI, they are in the move fast and break things universe, you need to be bringing alot of money to the table to get terms changes, and you can easily stumble into a situation where they are retaining data in a manner that your customer will not like. So unless the customer is informed and accepting of that risk, proceed with caution. I've had some success using self-hosted inference for these scenarios. For development of software, totally different story -- it's your IP and you make the risk call.