6 ms·
The spend at my organization has reached beyond the $200,000 per month level on Anthropic's enterprise tier. The amount of outages we have had over these past f
by SimianSci 5mo ago
The spend at my organization has reached beyond the $200,000 per month level on Anthropic's enterprise tier.
The amount of outages we have had over these past few months are astounding and coupled with their horrendous support it has our executive team furious.
its alot of money to be spending for a single 9 of reliablility.
- deadbabe 5mo agoWe are spending the equivalent of 32 monthly software engineer salaries on Claude per month.
- cactusplant7374 5mo agoIs it worth it?
- lolive 5mo agoHe was fired before answering. [but as his manager I can tell you:] YES !!!!
- deadbabe 5mo agoNo, we can literally buy our own hardware for what we spend in a month and host our own local LLMs for company usage.
- nomel 5mo ago> and host our own local LLMs for company usage. What local alternative could replace your Anthropic use? I have found none. I don't think many have, which is why most of us pay Anthropic, rather than using one of the numerous, far cheaper, cloud services that host "local" class models. Most of us are paying for access to proprietary SOTA models, rather than hosting.
- deleted 5mo ago[deleted]
- SimianSci 5mo agoOur expense is roughly around 12.3 software developers when you break it down across all people related expenses. But we've spent alot of time and energy prior to this focusing on our ability to measure our software development output across multiple teams. The delivery improvements are not evenly applied across all teams, but the increases that we have seen suggest a better ROI than if we had hired 12 developers.
- protonbob 5mo agoI guess if you think about your teammates as purely inputs and outputs and not people that can improve and contribute in the workplace in other ways.
- SimianSci 5mo agoRespectfully, After a certain level of compensation, you are indeed judged purely off of input and output. Workplace improvement does not justify your salary. You will also find that many problems in the harder sciences do not get easier by throwing more bodies at them. Comments like these remind me that some project managers think they'd be able to delivery a baby in 1 month if they simply had 9 women.
- oarsinsync 5mo ago> Respectfully, After a certain level of compensation, you are indeed judged purely off of input and output. Workplace improvement does not justify your salary. I'd have to disagree. There's a narrow band in the middle where that's true, but once you exceed that, your personal inputs and outputs matter less and less, and the contributions you make to the overall workplace, and how well you enable those around you, make a larger part of why you're compensated. Even as an IC, the more you're able to mentor and elevate the people around you, the more your compensation will grow (if you're in the right place, and thus already at the right earnings bracket)
- paganel 5mo ago> you are indeed judged purely off of input and output That's not how successful (software, in this case) teams are made.
- jonny_eh 5mo agoInfo like this is useless without context like, how much revenue does the company earn? How many engineers do they employ? etc.
- cactusplant7374 5mo agoImagine how much money they would save if they switched to Codex.
- subscribed 5mo agoNot everyone can (due to the corporate compliance requirements, eg the ease of making the LLM not to train on anything). Besides, codex wasn't always the answer.
- noosphr 5mo agoA single nine so far. If github is any guide thing will get worse.
- smt88 5mo agoWhy would Github be a guide? It's also terrible, but it's a radically different stack from an unrelated company
- StableAlkyne 5mo agoThat, and even before AI, MS was having trouble with GH reliability
- shimman 5mo agoGitHub, along with MSFT in general, have massive copilot mandates where workers are being shamed into using slop tools to fix serious on-going issues. GitHub seems wholly incapable of resolving their issues: money isn't a problem, talent isn't a problem, but business leadership is definitely a major problem. Look at how other companies are suffering massive outages due to LLMs too like AWS and Cloudflare. Two companies that use to be the best in the industry at uptime but have suddenly faltered quite quickly. Companies that have even worse standards will quickly realize how problematic these tools are. Hopefully before a recession because this industry seems to be allergic to profitable businesses and leaders that have been around since ZIRP have shown zero intelligence in navigating these times.
- deleted 5mo ago[deleted]
- kentonv 5mo agoNone of the three major Cloudflare outages in the past six months had anything to do with LLMs. They were regular old human mistakes. We did, however, determine that at least one of them (and perhaps all) would have been easily caught by AI code reviewers, had AI code reviewers been in use. So now we mandate that. And honestly, I love it, the AI reviewer spots all sorts of things that humans would probably miss. (We also fixed a number of problems around configuration that would roll out globally too fast, leaving no time to notice errors and stop a bad rollout, as well as cases where services being down actually made it hard to revert the change... should be in a much better place now. But again, none of that had to do with LLMs.)
- bayarearefugee 5mo ago> has our executive team furious And yet they will continue to spend wheelbarrows full of money with Anthropic because they want so badly to reach the point where they can fire you.
- SimianSci 5mo agoI think there is alot of baseless fury behind your words, but my regular interactions with my leadership dont lead me to think they have the end goal of replacing labor. We're blessed to have leadership with technical backgrounds, so the tools are regarded more as significant intelligence enhancers of already exceptionally smart engineers, rather than replacements. Doesnt seem to us to be wheelbarrows of money, when you consider the average AWS/Azure bill.
- protonbob 5mo agoNot ever hiring juniors and eventually mids is just replacing labor with extra steps.
- subscribed 5mo agoI think the message you responded to already refuted your point of view.
- SimianSci 5mo agoThrowing bodies at a problem doesn't always scale. There are many difficult problems that do not get easier by throwing more juniors or mid level engineers at them.
- throwaway277432 5mo agoHaving just worked my behind off for the last months to deliver on an impossible deadline, successfully: more bodies definitely would have helped. Even just to keep the fluff off my back and to allow me to fully concentrate on what's important. The situation will repeat itself in 6 months and I'm not going to do that again. Hiring now would fix that.
- walrus01 5mo agoFive nines? No, nine fives
- Someone1234 5mo agoObviously there is only so much you can say; but is that $200K due to the raw number of seats you have, or are you burning through a lot on raw API usage? I guess I'm trying to understand, large business, or large usage.
- SimianSci 5mo agowe are in the SMB space, the spend is almost entirely usage for us at this point, rather than seat cost. For context, we are a software firm focused on difficult engineering problems, but I cant divulge much else.
- 2ndorderthought 5mo agoHave you guys considered running your own local models? 200k a month is a ton of money and puts all your eggs in one basket. Or is it easier to just be able to run away from it all if you are done with it or something changes?
- SimianSci 5mo agoI led the team that did the math and analysis for determining our direction in selecting Anthropic. We initially assumed this was where we would end up, but after some investment exploring our options we found it not worth the trouble. Local models sound great until you realize you dont get alot of the features that we implicitly expect from hosted models. Many things would require additional investment into the operations and setup to get to a comparable system. We ended up wanting things that would require us to roll our own memory system, harnesses for the model, compliance needs, and security. It was possible for us to invest in this, but it would require additional investment in hiring or training to get us to a state comparable to the hosted options. Eventually, I had to recommend against the project as it was more likely to be an investment in the leading team's resume, than an actual investment into our organization.
- joefourier 5mo ago> Local models sound great until you realize you dont get alot of the features that we implicitly expect from hosted models. Many things would require additional investment into the operations and setup to get to a comparable system. We ended up wanting things that would require us to roll our own memory system, harnesses for the model, compliance needs, and security. That's not local models vs hosted models, that's using the enterprise services from Anthropic. Any local LLM inference engine such as VLLM gives you an OpenAI compatible API with the exact same features as a hosted model. I'm not sure what your use case is, but I personally found Anthropic's offerings lacking and inferior to open source or custom-built solutions. I have yet to see any "memory" system that's better than markdown files or search, and harnesses for agentic AIs are dime a dozen.
- boc 5mo agoSeems to be back now (claude code at least)
- wg0 5mo agoSpeaking of developer tooling spend - IDEs are far harder to build such as JetBrain etc and don't think any IDE would be charging this amount to any customer per month. Not sure how much of a productivity gain a 2.5 million per year it is?
- theptip 5mo agoSupply and demand - if you think it’s not worth the price, take your dollars elsewhere. This is the brutal reality; even with the crazy reliability issues, demand is still far outstripping supply at the current price.
- wg0 5mo agoRun Facebook on a single Proxmox box and demand would still outstrip the supply. What yet needs to be seen is if that demand sustains in the long run at that price point or flattens out proving to be super elastic given that there are many other providers that are catching up pretty fast.
- esafak 5mo agoIDEs don't need expensive GPUs to create and serve.
- nubinetwork 5mo ago> single 9 of reliability Out of curiosity, do you actually use it 24/7? The world doesn't collapse every time o365 goes down... (which is also pretty often)
- mgh95 5mo agoif it's judged only by the time it is expected to be in use (work hours), reliability is likely even worse than the 24/7 measure.
- manacit 5mo agoIn my experience the downtime tends to coincide with peak PT timezones. If you're in PT, it's very inconvienent.
- Hamuko 5mo agoYeah, I feel like all of the bad downtimes happen during American business hours. We use GitHub at work in Europe and I don't remember it ever being down or broken between 0700 and 1700 local time.
- anonyfox 5mo agoThat’s statistically just luck then - plenty of outages this year already in Berlin time during work hours - I do remember the forced breaks with colleagues for sure.
- simianparrot 5mo agoJust give them more money, surely it'll get better. /s
- SilverElfin 5mo agoThey must have hired absolutely incompetent leaders on the core software and infrastructure side. Sure their AI research is great but it’s amateur hour. Or just vibe coded slop top to bottom. It seems like every single day people are talking about outages or billing issues or secret changes to how Claude works.
- 33MHz-i486 5mo agotheyre getting high on their own supply, and instead really need to hire some senior engineers
- Shakahs 5mo agoIf you are paying API rates (not using Max subscriptions) there's no reason to use Anthropic's API directly, the same models are hosted by both AWS and Google with better uptime than Anthropic.
- JamesSwift 5mo agoHow do things like prompt caching etc play into that? Would I theoretically have a more stable harness backing my usage? Im seriously over the current claude experience. After seemingly fixing my 4.6 usage by disabling adaptive thinking and moving to max effort, it seems that the release of 4.7 has broken that workflow and Im 99% certain that disabling adaptive thinking does nothing even on 4.6 now. Just egregious errors in 2 days this week after coming back from vacation.
- thepasch 5mo ago> Would I theoretically have a more stable harness backing my usage? If you don’t mind an opinionated harness that asks for a pretty specific workflow, but one that works well, use OpenCode. If you want to spread your wings and feel the sweet kiss of freedom, use Pi.
- JamesSwift 5mo agoIm looking at moving to Pi and I like the minimal nature, but I disagree with a handful of decisions they make. So Id likely need to maintain a fork which is less than ideal.
- wyre 5mo agoWhat decisions is Mario making that you disagree with? My impression is Pi is minimal so any changes can live on top of Pi without needing to maintain a fork? I started developing my own coding agent after using Pi for a couple months, so I’m curious what you don’t like about pi.
- 5mo ago
- mihaaly 5mo agoI wonder if self-hosted models would be a sensible step for your organization.
- Scarbutt 5mo agoIs the $200k just development or are the products being developed require AI?