4 ms·

A bit surprised by the snarky comments here -- I also want Claude to work reliably but very few (no?) companies have ever seen this level of rapid growth. We're

by tao_oat 6mo ago

A bit surprised by the snarky comments here -- I also want Claude to work reliably but very few (no?) companies have ever seen this level of rapid growth. We're going to go through a long fail-whale-style period and I can imagine very, very few companies that could avoid that.

rvz 6mo ago
How can Claude work reliably if Claude keeps going on vacation for several hours? Maybe it is recovering from the weekend a few days ago, but wanted to take an extra day off like it did on Monday, hence the "outage".
- ben_w 6mo ago
  > How can Claude work reliably if Claude keeps going on vacation for several hours? Not that I wish to anthropomorphise it in this answer, but businesses have managed just fine when humans do this for "lunch breaks" and "going home for the evening to sleep". (And even mandatory meetings which should have been emails).
  - sassymuffinz 6mo ago
    Thing is, if Dave the programmer goes on vacation or calls in sick for the day, hopefully you have a larger team to fall back on and your business doesn't grind to a halt. No one is apparently noticing that if they build their entire business model around AI being a certain price and availability they're essentially building one giant point of failure into their productivity. What if the price shoots up 10x or Claude goes down for a day, or what if he's occasionally drunk (hallucinating). Reliability is sometimes a more important facet of business than ultra speed and productivity.
    - ben_w 6mo ago
      Aye, correlated failure is not something to be overlooked. Mistaking correlated risk for uncorrelated risk was a big part of the global financial crisis. There are fallback mechanisms when the risk is per model provider (as in, "What if the price shoots up 10x or Claude goes down for a day" is a manageable concern), but I'd be more worried about the way all models regardless of provider have similar failure modes, i.e. that some tasks fail in similar ways for all models. In some ways, LLMs are collectively like Star Trek's Borg: you've met one, you've met all of them.
- deleted 6mo ago
  [deleted]
deleted 6mo ago
[deleted]
yakattak 6mo ago
They’re asking for $100+/mo for the plans that are actually usable at scale. If I’m paying that much I have very high expectations. There’s also the fact that they’re known for dogfooding heavily, I imagine that contributes to it a lot.
- forrestthewoods 6mo ago
  > They’re asking for $100+/mo for the plans that are actually usable at scale. If I’m paying that much I have very high expectations. They’re losing money on you at that price point. Or more precisely you’re paying for it by giving them training data.
  - omega3 6mo ago
    I'm not convinced, Kimi 2.5, GLM 5.1, Minimax M2.7 are all fraction of the price and still make money on inference.
- baq 6mo ago
  they can adjust prices or you can adjust your expectations. ...and let's be realistic, it'll be both.
- stingraycharles 6mo ago
  > They’re asking for $100+/mo for the plans that are actually usable at scale. If I’m paying that much I have very high expectations. If you think $100 is that much and get very high expectations from it, you're not the target customer. You're a loss leader to Anthropic, and the fact that you don't see that / still have high expectations means your expectations are unrealistic.
  - dijit 6mo ago
    $100/m for SaaS is very steep. For an entire productivity suite including mail, meetings and terabytes of backed up redundant storage with nearly no bandwidth limitations it's like $35/m for even the most expensive option.
trvz 6mo ago
They always have the option to stop accepting new customers when their infrastructure is peaked out instead of lowering quality for everyone.
- stingraycharles 6mo ago
  You don't know whether this is due to infrastructure capacity or has other reasons (organizational). Also, "let's stop accepting new customers" is probably not a realistic choice for a hundred reasons.
- alkonaut 6mo ago
  That would mean in a way accepting that they are suddenly a service company with the aim to create revenue by selling services to customers for money.
- arcfour 6mo ago
  You can't stop accepting new customers unless you're fine with killing your potential future customer base. That's a ridiculous suggestion. Either your current customers or your potential future customers are going to be unhappy so long as compute resources are finite. Take your pick.
  - ed_mercer 6mo ago
    > That's a ridiculous suggestion. Is it though? Claude's reliability is now at an all-time low of 98.7%. It's not a stretch to think that large companies will have second doubts about about adopting claude for their production environments.
  - baq 6mo ago
    > You can't stop accepting new customers unless you're fine with killing your potential future customer base. That's a ridiculous suggestion. what? they already have, they aren't releasing mythos except to a limited pre-approved customer base who is practically begging them to take their money. they can do that for lower tier models and at this point they should.
logicchains 6mo ago
>I also want Claude to work reliably but very few (no?) companies have ever seen this level of rapid growth. We're going to go through a long fail-whale-style period and I can imagine very, very few companies that could avoid that. Their main competitor OpenAI has much better uptime and more generous usage limits.
- code51 6mo ago
  Exactly this. OpenAI is running huge workloads silently, without anybody patting their back.
techpression 6mo ago
Doesn’t matter, they are not handling it correctly, but instead keep selling while far over capacity. They should not accept more users until they can supply the service. We solved this thousands of years ago, it’s called waiting in line. And yes, it’s not common to see, but that doesn’t excuse not doing it.
al_borland 6mo ago
When the narrative around AI is that people should rely on it all the time, people will be judged by your token use (it better be high), the AI is smarter than everyone and will take all the jobs, the AI is the best programmer, and more… When things fail repeatedly, it highlights that the emperor has no clothes. If it’s as good as they say, why can’t it figure out how to not go down every day? How can people rely on it for their job if it goes down everyday? Maybe they shouldn’t rely on it. If it’s supposed to be such a good engineer, why should it have the same scaling issues as Twitter did 20 years ago with 20 years of lessons learned and 20 years of development for more modern and scalable infrastructures? Shouldn’t it know all the tricks to scale and have redundancy to keep availability high? Does it not know the demands? When expectations are out of line with reality, there will be snark when things fail. Those expectations have been force fed to us by these AI companies for years now, so I don’t have much sympathy or patience to offer them. They created these expectations of their platforms and if they can’t live up to them, then maybe it’s time for recalibrate the public image of what AI really is and what it can do… and what its limitations are.
- runeblaze 6mo ago
  I mean I used to work on model reliability with my little PhD degree and the models i manage go down all the time. Some profs have a team of PhDs and things go to shit all the time. I don’t know why we expect $FRONTIER_LLM to do better
  - al_borland 6mo ago
    We accept human errors, limitations, and failures. We can empathize with team of humans doing the best they can, and we know any failure is a chance for them to learn and grow. The sales pitch of AI is that it’s better than humans and has no real limits; it will make us all obsolete. This framing they created means I expect it not to make errors, not to have limits, and not to fail. I expect it to be able to learn and adapt at the speed of light and solve complex problems beyond what a PhD could do. This is what we’ve been told with the narratives around future jobs, AI performance on PhD level tests, how coding is a solved problem, and pictures painted of what a future with AI will look like. While we may know this isn’t true, this is what they are selling, and that’s the standard I’m going to hold them to. I don’t blame the customer for being upset the snake oil didn’t live up to its promises, I blame the snake oil salesman. We have every right to be upset with the snake oil salesman and ridicule him when his product doesn’t work. Maybe we don’t need better more reliable snake oil, maybe we need real medicine. If real medicine don’t exist, its better to be honest than to mislead people and say it does. This isn’t to say AI is completely useless, but it’s not what’s being sold. The downtime just proves that, unless they aren’t using their own product. If that’s the case, why not?
pllbnk 6mo ago
They have this new Mythos model. I am sure it can fix all the bugs and reliability issues since it's nearly AGI. /s
coldtea 6mo ago
>I also want Claude to work reliably but very few (no?) companies have ever seen this level of rapid growth. You do understand however that aside from the growth/maturity path, this is also a path to enshittification and skinning their users, which might come even faster to LMMs than say Google , because the latter managed to have hundres of billions in investments in record time to recoup and IPOs on sight.

by tao_oat 6mo ago

rvz 6mo ago

How can Claude work reliably if Claude keeps going on vacation for several hours? Maybe it is recovering from the weekend a few days ago, but wanted to take an extra day off like it did on Monday, hence the "outage".

ben_w 6mo ago
> How can Claude work reliably if Claude keeps going on vacation for several hours? Not that I wish to anthropomorphise it in this answer, but businesses have managed just fine when humans do this for "lunch breaks" and "going home for the evening to sleep". (And even mandatory meetings which should have been emails).
- sassymuffinz 6mo ago
  Thing is, if Dave the programmer goes on vacation or calls in sick for the day, hopefully you have a larger team to fall back on and your business doesn't grind to a halt. No one is apparently noticing that if they build their entire business model around AI being a certain price and availability they're essentially building one giant point of failure into their productivity. What if the price shoots up 10x or Claude goes down for a day, or what if he's occasionally drunk (hallucinating). Reliability is sometimes a more important facet of business than ultra speed and productivity.
  - ben_w 6mo ago
    Aye, correlated failure is not something to be overlooked. Mistaking correlated risk for uncorrelated risk was a big part of the global financial crisis. There are fallback mechanisms when the risk is per model provider (as in, "What if the price shoots up 10x or Claude goes down for a day" is a manageable concern), but I'd be more worried about the way all models regardless of provider have similar failure modes, i.e. that some tasks fail in similar ways for all models. In some ways, LLMs are collectively like Star Trek's Borg: you've met one, you've met all of them.
deleted 6mo ago
[deleted]

deleted 6mo ago

[deleted]

yakattak 6mo ago

They’re asking for $100+/mo for the plans that are actually usable at scale. If I’m paying that much I have very high expectations. There’s also the fact that they’re known for dogfooding heavily, I imagine that contributes to it a lot.

forrestthewoods 6mo ago
> They’re asking for $100+/mo for the plans that are actually usable at scale. If I’m paying that much I have very high expectations. They’re losing money on you at that price point. Or more precisely you’re paying for it by giving them training data.
- omega3 6mo ago
  I'm not convinced, Kimi 2.5, GLM 5.1, Minimax M2.7 are all fraction of the price and still make money on inference.
baq 6mo ago
they can adjust prices or you can adjust your expectations. ...and let's be realistic, it'll be both.
stingraycharles 6mo ago
> They’re asking for $100+/mo for the plans that are actually usable at scale. If I’m paying that much I have very high expectations. If you think $100 is that much and get very high expectations from it, you're not the target customer. You're a loss leader to Anthropic, and the fact that you don't see that / still have high expectations means your expectations are unrealistic.
- dijit 6mo ago
  $100/m for SaaS is very steep. For an entire productivity suite including mail, meetings and terabytes of backed up redundant storage with nearly no bandwidth limitations it's like $35/m for even the most expensive option.

trvz 6mo ago

They always have the option to stop accepting new customers when their infrastructure is peaked out instead of lowering quality for everyone.

stingraycharles 6mo ago
You don't know whether this is due to infrastructure capacity or has other reasons (organizational). Also, "let's stop accepting new customers" is probably not a realistic choice for a hundred reasons.
alkonaut 6mo ago
That would mean in a way accepting that they are suddenly a service company with the aim to create revenue by selling services to customers for money.
arcfour 6mo ago
You can't stop accepting new customers unless you're fine with killing your potential future customer base. That's a ridiculous suggestion. Either your current customers or your potential future customers are going to be unhappy so long as compute resources are finite. Take your pick.
- ed_mercer 6mo ago
  > That's a ridiculous suggestion. Is it though? Claude's reliability is now at an all-time low of 98.7%. It's not a stretch to think that large companies will have second doubts about about adopting claude for their production environments.
- baq 6mo ago
  > You can't stop accepting new customers unless you're fine with killing your potential future customer base. That's a ridiculous suggestion. what? they already have, they aren't releasing mythos except to a limited pre-approved customer base who is practically begging them to take their money. they can do that for lower tier models and at this point they should.

logicchains 6mo ago

>I also want Claude to work reliably but very few (no?) companies have ever seen this level of rapid growth. We're going to go through a long fail-whale-style period and I can imagine very, very few companies that could avoid that. Their main competitor OpenAI has much better uptime and more generous usage limits.

code51 6mo ago
Exactly this. OpenAI is running huge workloads silently, without anybody patting their back.

techpression 6mo ago

Doesn’t matter, they are not handling it correctly, but instead keep selling while far over capacity. They should not accept more users until they can supply the service. We solved this thousands of years ago, it’s called waiting in line. And yes, it’s not common to see, but that doesn’t excuse not doing it.

al_borland 6mo ago

When the narrative around AI is that people should rely on it all the time, people will be judged by your token use (it better be high), the AI is smarter than everyone and will take all the jobs, the AI is the best programmer, and more… When things fail repeatedly, it highlights that the emperor has no clothes. If it’s as good as they say, why can’t it figure out how to not go down every day? How can people rely on it for their job if it goes down everyday? Maybe they shouldn’t rely on it. If it’s supposed to be such a good engineer, why should it have the same scaling issues as Twitter did 20 years ago with 20 years of lessons learned and 20 years of development for more modern and scalable infrastructures? Shouldn’t it know all the tricks to scale and have redundancy to keep availability high? Does it not know the demands? When expectations are out of line with reality, there will be snark when things fail. Those expectations have been force fed to us by these AI companies for years now, so I don’t have much sympathy or patience to offer them. They created these expectations of their platforms and if they can’t live up to them, then maybe it’s time for recalibrate the public image of what AI really is and what it can do… and what its limitations are.

runeblaze 6mo ago
I mean I used to work on model reliability with my little PhD degree and the models i manage go down all the time. Some profs have a team of PhDs and things go to shit all the time. I don’t know why we expect $FRONTIER_LLM to do better
- al_borland 6mo ago
  We accept human errors, limitations, and failures. We can empathize with team of humans doing the best they can, and we know any failure is a chance for them to learn and grow. The sales pitch of AI is that it’s better than humans and has no real limits; it will make us all obsolete. This framing they created means I expect it not to make errors, not to have limits, and not to fail. I expect it to be able to learn and adapt at the speed of light and solve complex problems beyond what a PhD could do. This is what we’ve been told with the narratives around future jobs, AI performance on PhD level tests, how coding is a solved problem, and pictures painted of what a future with AI will look like. While we may know this isn’t true, this is what they are selling, and that’s the standard I’m going to hold them to. I don’t blame the customer for being upset the snake oil didn’t live up to its promises, I blame the snake oil salesman. We have every right to be upset with the snake oil salesman and ridicule him when his product doesn’t work. Maybe we don’t need better more reliable snake oil, maybe we need real medicine. If real medicine don’t exist, its better to be honest than to mislead people and say it does. This isn’t to say AI is completely useless, but it’s not what’s being sold. The downtime just proves that, unless they aren’t using their own product. If that’s the case, why not?

pllbnk 6mo ago

They have this new Mythos model. I am sure it can fix all the bugs and reliability issues since it's nearly AGI. /s

coldtea 6mo ago

>I also want Claude to work reliably but very few (no?) companies have ever seen this level of rapid growth. You do understand however that aside from the growth/maturity path, this is also a path to enshittification and skinning their users, which might come even faster to LMMs than say Google , because the latter managed to have hundres of billions in investments in record time to recoup and IPOs on sight.