4 ms·

They later said: https://twitter.com/TheAmolAvasare/status/2046725498592722972 https://twitter.com/TheAmolAvasare/status/204672549859272297... > When we do lan

by simonw 7mo ago

They later said: https://twitter.com/TheAmolAvasare/status/2046725498592722972 https://twitter.com/TheAmolAvasare/status/204672549859272297... > When we do land on something, if it affects existing subscribers you'll get plenty of notice before anything changes. Will hear it from us, not a screenshot on X or Reddit. If you don't want things like this spreading through screenshots of X and Reddit, don't run "tests" like this in the first place! (Also "if it affects existing subscribers" is a cop-out, I need to know the pricing of Claude Code for NEW subscribers if I'm going to adopt it at a company with a growing team, or recommend it to other people, write tutorials etc.)

rsynnott 7mo ago
A screenshot is, however, apparently good enough for _new_ subscribers.
minimaxir 7mo ago
A/B tests only work if the subjects don't realize they are in a A/B test.
- inetknght 7mo ago
  Solution: don't A/B test your users. A/B testing people without their informed consent is immoral, unethical, and should be illegal.
  - vehemenz 7mo ago
    Depends entirely on the stakes and whether personal data is involved
    - inetknght 7mo ago
      > Depends entirely on the stakes and whether personal data is involved Sure. Let me just A/B test whether or not you'll respond positively or negatively to having your news delivered via push notification or delayed by 10 minutes. I'm sure you would appreciate being tested on without your consent, just so that I can make an extra quick buck at your expense. Nothing amoral or unethical about it.
      - pitched 7mo ago
        What do you think about slow rollouts for new features? Like, we think this new push notification system will be loved but let’s ship to only 1% of users in case there’s a horrible unforeseen consequence like occasional 10min delays? Dashboard goes upside down -> revert then work through logs to figure out what the hell went wrong.
        inetknght 7mo ago
        What do you think of things you purchased changing over time into something you didn't purchase?
        estomagordo 7mo ago
        That's literally any software subscription ever.
        inetknght 7mo ago
        So you're perfectly okay with repeatedly paying for a shit product, getting shat on by the company in the form of being tested for feedback, and "maybe" getting a better product in the future. Mind you, that "better" isn't necessarily better for you but more explicitly better for the company you're paying. Sounds like someone who doesn't care about being a sheep. Or maybe someone whose salary depends on having sheep.
        nocman 7mo ago
        I think you are making far too wide-sweeping statements. I think most people here probably agree that if Anthropic drops Claude Code from the Pro plan after people have paid with the understanding that it is part of the package, that would be wrong, and they deserve to lose business over it. However, there are plenty of situations where A/B testing is entirely benign, and I would not have any problem with a company doing that testing without getting consent first. Every form of A/B testing is not done just for the gain of the company doing the testing.
  - skeledrew 7mo ago
    To play devil's advocate, without A/B testing a lot of decisions would be made with insufficient relevant data, and lead to subpar results that affect the many negatively form the road.
    - wat10000 7mo ago
      A lot of decisions made with A/B testing are also made with insufficient relevant data, but it's less obvious since it's easy to think the A/B results cover everything.
    - serf 7mo ago
      counter-point : the companies that are most famous for A/B testing routinely are also the ones with the most notoriously non-existent customer service departments globally, facebook/google/amazon/ebay. Groups that harbor dissatisfied customers by essentially being 'the only show in town.'. so, what i'm saying is : I think a lot of companies align themselves with the cash first and then measure whether or not the negative image/user impact is manageable . (in fact I know they operate this way.)
  - shimman 7mo ago
    Agreed and I can't wait until they regulate this stuff out of existence. It's absolutely hostile software technique that is deeply anti-human.
  - zorobo 7mo ago
    It is necessary to have a control group, just as in trials for new drugs.
    - greenchair 6mo ago
      no its not.
  - cycomanic 7mo ago
    So you're saying software should never change or you're happy with A testing, but not A/B testing
    - inetknght 7mo ago
      > So you're saying software should never change Generally, yes. Make your software better first before releasing it and you won't need to make changes to it. Want a new feature that you didn't have before? That's a new software product. > or you're happy with A testing, but not A/B testing I'm happy with testing when the user has explicitly opted-in for it.
- abtinf 7mo ago
  Perhaps vibe coding the A/B testing engine isn't the best idea.
abtinf 7mo ago
That tweet only makes things worse. On top of all their other nonsense recently, it actually convinced me to cancel my subscription. I can't trust Anthropic to manage their products in a way that supports my workflow.
- kelsey98765431 7mo ago
  I just cancelled before seeing this news. i was already pissed about constantly hitting limits on the 20 a month plan and looking for alternatives and this seals the deal. Bye bye!
  - anakaine 7mo ago
    I just paid for Pro for the first time 24 hours ago. Its been great, but the limits are crazy. It's nice not dealing with ChatGPTs sycophantic gaslighting, and not having random bugs. That said, I seem to be caught in that 2% test if I open in a private tab. What nonsense. I wouldn't be paying for Claude if it wasn't for its quality abilities, which necessarily includes Claude Code.
    - jsLavaGoat 7mo ago
      I can easily hit the weekly limit on Claude even on the $200 plan. I have yet to ever hit a rate limit on Codex $100. And the results are almost as good. And don't get me started on Anthropic's extra usage scam.
      - sroussey 7mo ago
        How do you hit that limit? I’m never close to it.
        karl_gluck 7mo ago
        Not the op, but it’s fairly easy to hit if you automate a kanban and have some stuff you want to get done. All those little “wouldn’t it be great if” tasks that show up after doing a big task become very doable, it just soaks your tokens.
sally_glance 7mo ago
Maybe a silly bet where the head of sales had 1-2 glasses of wine too much... "I bet they will still pay us 20 bucks/mo without CC! Don't believe me? I'm going to prove it!"
theptip 7mo ago
It’s pretty reasonable to say “demand is way up, quality is up, supply is constrained, and so price needs to rise”. It seems weird to segment this way though. Surely it’s better to just give Sonnet to your bottom tier, rather than cut out the entire Claide Code product entirely? Give folks a taste rather than lock the whole product behind a $100/mo plan.
- mewpmewp2 7mo ago
  But if Sonnet is bad it would give bad impression of the product, no? And it also takes compute, so you give a bad hallucinating impression of your product while still losing compute.
  - theptip 7mo ago
    It’s not bad though, it’s crazy good in comparison to any model older than 1y old. If you don’t have access to any vibe coding at all it’s gonna be life changing. But I think you are right, as long as Codex and Gemini are cheap alternatives then vs. 1yo models isn’t the correct comp. Then it’s probably better to just resegment the whole Claude Code product as an enterprise only tier. (That also has the advantage of kicking out all the Claw subscribers that screw over the token limit economics for normal $20/mo users.)
- wobfan 7mo ago
  I mean, this is why they do A/B testing. This way of testing stuff is not new at all, people who act genuinely surprised need to do a reality check. Companies want to maximize profit. They do this by testing what creates the biggest profit. A/B Testing is one of the ways to do this, and it has been used for decades in precisely this way.
ochronus 7mo ago
Haha, right, just like the recent uncommunicated changes to limits, cache, etc.
helsinkiandrew 7mo ago
> I need to know the pricing of Claude Code for NEW subscribers if I'm going to adopt it at a company with a growing team. I agree, but can you really use Claude Code on the Pro plan as a full time developer, or professional 'knowledge worker' without hitting the usage limits fairly early in the day anyway?
- ulimn 7mo ago
  I think it's more about how they approach their users in general that is the problem here.
- qingcharles 7mo ago
  Just the Pro Plan Claude Code on its own? Maybe you could last a full day on just using Sonnet. Maybe one Opus dab in the morning to plan your Haiku/Sonnet day? I have Pro Claude, Plus GPT and Pro Gemini. When one runs out I switch to another project on the next LLM. If I really need a task finished I'll restart it on another LLM, but I'm loathe to do that as it eats tokens just getting back up to speed.
- jltsiren 7mo ago
  It depends on the kind of work you do. I'm in the academia, and Claude's performance in my field could be described as a very fast junior grad student. When I use Claude Code, I typically spend a few hours figuring out what needs to be done exactly, and describing it in sufficient detail. Then Claude does it in 30 minutes, while an actual student would need days. And then I spend anything from minutes to days evaluating the results, depending on if it needs to be tested with real data and how much weirdness those tests uncover. But I also have other work to do beyond guiding the automated grad student. Which means my Claude Code usage rarely exceeds 1–2 hours/week.
- atraac 7mo ago
  I use Pro professionally and didn't hit limits most of the time. I believe I used up 5hr quota once or twice. We switched to Team sub and I'm on Standard(which is Pro x1.25 I believe). I don't vibecode entire applications, I ask it to make boilerplate, smaller, well scoped features or fix some errors. I don't let it go off with a prompt "make another netflix clone" cause I just don't see any real value in that
kingstnap 7mo ago
> Will hear it from us, not a screenshot on X or Reddit. Has this ever been true? You will almost always see some anecdotal screenshot a long time before any company would rat on themselves. Yes the random screenshots include a lot of false positives. But official comms have a lot of their own problems given how companies behave nowadays.
nkotov 7mo ago
Just like they gave plenty of notice regarding OpenClaw?
adastra22 7mo ago
> I need to know the pricing of Claude Code for NEW subscribers if I'm going to adopt it at a company with a growing team Subscriptions aren’t for company use.
vanessafridman6 7mo ago
[dead]

by simonw 7mo ago

rsynnott 7mo ago

A screenshot is, however, apparently good enough for _new_ subscribers.

minimaxir 7mo ago

A/B tests only work if the subjects don't realize they are in a A/B test.

inetknght 7mo ago
Solution: don't A/B test your users. A/B testing people without their informed consent is immoral, unethical, and should be illegal.
- vehemenz 7mo ago
  Depends entirely on the stakes and whether personal data is involved
  - inetknght 7mo ago
    > Depends entirely on the stakes and whether personal data is involved Sure. Let me just A/B test whether or not you'll respond positively or negatively to having your news delivered via push notification or delayed by 10 minutes. I'm sure you would appreciate being tested on without your consent, just so that I can make an extra quick buck at your expense. Nothing amoral or unethical about it.
    - pitched 7mo ago
      What do you think about slow rollouts for new features? Like, we think this new push notification system will be loved but let’s ship to only 1% of users in case there’s a horrible unforeseen consequence like occasional 10min delays? Dashboard goes upside down -> revert then work through logs to figure out what the hell went wrong.
      - inetknght 7mo ago
        What do you think of things you purchased changing over time into something you didn't purchase?
        estomagordo 7mo ago
        That's literally any software subscription ever.
        inetknght 7mo ago
        So you're perfectly okay with repeatedly paying for a shit product, getting shat on by the company in the form of being tested for feedback, and "maybe" getting a better product in the future. Mind you, that "better" isn't necessarily better for you but more explicitly better for the company you're paying. Sounds like someone who doesn't care about being a sheep. Or maybe someone whose salary depends on having sheep.
        nocman 7mo ago
        I think you are making far too wide-sweeping statements. I think most people here probably agree that if Anthropic drops Claude Code from the Pro plan after people have paid with the understanding that it is part of the package, that would be wrong, and they deserve to lose business over it. However, there are plenty of situations where A/B testing is entirely benign, and I would not have any problem with a company doing that testing without getting consent first. Every form of A/B testing is not done just for the gain of the company doing the testing.
- skeledrew 7mo ago
  To play devil's advocate, without A/B testing a lot of decisions would be made with insufficient relevant data, and lead to subpar results that affect the many negatively form the road.
  - wat10000 7mo ago
    A lot of decisions made with A/B testing are also made with insufficient relevant data, but it's less obvious since it's easy to think the A/B results cover everything.
  - serf 7mo ago
    counter-point : the companies that are most famous for A/B testing routinely are also the ones with the most notoriously non-existent customer service departments globally, facebook/google/amazon/ebay. Groups that harbor dissatisfied customers by essentially being 'the only show in town.'. so, what i'm saying is : I think a lot of companies align themselves with the cash first and then measure whether or not the negative image/user impact is manageable . (in fact I know they operate this way.)
- shimman 7mo ago
  Agreed and I can't wait until they regulate this stuff out of existence. It's absolutely hostile software technique that is deeply anti-human.
- zorobo 7mo ago
  It is necessary to have a control group, just as in trials for new drugs.
  - greenchair 6mo ago
    no its not.
- cycomanic 7mo ago
  So you're saying software should never change or you're happy with A testing, but not A/B testing
  - inetknght 7mo ago
    > So you're saying software should never change Generally, yes. Make your software better first before releasing it and you won't need to make changes to it. Want a new feature that you didn't have before? That's a new software product. > or you're happy with A testing, but not A/B testing I'm happy with testing when the user has explicitly opted-in for it.
abtinf 7mo ago
Perhaps vibe coding the A/B testing engine isn't the best idea.

abtinf 7mo ago

That tweet only makes things worse. On top of all their other nonsense recently, it actually convinced me to cancel my subscription. I can't trust Anthropic to manage their products in a way that supports my workflow.

kelsey98765431 7mo ago
I just cancelled before seeing this news. i was already pissed about constantly hitting limits on the 20 a month plan and looking for alternatives and this seals the deal. Bye bye!
- anakaine 7mo ago
  I just paid for Pro for the first time 24 hours ago. Its been great, but the limits are crazy. It's nice not dealing with ChatGPTs sycophantic gaslighting, and not having random bugs. That said, I seem to be caught in that 2% test if I open in a private tab. What nonsense. I wouldn't be paying for Claude if it wasn't for its quality abilities, which necessarily includes Claude Code.
  - jsLavaGoat 7mo ago
    I can easily hit the weekly limit on Claude even on the $200 plan. I have yet to ever hit a rate limit on Codex $100. And the results are almost as good. And don't get me started on Anthropic's extra usage scam.
    - sroussey 7mo ago
      How do you hit that limit? I’m never close to it.
      - karl_gluck 7mo ago
        Not the op, but it’s fairly easy to hit if you automate a kanban and have some stuff you want to get done. All those little “wouldn’t it be great if” tasks that show up after doing a big task become very doable, it just soaks your tokens.

sally_glance 7mo ago

Maybe a silly bet where the head of sales had 1-2 glasses of wine too much... "I bet they will still pay us 20 bucks/mo without CC! Don't believe me? I'm going to prove it!"

theptip 7mo ago

It’s pretty reasonable to say “demand is way up, quality is up, supply is constrained, and so price needs to rise”. It seems weird to segment this way though. Surely it’s better to just give Sonnet to your bottom tier, rather than cut out the entire Claide Code product entirely? Give folks a taste rather than lock the whole product behind a $100/mo plan.

mewpmewp2 7mo ago
But if Sonnet is bad it would give bad impression of the product, no? And it also takes compute, so you give a bad hallucinating impression of your product while still losing compute.
- theptip 7mo ago
  It’s not bad though, it’s crazy good in comparison to any model older than 1y old. If you don’t have access to any vibe coding at all it’s gonna be life changing. But I think you are right, as long as Codex and Gemini are cheap alternatives then vs. 1yo models isn’t the correct comp. Then it’s probably better to just resegment the whole Claude Code product as an enterprise only tier. (That also has the advantage of kicking out all the Claw subscribers that screw over the token limit economics for normal $20/mo users.)
wobfan 7mo ago
I mean, this is why they do A/B testing. This way of testing stuff is not new at all, people who act genuinely surprised need to do a reality check. Companies want to maximize profit. They do this by testing what creates the biggest profit. A/B Testing is one of the ways to do this, and it has been used for decades in precisely this way.

ochronus 7mo ago

Haha, right, just like the recent uncommunicated changes to limits, cache, etc.

helsinkiandrew 7mo ago

> I need to know the pricing of Claude Code for NEW subscribers if I'm going to adopt it at a company with a growing team. I agree, but can you really use Claude Code on the Pro plan as a full time developer, or professional 'knowledge worker' without hitting the usage limits fairly early in the day anyway?

ulimn 7mo ago
I think it's more about how they approach their users in general that is the problem here.
qingcharles 7mo ago
Just the Pro Plan Claude Code on its own? Maybe you could last a full day on just using Sonnet. Maybe one Opus dab in the morning to plan your Haiku/Sonnet day? I have Pro Claude, Plus GPT and Pro Gemini. When one runs out I switch to another project on the next LLM. If I really need a task finished I'll restart it on another LLM, but I'm loathe to do that as it eats tokens just getting back up to speed.
jltsiren 7mo ago
It depends on the kind of work you do. I'm in the academia, and Claude's performance in my field could be described as a very fast junior grad student. When I use Claude Code, I typically spend a few hours figuring out what needs to be done exactly, and describing it in sufficient detail. Then Claude does it in 30 minutes, while an actual student would need days. And then I spend anything from minutes to days evaluating the results, depending on if it needs to be tested with real data and how much weirdness those tests uncover. But I also have other work to do beyond guiding the automated grad student. Which means my Claude Code usage rarely exceeds 1–2 hours/week.
atraac 7mo ago
I use Pro professionally and didn't hit limits most of the time. I believe I used up 5hr quota once or twice. We switched to Team sub and I'm on Standard(which is Pro x1.25 I believe). I don't vibecode entire applications, I ask it to make boilerplate, smaller, well scoped features or fix some errors. I don't let it go off with a prompt "make another netflix clone" cause I just don't see any real value in that

kingstnap 7mo ago

> Will hear it from us, not a screenshot on X or Reddit. Has this ever been true? You will almost always see some anecdotal screenshot a long time before any company would rat on themselves. Yes the random screenshots include a lot of false positives. But official comms have a lot of their own problems given how companies behave nowadays.

nkotov 7mo ago

Just like they gave plenty of notice regarding OpenClaw?

adastra22 7mo ago

> I need to know the pricing of Claude Code for NEW subscribers if I'm going to adopt it at a company with a growing team Subscriptions aren’t for company use.

vanessafridman6 7mo ago

[dead]