6 ms·

I'm not sure how this will play out long term, but I really am not a fan of having to feel like I'm using a limited resource whenever I use an LLM. People like

I'm not sure how this will play out long term, but I really am not a fan of having to feel like I'm using a limited resource whenever I use an LLM. People like unlimited plans, we are used to them for internet, text messaging, etc. The current pricing models just feel bad.

andix 1y ago
I guess you need to get used to it. LLM token usage directly translates to energy consumption. There are also no flat fee electricity plans, it doesn't make any sense.
- idunnoboutthat 1y ago
  that's true of everything on the internet.
  - andix 1y ago
    Yes, but for most things it's not significant. For example Stack Overflow used to handle all their traffic from 9 on-prem servers (not sure if this is still the case). Millions of daily users. Power consumption and hardware cost is completely insignificant in this case. LLM inference pricing is mostly driven by power consumption and hardware cost (which also takes a lot of power/heat to manufacture).
    - Twirrim 1y ago
      > For example Stack Overflow used to handle all their traffic from 9 on-prem servers (not sure if this is still the case). Millions of daily users. Power consumption and hardware cost is completely insignificant in this case. They just finished their migration to the cloud, unracked their servers a few weeks ago https://stackoverflow.blog/2025/07/16/the-great-unracking-saying-goodbye-to-the-servers-at-our-physical-datacenter/ https://stackoverflow.blog/2025/07/16/the-great-unracking-sa...
      - jononor 1y ago
        Would have loved to get some more insights. Cost estimates, before and after, for example. But also if any architectural changes where needed, or what kind of other challenges and learnings they got from the migration.
  - tracker1 1y ago
    An "AI" box with a few high end gpu/npu cards takes more energy in a 4u box than an entire rack of commodity hardware takes. It's not nearly comparible... meaning entirely new and expansive infrastructure costs to support the high energy. That also doesn't count the needs for really high bandwidth networking to these systems. Not to mention the insanely more expensive hardware costs. The infrastructure and hardware costs are seriously more costly than typical internet apps and storage.
andruby 1y ago
Unlimited works well for everything that is “too cheap to meter”. Internet, text messages, etc are roughly that: the direct costs are so cheap. That’s not the case with LLM’s at this moment. There are significant direct costs to each long-running agent.
- rmujica 1y ago
  Internet and SMS used to be expensive and metered until they weren't thanks to technological advances and expanded use. I think LLMs will follow the same path, maybe on a shorter timespan.
  - cmsjustin 1y ago
    They were not expensive to operate, they were only expensive for consumers
    - KaiserPro 1y ago
      To lay the cables required a huge amount of capital, to make that feasible its required financial engineering. That translates to high operating expenses.
      - AlotOfReading 1y ago
        SMS was originally piggybacking off unused bytes in packets already being sent to the tower, which was being paid for by existing phone bills. The only significant expenses involved transiting between networks. That was a separate surcharge in the early days.
        ssk42 1y ago
        Used to be? What changed?
        daveguy 1y ago
gedy 1y ago
I get it, and feel the same way but the current LLMs are very resource intensive. To the point that I'm reluctant to go all in on these tools if in future we get rug-pulled once companies admit "okay, this was not sustainable at that price.."
- andix 1y ago
  Some people claim we already reached peak-LLM. It's cheap and powerful right now, in the future it might just get more expensive, or worse quality for the same price.
- dust42 1y ago
  I am really afraid of this as well. When using one of the plugins for vscode, I would easily use a few million tokens a day. Thus with Claude Code I assume it isn't much different under the hood. The prices on openrouter are roughly $5/M for the better models. Therefore paying $20/month can't be sustainable. Once enough developers are addicted to AI assisted coding the VCs will inevitably pull the rug. I wonder if Alibaba will put out a 100B A10B coder model which could probably run for $0.5/M while giving decent output. That would be easily affordable for most developers/companies.
sergiotapia 1y ago
Think of an insane number of requests. Now 20x it, that's what the top 1% of Claude users are at. Just fleecing the service dry. hard problem, what else could Claude do tbh
- handfuloflight 1y ago
  ...maybe use their superintelligent AI to come up with a solution that specifically targets the abusers?
  - cyanydeez 1y ago
    [flagged]
  - SatvikBeri 1y ago
    ...like adding limits that only affect a small fraction of users?
    - Yossarrian22 1y ago
      Is 1/20 small?
      - dotancohen 1y ago
        I'm Jewish. We take that right off the top shortly after birth.
  - blitzar 1y ago
    Claude says - The key is maintaining user agency—let them choose how to manage their usage rather than imposing arbitrary cutoffs. It suggests: Transparent queueing - Instead of blocking, queue requests with clear wait time estimates. Users can choose to wait or reschedule. Usage smoothing - Soft caps with gradually increasing response times (e.g., 2s → 5s → 10s) rather than hard cutoffs. Declared priority queues - Let users specify request urgency. Background tasks get lower priority but aren't blocked. Time-based scheduling - Allow users to schedule non-urgent work during off-peak hours at standard rates. Burst credits - Banking system where users accumulate credits during low usage periods for occasional heavy use.
qiller 1y ago
I'm ok using a limited resource _if_ I know how much of it I am using. The lack of visible progress towards limits is annoying.
- flkiwi 1y ago
  It's not exactly the same thing, but imagine my complete surprise when, in the middle of a discussion with Copilot and without warning, it announced that the conversation had reached its length limit and I had to start a new one with absolutely no context from the current one. Copilot has many, many usability quirks, but that was the first that actually made me mad.
  - jononor 1y ago
    ChatGPT and Claude do the same. And I have noticed that model performance can often degrade a lot before such a hard limit. So even when not hitting the hard limit, splitting out to a new session can be useful. Context management is the new prompt engineering...
  - stronglikedan 1y ago
    The craziest thing to me is that it actually completely stopped you in your tracks instead of upselling you on the spot to continue.
- blalezarian 1y ago
  Totally agree with this. I live in constant anxiety not knowing how far into my usage I am in all the time.
- steveklabnik 1y ago
  npx ccusage@latest I'm assuming it'll get updated to include these windows as well. Pass in "blocks --live" to get a live dashboard!
  - mtmail 1y ago
    Package page (with screenshot) https://www.npmjs.com/package/ccusage https://www.npmjs.com/package/ccusage
thorum 1y ago
The long term is unlimited access to local LLMs that are better than 2025’s best cloud models and good enough for 99% of your needs, and limited access to cloud models for when you need to bring more intelligence to bear on a problem. LLMs will become more efficient, GPUs, memory and storage will continue to become cheaper and more commonplace. We’re just in the awkward early days where things are still being figured out.
- pakitan 1y ago
  I'm often using LLMs for stuff that requires recent data. No way I'm running a web crawler in addition to my local LLM. For coding it could theoretically work as you don't always need latest and greatest but would still make me anxious.
  - data-ottawa 1y ago
    That’s a perfect use case with MCP though. My biggest issue is local models I can run on my m1/m4 mbp are not smart enough to use tools consistently, and the context windows are too small for iterative uses. The last year has seen a lot of improvement in small models though (gemma 3n is fantastic), so hopefully it’s only a matter of time.
furyofantares 1y ago
The Sonnet usage does not really look limited at 240-480 hours per week (a week has 168 hours in it). Opus at 24-40 looks pretty good too. A little hard to believe they aren't losing a bunch of money still if you're using those limits tbh.
- clharman 1y ago
  Pretty sure they are still losing money on it, which is great for us. And these limits wouldn't even be happening if there weren't people bragging about having their CC running constantly for 30 hours writing 2 million lines of (doubtless bad) code. And sharing accounts to try to get even MORE usage. It's all that swarm guy tbh and he's proud of it.
- wrs 1y ago
  You can run multiple instances of Claude Code at the same time.
  - furyofantares 1y ago
    I know, I do it all the time! I wasn't calling out 240 hours as being impossible to hit in 168 hour weeks. I suppose "does not really look limited" could be read multiple ways - I did not mean it's literally unlimited, just that it doesn't look very limited. You can make your own comparison to however many hours you usually spend working in a week and how many sessions you have active on average during that time.
- j45 1y ago
  The calculation of hours is a little tough to imagine sometimes, is it the inference time itself, or the period of time used? Is there an average token cost per hour of use (average or explicit?)
  - Filligree 1y ago
    It had better be inference time; I regularly have Claude call out to tools that take hours to run.
KronisLV 1y ago
I personally take an issue with them expecting that your usage would be more or less consistent throughout the month. Instead, I might have low usage throughout most of the month and then an 11 hour binge a few days, which in most cases would involve running into rate limits (either that, or just token limitations for inputs). That's why using the API directly and paying for tokens anything past that basic usage feels a bit nicer, since it's my wallet that becomes the limitation then, not some arbitrary limits dreamed up by others. Plus with something like OpenRouter, you can also avoid subscription tier related limits like https://docs.anthropic.com/en/api/rate-limits#rate-limits https://docs.anthropic.com/en/api/rate-limits#rate-limits Though for now Gemini 2.5 Pro seems to work a bit better than Claude for my code writing/refactoring/explanation/exploration needs. Curious what other cost competitive options are out there.
- bugglebeetle 1y ago
  Gemini 2.5 Pro is a better coding model, but Gemini CLI is way behind Claude Code, perhaps because the model itself isn’t well-tuned for agentic work. If you’re just doing targeted refactoring and exploration, you can copy and paste back and forth from the web app for $20 a month.
  - ewoodrich 1y ago
    Gemini 2.5 Pro made some big post-launch improvements for tool calling/agentic usage that made it go from “pulling teeth” to “almost as smooth as Claude” in Cline/Roo Code (which is saying something since Cline was originally built around Claude tool use specifically). So the team at least seems to be aware of its shortcomings in that area and working to improve it with some success which I appreciate. But you are correct that Gemini CLI still lags behind for whatever reason. It gets stuck in endless thought loops way too often for me, like maybe 1/15 tasks hits a thought loop burning API credits or it just never exits from the “Completing task, Verifying completion, Reviewing completion, Assessing completion status…” phase (watching the comical number of ways it rephrases it is pretty funny though). Meanwhile I’ve only had maybe one loop over a period of a couple months using Gemini 2.5 Pro heavily in Roo Code with the most recent version so it seems like an issue with the CLI specifically.
belter 1y ago
The real bottleneck isn’t Jevons paradox, it’s the Theory of Constraints. A human brain runs on under 20 W, yet every major LLM vendor is burning cash and running up against power supply limits. If anything pops this bubble, it won’t be ethics panels or model tweaks but subscription prices finally reflecting those electricity bills. At that point, companies might rediscover the ROI of good old meat based AI.
- hkt 1y ago
  I suspect for this reason we are going to see a lot of attempts at applied AI: I saw an article semi-recently about an AI weather forecasting model using considerably less power than it's algorithmic predecessor, for instance. The answer is, as ever, to climb the value chain and make every penny (and joule) count.
- dotancohen 1y ago
  > good old meat based AI. NI, or really just I. Though some of us might fall into the NS category instead.
- TeMPOraL 1y ago
  Where is this oft-repeated idea coming from? Inference isn't that expensive.
  - belter 1y ago
    My back of envelope estimate, is that even a partly restricted plan, would need to cost roughly $4,000–$4,500 per month just to break even.
- margalabargala 1y ago
  Meat has far higher input requirements for good performance above raw energy
  - belter 1y ago
    Hire Vegan Developers... :-)
vlan0 1y ago
Do you wonder why that feeling arising inside of you?
beepbooptheory 1y ago
I'm sure people hate this mindset here, but any time I use an LLM I just picture the thousands of fans spinning, the heat of the datacenter... I treat each prompt like a painful interval where I am leaving my door open on a hot day. I know nobody else really cares.. In some ways I wish I didn't think like this.. But its at this point not even an ethical thing, its just a weird fixation. Like I can't help but feel we are all using ovens when we would be fine with a toasters.
Jcampuzano2 1y ago
My opinion is all of these tools should completely get rid of the "pay 20/month, 200/month", etc just to get access to some beholden rate limited amount that becomes hard to track. Mask off completely and just make it completely usage based for everyone. You could do something for trial users like first 20 (pick your number here) requests are free if you really need to in order to get people on board. Or you could do tiered pricing like first 20 free, next 200 for X rate, next 200 for X*1.25 rate, and then for really high usage users charge the full cost to make up for their extreme patterns. With this they can still subsidize for the people who stay lower on usage rates for market share. Of course you can replace 200 requests with just token usage if that makes sense but I'm sure they can do the math to make it work with request limits if they work hard enough. Offer better than open-router pricing and that keeps people in your system instead of reaching for 3rd party tools. If your tool is that good, even with usage based it will get users. The issue is all the providers are both subsidizing users to get market share, but also trying to prohibit bad actors and the most egregious usage patterns. The only way this 100% becomes a non-issue is usage based for everything with no entry fee. But this also hurts some who pay a subscription but DONT use enough to account for the usage based fees. So some sales people probably don't like that option either. It also makes it easier for people to shop around instead of feeling stuck for a month or two since most people don't want multiple subs at once.
- CodeBrad 1y ago
  I think Claude Code also already has the option to provide an API key directly for usage based pricing. I'm a fan of having both a subscription and a usage based plan available. The subscription is effectively a built in spending limit. If I regularly hit it and need more value, I can switch to an API key for unlimited usage. The downside is you are potentially paying for something you don't use, but that is the same for all subscription services.
  - tqwhite 1y ago
    I use API but think about getting the $100/mo plan and using API for overflow if it occurs. But I have slow months and think that might not actually be the winner. Basically I'm going to wait and see before I sign up for auto-pay.
jm4 1y ago
Blame the idiots who abused it. Like that guy who posted a video a couple weeks ago where he had like 6 instances going nonstop and he was controlling it with his voice. There was some other project posted recently that was queuing up requests so that you could hit the limits in every time block. I've seen reddit posts where people were looking for others to share team accounts. It's always the morons who ruin a good thing. Unless/until I start having problems with limits, I'm willing to reserve judgment. On a max plan, I expect to be able to use it throughout my workday without hitting limits. Occasionally, I run a couple instances because I'm multitasking and those were the only times I would hit limits on the 5x plan. I can live with that. I don't hit limits on the 20x plan.
- jjani 1y ago
  Such abusers are very rarely a whole 5% of accounts, almost certainly <=2%.
  - npc_anon 1y ago
    In a recent tweet the company gave an example of a user running Claude 24/7 costing them "tens of thousands of dollars". For any such users it requires hundreds if not thousands of "normal" users just to break even.
WD-42 1y ago
Try running one locally and observe as the temperature in your office rises a few degrees and the lights dim with every prompt. I didn’t really get the pricing myself until I got a desktop to do local inference. There’s a reason why these companies want to build nuclear plants next to their data centers.
volleygman180 1y ago
Agreed. The internet would be livid if Apple Music or Hulu limited how many hours you were allowed to stream per week. Especially the users who pay for the top-tier packages that include 4K (or lossless for music), extra channel add-ons, etc.
xtracto 1y ago
This is actually so fascinating to me. I remember when we had metered & very expensive long distance calls, "metered" dial-up Internet (Todito Card!), then capped DSL internet, then metered Mobile calls, SMSs and then Metered Mobile internet (that last one we still do). The stuff that we do now, my 13 year old self in 1994 would never dream of! When I dialed my 33.6kbps modem and left it going the whole night, to download an mp3. It's exciting that nowadays we complain about Intelligent Agents bandwidth plans!! Can you imagine! I cannot imagine the stuff that will be built when this tech has the same availability as The Internet, or POTS!
NicuCalcea 1y ago
> I really am not a fan of having to feel like I'm using a limited resource whenever I use an LLM Well, it is a limited resource, I'm glad they're making that clear.
- hn_throwaway_99 1y ago
  Yeah, this sentence "I really am not a fan of having to feel like I'm using a limited resource whenever I use an LLM" felt like "I'm not a fan of reality" to me. Lots of things still have usage-based pricing (last I checked no gas stations are offering "all you can fill up" specials), and those things work out fine.
  - NicuCalcea 1y ago
    These subscriptions only work because lighter users subsidise heavier users, but I guess the really heavy users are such big outliers that the maths isn't working out for Anthropic. I'm really not liking this neo-rentier capitalism we find ourselves in.
raincole 1y ago
Unlimited plans = the users who use it least subsidize the users who use it a lot. I don't really know how it's sustainable for something like SOTA LLMs.
nine_k 1y ago
Resources that are "Unlimited" in marketing speak are rate-limited in practice. Your unlimited internet connection limits your daily transfer by bandwidth, both at your port and at the remote service ports. Your daily amount of SMS is limited by sending rate. Your all-you-can-eat restaurant order is limited by your belly. No wonder that access to an expensive API which is an LLM is also rate-limited. What does surprise me is that you can't buy an extra serving by paying more (twice the limit for 3x the cost, for instance). Either subscriptions don't make enough money, or their limits are at their datacenters and they have no spare capacity for premium plans.
- FanaHOVA 1y ago
  You can pay more. It's unlimited (sorta) through API at API pricing.
smcl 1y ago
The problem is that this company is haemhorraging money and cannot possibly offer an unlimited plan.
- pluto_modadic 1y ago
  *cannot possibly subsidize an unlimited plan and must course correct on pricing to reflect cost-plus-pricing instead.
  - smcl 1y ago
    Last year they had $900 million in revenue and ended up losing $5.6 billion. I suspect cutting off a few whales isn't enough to reverse that and they're gonna need to "course correct" a bit further

andix 1y ago

I guess you need to get used to it. LLM token usage directly translates to energy consumption. There are also no flat fee electricity plans, it doesn't make any sense.

idunnoboutthat 1y ago
that's true of everything on the internet.
- andix 1y ago
  Yes, but for most things it's not significant. For example Stack Overflow used to handle all their traffic from 9 on-prem servers (not sure if this is still the case). Millions of daily users. Power consumption and hardware cost is completely insignificant in this case. LLM inference pricing is mostly driven by power consumption and hardware cost (which also takes a lot of power/heat to manufacture).
  - Twirrim 1y ago
    > For example Stack Overflow used to handle all their traffic from 9 on-prem servers (not sure if this is still the case). Millions of daily users. Power consumption and hardware cost is completely insignificant in this case. They just finished their migration to the cloud, unracked their servers a few weeks ago https://stackoverflow.blog/2025/07/16/the-great-unracking-saying-goodbye-to-the-servers-at-our-physical-datacenter/ https://stackoverflow.blog/2025/07/16/the-great-unracking-sa...
    - jononor 1y ago
      Would have loved to get some more insights. Cost estimates, before and after, for example. But also if any architectural changes where needed, or what kind of other challenges and learnings they got from the migration.
- tracker1 1y ago
  An "AI" box with a few high end gpu/npu cards takes more energy in a 4u box than an entire rack of commodity hardware takes. It's not nearly comparible... meaning entirely new and expansive infrastructure costs to support the high energy. That also doesn't count the needs for really high bandwidth networking to these systems. Not to mention the insanely more expensive hardware costs. The infrastructure and hardware costs are seriously more costly than typical internet apps and storage.

andruby 1y ago

Unlimited works well for everything that is “too cheap to meter”. Internet, text messages, etc are roughly that: the direct costs are so cheap. That’s not the case with LLM’s at this moment. There are significant direct costs to each long-running agent.

rmujica 1y ago
Internet and SMS used to be expensive and metered until they weren't thanks to technological advances and expanded use. I think LLMs will follow the same path, maybe on a shorter timespan.
- cmsjustin 1y ago
  They were not expensive to operate, they were only expensive for consumers
  - KaiserPro 1y ago
    To lay the cables required a huge amount of capital, to make that feasible its required financial engineering. That translates to high operating expenses.
    - AlotOfReading 1y ago
      SMS was originally piggybacking off unused bytes in packets already being sent to the tower, which was being paid for by existing phone bills. The only significant expenses involved transiting between networks. That was a separate surcharge in the early days.
      - ssk42 1y ago
        Used to be? What changed?
        daveguy 1y ago

gedy 1y ago

I get it, and feel the same way but the current LLMs are very resource intensive. To the point that I'm reluctant to go all in on these tools if in future we get rug-pulled once companies admit "okay, this was not sustainable at that price.."

andix 1y ago
Some people claim we already reached peak-LLM. It's cheap and powerful right now, in the future it might just get more expensive, or worse quality for the same price.
dust42 1y ago
I am really afraid of this as well. When using one of the plugins for vscode, I would easily use a few million tokens a day. Thus with Claude Code I assume it isn't much different under the hood. The prices on openrouter are roughly $5/M for the better models. Therefore paying $20/month can't be sustainable. Once enough developers are addicted to AI assisted coding the VCs will inevitably pull the rug. I wonder if Alibaba will put out a 100B A10B coder model which could probably run for $0.5/M while giving decent output. That would be easily affordable for most developers/companies.

sergiotapia 1y ago

Think of an insane number of requests. Now 20x it, that's what the top 1% of Claude users are at. Just fleecing the service dry. hard problem, what else could Claude do tbh

handfuloflight 1y ago
...maybe use their superintelligent AI to come up with a solution that specifically targets the abusers?
- cyanydeez 1y ago
  [flagged]
- SatvikBeri 1y ago
  ...like adding limits that only affect a small fraction of users?
  - Yossarrian22 1y ago
    Is 1/20 small?
    - dotancohen 1y ago
      I'm Jewish. We take that right off the top shortly after birth.
- blitzar 1y ago
  Claude says - The key is maintaining user agency—let them choose how to manage their usage rather than imposing arbitrary cutoffs. It suggests: Transparent queueing - Instead of blocking, queue requests with clear wait time estimates. Users can choose to wait or reschedule. Usage smoothing - Soft caps with gradually increasing response times (e.g., 2s → 5s → 10s) rather than hard cutoffs. Declared priority queues - Let users specify request urgency. Background tasks get lower priority but aren't blocked. Time-based scheduling - Allow users to schedule non-urgent work during off-peak hours at standard rates. Burst credits - Banking system where users accumulate credits during low usage periods for occasional heavy use.

qiller 1y ago

I'm ok using a limited resource _if_ I know how much of it I am using. The lack of visible progress towards limits is annoying.

flkiwi 1y ago
It's not exactly the same thing, but imagine my complete surprise when, in the middle of a discussion with Copilot and without warning, it announced that the conversation had reached its length limit and I had to start a new one with absolutely no context from the current one. Copilot has many, many usability quirks, but that was the first that actually made me mad.
- jononor 1y ago
  ChatGPT and Claude do the same. And I have noticed that model performance can often degrade a lot before such a hard limit. So even when not hitting the hard limit, splitting out to a new session can be useful. Context management is the new prompt engineering...
- stronglikedan 1y ago
  The craziest thing to me is that it actually completely stopped you in your tracks instead of upselling you on the spot to continue.
blalezarian 1y ago
Totally agree with this. I live in constant anxiety not knowing how far into my usage I am in all the time.
steveklabnik 1y ago
npx ccusage@latest I'm assuming it'll get updated to include these windows as well. Pass in "blocks --live" to get a live dashboard!
- mtmail 1y ago
  Package page (with screenshot) https://www.npmjs.com/package/ccusage https://www.npmjs.com/package/ccusage

thorum 1y ago

The long term is unlimited access to local LLMs that are better than 2025’s best cloud models and good enough for 99% of your needs, and limited access to cloud models for when you need to bring more intelligence to bear on a problem. LLMs will become more efficient, GPUs, memory and storage will continue to become cheaper and more commonplace. We’re just in the awkward early days where things are still being figured out.

pakitan 1y ago
I'm often using LLMs for stuff that requires recent data. No way I'm running a web crawler in addition to my local LLM. For coding it could theoretically work as you don't always need latest and greatest but would still make me anxious.
- data-ottawa 1y ago
  That’s a perfect use case with MCP though. My biggest issue is local models I can run on my m1/m4 mbp are not smart enough to use tools consistently, and the context windows are too small for iterative uses. The last year has seen a lot of improvement in small models though (gemma 3n is fantastic), so hopefully it’s only a matter of time.

furyofantares 1y ago

The Sonnet usage does not really look limited at 240-480 hours per week (a week has 168 hours in it). Opus at 24-40 looks pretty good too. A little hard to believe they aren't losing a bunch of money still if you're using those limits tbh.

clharman 1y ago
Pretty sure they are still losing money on it, which is great for us. And these limits wouldn't even be happening if there weren't people bragging about having their CC running constantly for 30 hours writing 2 million lines of (doubtless bad) code. And sharing accounts to try to get even MORE usage. It's all that swarm guy tbh and he's proud of it.
wrs 1y ago
You can run multiple instances of Claude Code at the same time.
- furyofantares 1y ago
  I know, I do it all the time! I wasn't calling out 240 hours as being impossible to hit in 168 hour weeks. I suppose "does not really look limited" could be read multiple ways - I did not mean it's literally unlimited, just that it doesn't look very limited. You can make your own comparison to however many hours you usually spend working in a week and how many sessions you have active on average during that time.
j45 1y ago
The calculation of hours is a little tough to imagine sometimes, is it the inference time itself, or the period of time used? Is there an average token cost per hour of use (average or explicit?)
- Filligree 1y ago
  It had better be inference time; I regularly have Claude call out to tools that take hours to run.

KronisLV 1y ago

I personally take an issue with them expecting that your usage would be more or less consistent throughout the month. Instead, I might have low usage throughout most of the month and then an 11 hour binge a few days, which in most cases would involve running into rate limits (either that, or just token limitations for inputs). That's why using the API directly and paying for tokens anything past that basic usage feels a bit nicer, since it's my wallet that becomes the limitation then, not some arbitrary limits dreamed up by others. Plus with something like OpenRouter, you can also avoid subscription tier related limits like https://docs.anthropic.com/en/api/rate-limits#rate-limits https://docs.anthropic.com/en/api/rate-limits#rate-limits Though for now Gemini 2.5 Pro seems to work a bit better than Claude for my code writing/refactoring/explanation/exploration needs. Curious what other cost competitive options are out there.

bugglebeetle 1y ago
Gemini 2.5 Pro is a better coding model, but Gemini CLI is way behind Claude Code, perhaps because the model itself isn’t well-tuned for agentic work. If you’re just doing targeted refactoring and exploration, you can copy and paste back and forth from the web app for $20 a month.
- ewoodrich 1y ago
  Gemini 2.5 Pro made some big post-launch improvements for tool calling/agentic usage that made it go from “pulling teeth” to “almost as smooth as Claude” in Cline/Roo Code (which is saying something since Cline was originally built around Claude tool use specifically). So the team at least seems to be aware of its shortcomings in that area and working to improve it with some success which I appreciate. But you are correct that Gemini CLI still lags behind for whatever reason. It gets stuck in endless thought loops way too often for me, like maybe 1/15 tasks hits a thought loop burning API credits or it just never exits from the “Completing task, Verifying completion, Reviewing completion, Assessing completion status…” phase (watching the comical number of ways it rephrases it is pretty funny though). Meanwhile I’ve only had maybe one loop over a period of a couple months using Gemini 2.5 Pro heavily in Roo Code with the most recent version so it seems like an issue with the CLI specifically.

belter 1y ago

The real bottleneck isn’t Jevons paradox, it’s the Theory of Constraints. A human brain runs on under 20 W, yet every major LLM vendor is burning cash and running up against power supply limits. If anything pops this bubble, it won’t be ethics panels or model tweaks but subscription prices finally reflecting those electricity bills. At that point, companies might rediscover the ROI of good old meat based AI.

hkt 1y ago
I suspect for this reason we are going to see a lot of attempts at applied AI: I saw an article semi-recently about an AI weather forecasting model using considerably less power than it's algorithmic predecessor, for instance. The answer is, as ever, to climb the value chain and make every penny (and joule) count.
dotancohen 1y ago
> good old meat based AI. NI, or really just I. Though some of us might fall into the NS category instead.
TeMPOraL 1y ago
Where is this oft-repeated idea coming from? Inference isn't that expensive.
- belter 1y ago
  My back of envelope estimate, is that even a partly restricted plan, would need to cost roughly $4,000–$4,500 per month just to break even.
margalabargala 1y ago
Meat has far higher input requirements for good performance above raw energy
- belter 1y ago
  Hire Vegan Developers... :-)

vlan0 1y ago

Do you wonder why that feeling arising inside of you?

beepbooptheory 1y ago

I'm sure people hate this mindset here, but any time I use an LLM I just picture the thousands of fans spinning, the heat of the datacenter... I treat each prompt like a painful interval where I am leaving my door open on a hot day. I know nobody else really cares.. In some ways I wish I didn't think like this.. But its at this point not even an ethical thing, its just a weird fixation. Like I can't help but feel we are all using ovens when we would be fine with a toasters.

Jcampuzano2 1y ago

My opinion is all of these tools should completely get rid of the "pay 20/month, 200/month", etc just to get access to some beholden rate limited amount that becomes hard to track. Mask off completely and just make it completely usage based for everyone. You could do something for trial users like first 20 (pick your number here) requests are free if you really need to in order to get people on board. Or you could do tiered pricing like first 20 free, next 200 for X rate, next 200 for X*1.25 rate, and then for really high usage users charge the full cost to make up for their extreme patterns. With this they can still subsidize for the people who stay lower on usage rates for market share. Of course you can replace 200 requests with just token usage if that makes sense but I'm sure they can do the math to make it work with request limits if they work hard enough. Offer better than open-router pricing and that keeps people in your system instead of reaching for 3rd party tools. If your tool is that good, even with usage based it will get users. The issue is all the providers are both subsidizing users to get market share, but also trying to prohibit bad actors and the most egregious usage patterns. The only way this 100% becomes a non-issue is usage based for everything with no entry fee. But this also hurts some who pay a subscription but DONT use enough to account for the usage based fees. So some sales people probably don't like that option either. It also makes it easier for people to shop around instead of feeling stuck for a month or two since most people don't want multiple subs at once.

CodeBrad 1y ago
I think Claude Code also already has the option to provide an API key directly for usage based pricing. I'm a fan of having both a subscription and a usage based plan available. The subscription is effectively a built in spending limit. If I regularly hit it and need more value, I can switch to an API key for unlimited usage. The downside is you are potentially paying for something you don't use, but that is the same for all subscription services.
- tqwhite 1y ago
  I use API but think about getting the $100/mo plan and using API for overflow if it occurs. But I have slow months and think that might not actually be the winner. Basically I'm going to wait and see before I sign up for auto-pay.

jm4 1y ago

Blame the idiots who abused it. Like that guy who posted a video a couple weeks ago where he had like 6 instances going nonstop and he was controlling it with his voice. There was some other project posted recently that was queuing up requests so that you could hit the limits in every time block. I've seen reddit posts where people were looking for others to share team accounts. It's always the morons who ruin a good thing. Unless/until I start having problems with limits, I'm willing to reserve judgment. On a max plan, I expect to be able to use it throughout my workday without hitting limits. Occasionally, I run a couple instances because I'm multitasking and those were the only times I would hit limits on the 5x plan. I can live with that. I don't hit limits on the 20x plan.

jjani 1y ago
Such abusers are very rarely a whole 5% of accounts, almost certainly <=2%.
- npc_anon 1y ago
  In a recent tweet the company gave an example of a user running Claude 24/7 costing them "tens of thousands of dollars". For any such users it requires hundreds if not thousands of "normal" users just to break even.

WD-42 1y ago

Try running one locally and observe as the temperature in your office rises a few degrees and the lights dim with every prompt. I didn’t really get the pricing myself until I got a desktop to do local inference. There’s a reason why these companies want to build nuclear plants next to their data centers.

volleygman180 1y ago

Agreed. The internet would be livid if Apple Music or Hulu limited how many hours you were allowed to stream per week. Especially the users who pay for the top-tier packages that include 4K (or lossless for music), extra channel add-ons, etc.

xtracto 1y ago

This is actually so fascinating to me. I remember when we had metered & very expensive long distance calls, "metered" dial-up Internet (Todito Card!), then capped DSL internet, then metered Mobile calls, SMSs and then Metered Mobile internet (that last one we still do). The stuff that we do now, my 13 year old self in 1994 would never dream of! When I dialed my 33.6kbps modem and left it going the whole night, to download an mp3. It's exciting that nowadays we complain about Intelligent Agents bandwidth plans!! Can you imagine! I cannot imagine the stuff that will be built when this tech has the same availability as The Internet, or POTS!

NicuCalcea 1y ago

> I really am not a fan of having to feel like I'm using a limited resource whenever I use an LLM Well, it is a limited resource, I'm glad they're making that clear.

hn_throwaway_99 1y ago
Yeah, this sentence "I really am not a fan of having to feel like I'm using a limited resource whenever I use an LLM" felt like "I'm not a fan of reality" to me. Lots of things still have usage-based pricing (last I checked no gas stations are offering "all you can fill up" specials), and those things work out fine.
- NicuCalcea 1y ago
  These subscriptions only work because lighter users subsidise heavier users, but I guess the really heavy users are such big outliers that the maths isn't working out for Anthropic. I'm really not liking this neo-rentier capitalism we find ourselves in.

raincole 1y ago

Unlimited plans = the users who use it least subsidize the users who use it a lot. I don't really know how it's sustainable for something like SOTA LLMs.

nine_k 1y ago

Resources that are "Unlimited" in marketing speak are rate-limited in practice. Your unlimited internet connection limits your daily transfer by bandwidth, both at your port and at the remote service ports. Your daily amount of SMS is limited by sending rate. Your all-you-can-eat restaurant order is limited by your belly. No wonder that access to an expensive API which is an LLM is also rate-limited. What does surprise me is that you can't buy an extra serving by paying more (twice the limit for 3x the cost, for instance). Either subscriptions don't make enough money, or their limits are at their datacenters and they have no spare capacity for premium plans.

FanaHOVA 1y ago
You can pay more. It's unlimited (sorta) through API at API pricing.

smcl 1y ago

The problem is that this company is haemhorraging money and cannot possibly offer an unlimited plan.

pluto_modadic 1y ago
*cannot possibly subsidize an unlimited plan and must course correct on pricing to reflect cost-plus-pricing instead.
- smcl 1y ago
  Last year they had $900 million in revenue and ended up losing $5.6 billion. I suspect cutting off a few whales isn't enough to reverse that and they're gonna need to "course correct" a bit further