6 ms·

Most people in this thread are quibbling about the exact degree of utility LLMs provide, which a tedious argument. What's more interesting to me is, per the ar

Most people in this thread are quibbling about the exact degree of utility LLMs provide, which a tedious argument. What's more interesting to me is, per the article, the concern regarding everyone who is leaning into LLMs without realizing (or downplaying) the exorbitant, externalized cost. Our current LLM usage is being subsidized to the point of being free by outside investment. One day when the well runs dry, you must be able to either pay the actual cost (barring grand technology breakthroughs), or switch back to non-LLM workflows. I run local LLMs infrequently, and every single prompt makes my beefy PC sounds like a jet engine taking off. It's a great reminder to not become codependent.

bogzz 10mo ago
I am sorry, but this kind of level-headed and realistic take is completely unacceptable on hackernews, and you should be ashamed of yourself. This is not a place for rational discussion when it comes to LLMs. LLMs are amazing and they will change everything, and then everything will be changed.
JeremyNT 10mo ago
This is a good point but you can see the price "ceiling" by examining the prices for PCs that can effectively run local models. A DGX Spark is ~$4k (plus power) for example. That's not nothing, but it's still not very much to pay compared to e.g. the cost of a FTE.
- __float 10mo ago
  You're counting the cost of running the model, but what about training it? You can't count the compute and data costs at $0.
  - philipkglass 10mo ago
    You can assume that already-published open weights models are available at $0, regardless of how much money was sunk into their original development. These models will look increasingly stale over time but most software development doesn't change quickly. If a model can generate capable and up-to-date Python, C++, Java, or Javascript code in 2025 then you can expect it to still be a useful model in 2035 (based on the observation that then-modern code in these languages from 2015 works fine today, even if styles have shifted).
    - sarchertech 10mo ago
      >2025-2035 Depending on other people to maintain backward compatibility so that you can keep coding like it’s 2025 is its own problematic dependency. You could certainly do it but it would be limiting. Imagine that you had a model trained on examples from before 2013 and your boss wants you to take over maintenance for a React app.
      - nikkwong 10mo ago
        You're all referencing the strange idea in a world where there would be no open-weight coding models trained in the future. Even in a world where VC spending vanished completely, coding models are such a valuable utility that I'm sure at the very least companies/individuals would crowdsource them on a reoccurring basis, keeping them up to date. The value of this technology has been established, it's not leaving anytime soon.
        sarchertech 10mo ago
        SOTA models cost hundreds of millions to train. I doubt anyone is crowdsourcing that. And that’s assuming you already have a lot of the infrastructure in place.
        nikkwong 10mo ago
        I think faang and the like would probably crowdsource it given that they would—according to the hypothesis presented—would only have to do it every few years, and ostensibly are realizing improved developer productivity from them.
        sarchertech 10mo ago
        I don’t think the incentive to open source is there for $200 million LLM models the same way it is for frameworks like React. And for closed source LLMs, I’ve yet to see any verifiable metrics that indicate that “productivity” increases are having any external impact—looking at new products released, new games on Steam, new startups founded etc… Certainly not enough to justify bearing the full cost of training and infrastructure.
- losvedir 10mo ago
  That can't come anywhere close to running the current SotA models, though.
  - Imustaskforhelp 10mo ago
    Most things don't require SotA models. But still, right now, you don't have to worry as even these SotA models are subsidized really so much and you can just use them for free on websites and then if you don't even want to type, go use a cheaper model or even a free model with something like opencode even to then act as a mini agent of things Usually I just end up it being more focused in a single file which isn't really the best practise but its usually for prototyping purposes anyway so it ends up being really good uv scripts are good for python, and I usually create golang single main.go files as well as I feel like it can be a binary, compile fast and cross compilation and still easy and simple so yeah :)
kenforthewin 10mo ago
I find the cost discussion to be exceedingly more tedious. This would be a more compelling line of thinking if we didn't have highly effective open-weight models like qwen3-coder, glm 4.7 etc. which allow us to directly measure the cost of running inference with large models without confounding factors like VC money. Regardless of the cost of training, the models that exist right now are cheap and effective enough to push the conversation right back to "quibbling about the exact degree of utility LLMs provide".
lelele 10mo ago
> One day when the well runs dry, you must be able to either pay the actual cost What multiple of the current cost do you expect? Currently, GitHub Copilot and ChatGPT for Business cost $19/month and €29/month respectively. Even a 10×–20× increase will still be economically viable in a professional setting if the tools continue to save hours of work.
- cl0ckt0wer 10mo ago
  Remember when Uber was cheap and showed you the surge pricing multiple?
- mock-possum 10mo ago
  Claude max is like $100/mo, and if you’re a daily user you’re likely going to need max
- automatic6131 10mo ago
  These tools (e.g. Chatgpt pro) lose money at $200/month So expect, maybe, $1000 a month? Until your business is dependant on these LLMs. Then they can extract basically all your margin lol
  - lelele 10mo ago
    At a $1,000/month price point, wouldn't the economics start favoring buying GPUs and running local LLMs? Even if they're weaker, local models can still cover enough use cases to justify the switch.
PTOB 10mo ago
As someone who works on the design and construction of datacenters, I cannot stress enough how apropos this comment is. Even before the first conversation in your IDE starts, the load on national and local government resources, local utility capacity, and roadway infrastructure is enormous. We're all paying whether we're using the tools or not.
- compiler-devel 10mo ago
  Nearly nobody cares about the load on “national and local government resources, local utility capacity, and roadway infrastructure” for any other day-to-day activity. Why should they care about the same for AI which for most people is “out there online” somewhere? Related my, crypto bros worried about electricity usage only so far as its expense went and whether they could move closer to hydro dams.
  - mcbishop 10mo ago
    The parent comment's point is that we _should_ care because cheap frontier-model access (that many of us have quickly become hopelessly dependent on) might be temporary.
    - Terretta 10mo ago
      If expert devs have junior devs to assign code to, that you review and integrate, do they become “hopelessly dependent” on junior devs? My experience of expert devs is those who are happy to have extra leverage are not slowed much by having to do it themselves. In no cases have I seen experts become “dependent” on the junior devs.
      - Ma8ee 10mo ago
        They do quite soon after they have become managers or product owners or “architects”.
        LtWorf 10mo ago
whazor 10mo ago
A competitive coding like devstral 2 runs fast enough to be very helpful: https://www.hardware-corner.net/devstral-2-hardware-requirements/ https://www.hardware-corner.net/devstral-2-hardware-requirem... The required hardware is fits the budget for a professional developer. Putting LLMs in the cloud allows the hardware to be utilised better and to have sharding of big models.
Mongoose 10mo ago
I always liken it to using Uber in ~2012. It was fun to get around major metro areas for dirt cheap. But then prices rose dramatically over the next decade+ as the company was forced to wean itself off of VC subsidies.
- dawnerd 10mo ago
  Same with Airbnb. Oh and Moviepass. Those were the days.
  - doxeddaily 10mo ago
    I watched a friend of mine minmax Moviepass so hard. They were so doomed.
  - XenophileJKO 10mo ago
    Except none of those cost structures are based primarily on a resource that gets cheaper over time.. a.k.a. compute.
    - coffeebeqn 10mo ago
      Training is getting exponentially more expensive. And inference isn’t that cheap unless you can do it locally
    - dawnerd 10mo ago
      Computer isn’t getting cheaper, growth right now is supply constrained to memory which if you haven’t seen the news recently…
    - Ma8ee 10mo ago
      The energy demand doesn’t decrease.
- mkozlows 10mo ago
  ... and people kept using Uber.
magic_hamster 10mo ago
There's a lot in this comment that doesn't exactly fit. First of all, there could be other solutions, such as B2B subsidizing individual user plans, or more fine grained model tiering per cost. Also, yes you can get some access for free, but even today the higher tiers of proprietary models is around $200/mo for individual users, which might still be subsidized but is definitely not free, and is quite a chunk of money at $2400 a year! I don't know what your setup is at the moment, but it's possible more efficient hardware and stack are available that you're not utilizing. Of course this depends on what models you're trying to run. I think that smaller models will become a lot better, and hardware will become more optimized as well. We're starting to see this with NPUs and TPUs. All this means running models will cost less, and maybe upgrading the power grid will also reduce cost of energy, making it more affordable. I don't see any way that AI will go away because it "hits a wall". We have long passed the point of no return.
- nosianu 10mo ago
  You are looking at it from the individual's PoV, but the OP is using the bird view from high above. It is the total amount of effort deployed today already to provide all the existing AI services, which is enormous. Data centers, electricity, planning/attention (entities focused on AI have less time to work on something else), components (Nvidia shortage, RAM shortage), etc. This is not about finance, but about the real economy and how much of it, and/or its growth, is diverted to AI. The real economy is being reshaped, influencing a lot of other sectors independent of AI use itself. AI heavily competes with other uses for many kinds of actual real resources - without having equally much to show for it yet. Just an example: https://www.technologyreview.com/2025/05/20/1116327/ai-energy-usage-climate-footprint-big-tech/ https://www.technologyreview.com/2025/05/20/1116327/ai-energ... (it is worth skip-reading it all, the headline on its own is useless)
anthonypasq 10mo ago
if you wrote this comment 70 years ago when computers were the size of rooms, it would make a lot of sense, and yet we know how history played out where everyone has a super computer in their pocket. for some reason it feels like people are under the assumption that hardware isnt going to improve or something?
Imustaskforhelp 10mo ago
Writing my comment on this post, I kind of feel like LLM's are like similar to wordpress/drag and drop tool although its more inconsistent too perhaps not sure I 100% share the codependent path too and had written a similar comment some 2-3 days ago but these AI companies which provide are either seriously negative/loss making subsidizing or they are barely net zero. I doubt that it will continue and so the bubble will burst I guess and prices of these will rise perhaps We will see perhaps something like google which can feed on advertising can perhaps still provide such subsidies for a longer time but the fact of the matter is that I have no alleigance to any model as some might have and I will simply shift to the cheapest thing which can still provide me / be enough for my queries or prototypes mostly I suppose.
robotnikman 10mo ago
>I run local LLMs infrequently, and every single prompt makes my beefy PC sounds like a jet engine taking off. It's a great reminder to not become codependent. I would try setting the GPU to run at a lower power level. I set my GPU power level to 80% and it becomes much quieter, and only runs maybe 5% slower at most. Also I 100% agree with the rest of your comment. We can only power the current growth we are seeing for so long.
akkad33 10mo ago
So what'll happen to all these companies building on top of openai license. I don't hear these warnings in professional circles, only online
12345hn6789 10mo ago
LLMs cannot generate coherent sentences LLMs writing prose is too robotic LLMs output is too dependent on prompts to be interesting LLMs take too much RAM to run effectively LLMs take too much electricity to run locally LLMs work locally but are a bit too slow for my taste LLMs output mostly correct code but it isn't applicable to my codebase LLMs make tool calls to pull in additional context LLMs outputted code works for most developers but not my codebase <---- you are currently here
- fuy 10mo ago
  isn't this template supposed to mean that all the previous considerations are now obsolete?
  - 12345hn6789 10mo ago
    I guess you could argue that the standard LLM sentence structure is too robotic but prompting mostly fixes that. The rest is no longer true, indeed
fnord77 10mo ago
The cost is coming down fast. You can get a $2000 desktop machine (AMD 395) that can run effectively chatGPT 3.5 levels of LLMs at over 100 tokens per second.

bogzz 10mo ago

I am sorry, but this kind of level-headed and realistic take is completely unacceptable on hackernews, and you should be ashamed of yourself. This is not a place for rational discussion when it comes to LLMs. LLMs are amazing and they will change everything, and then everything will be changed.

JeremyNT 10mo ago

This is a good point but you can see the price "ceiling" by examining the prices for PCs that can effectively run local models. A DGX Spark is ~$4k (plus power) for example. That's not nothing, but it's still not very much to pay compared to e.g. the cost of a FTE.

__float 10mo ago
You're counting the cost of running the model, but what about training it? You can't count the compute and data costs at $0.
- philipkglass 10mo ago
  You can assume that already-published open weights models are available at $0, regardless of how much money was sunk into their original development. These models will look increasingly stale over time but most software development doesn't change quickly. If a model can generate capable and up-to-date Python, C++, Java, or Javascript code in 2025 then you can expect it to still be a useful model in 2035 (based on the observation that then-modern code in these languages from 2015 works fine today, even if styles have shifted).
  - sarchertech 10mo ago
    >2025-2035 Depending on other people to maintain backward compatibility so that you can keep coding like it’s 2025 is its own problematic dependency. You could certainly do it but it would be limiting. Imagine that you had a model trained on examples from before 2013 and your boss wants you to take over maintenance for a React app.
    - nikkwong 10mo ago
      You're all referencing the strange idea in a world where there would be no open-weight coding models trained in the future. Even in a world where VC spending vanished completely, coding models are such a valuable utility that I'm sure at the very least companies/individuals would crowdsource them on a reoccurring basis, keeping them up to date. The value of this technology has been established, it's not leaving anytime soon.
      - sarchertech 10mo ago
        SOTA models cost hundreds of millions to train. I doubt anyone is crowdsourcing that. And that’s assuming you already have a lot of the infrastructure in place.
        nikkwong 10mo ago
        I think faang and the like would probably crowdsource it given that they would—according to the hypothesis presented—would only have to do it every few years, and ostensibly are realizing improved developer productivity from them.
        sarchertech 10mo ago
        I don’t think the incentive to open source is there for $200 million LLM models the same way it is for frameworks like React. And for closed source LLMs, I’ve yet to see any verifiable metrics that indicate that “productivity” increases are having any external impact—looking at new products released, new games on Steam, new startups founded etc… Certainly not enough to justify bearing the full cost of training and infrastructure.
losvedir 10mo ago
That can't come anywhere close to running the current SotA models, though.
- Imustaskforhelp 10mo ago
  Most things don't require SotA models. But still, right now, you don't have to worry as even these SotA models are subsidized really so much and you can just use them for free on websites and then if you don't even want to type, go use a cheaper model or even a free model with something like opencode even to then act as a mini agent of things Usually I just end up it being more focused in a single file which isn't really the best practise but its usually for prototyping purposes anyway so it ends up being really good uv scripts are good for python, and I usually create golang single main.go files as well as I feel like it can be a binary, compile fast and cross compilation and still easy and simple so yeah :)

kenforthewin 10mo ago

I find the cost discussion to be exceedingly more tedious. This would be a more compelling line of thinking if we didn't have highly effective open-weight models like qwen3-coder, glm 4.7 etc. which allow us to directly measure the cost of running inference with large models without confounding factors like VC money. Regardless of the cost of training, the models that exist right now are cheap and effective enough to push the conversation right back to "quibbling about the exact degree of utility LLMs provide".

lelele 10mo ago

> One day when the well runs dry, you must be able to either pay the actual cost What multiple of the current cost do you expect? Currently, GitHub Copilot and ChatGPT for Business cost $19/month and €29/month respectively. Even a 10×–20× increase will still be economically viable in a professional setting if the tools continue to save hours of work.

cl0ckt0wer 10mo ago
Remember when Uber was cheap and showed you the surge pricing multiple?
mock-possum 10mo ago
Claude max is like $100/mo, and if you’re a daily user you’re likely going to need max
automatic6131 10mo ago
These tools (e.g. Chatgpt pro) lose money at $200/month So expect, maybe, $1000 a month? Until your business is dependant on these LLMs. Then they can extract basically all your margin lol
- lelele 10mo ago
  At a $1,000/month price point, wouldn't the economics start favoring buying GPUs and running local LLMs? Even if they're weaker, local models can still cover enough use cases to justify the switch.

PTOB 10mo ago

As someone who works on the design and construction of datacenters, I cannot stress enough how apropos this comment is. Even before the first conversation in your IDE starts, the load on national and local government resources, local utility capacity, and roadway infrastructure is enormous. We're all paying whether we're using the tools or not.

compiler-devel 10mo ago
Nearly nobody cares about the load on “national and local government resources, local utility capacity, and roadway infrastructure” for any other day-to-day activity. Why should they care about the same for AI which for most people is “out there online” somewhere? Related my, crypto bros worried about electricity usage only so far as its expense went and whether they could move closer to hydro dams.
- mcbishop 10mo ago
  The parent comment's point is that we _should_ care because cheap frontier-model access (that many of us have quickly become hopelessly dependent on) might be temporary.
  - Terretta 10mo ago
    If expert devs have junior devs to assign code to, that you review and integrate, do they become “hopelessly dependent” on junior devs? My experience of expert devs is those who are happy to have extra leverage are not slowed much by having to do it themselves. In no cases have I seen experts become “dependent” on the junior devs.
    - Ma8ee 10mo ago
      They do quite soon after they have become managers or product owners or “architects”.
      - LtWorf 10mo ago

whazor 10mo ago

A competitive coding like devstral 2 runs fast enough to be very helpful: https://www.hardware-corner.net/devstral-2-hardware-requirements/ https://www.hardware-corner.net/devstral-2-hardware-requirem... The required hardware is fits the budget for a professional developer. Putting LLMs in the cloud allows the hardware to be utilised better and to have sharding of big models.

Mongoose 10mo ago

I always liken it to using Uber in ~2012. It was fun to get around major metro areas for dirt cheap. But then prices rose dramatically over the next decade+ as the company was forced to wean itself off of VC subsidies.

dawnerd 10mo ago
Same with Airbnb. Oh and Moviepass. Those were the days.
- doxeddaily 10mo ago
  I watched a friend of mine minmax Moviepass so hard. They were so doomed.
- XenophileJKO 10mo ago
  Except none of those cost structures are based primarily on a resource that gets cheaper over time.. a.k.a. compute.
  - coffeebeqn 10mo ago
    Training is getting exponentially more expensive. And inference isn’t that cheap unless you can do it locally
  - dawnerd 10mo ago
    Computer isn’t getting cheaper, growth right now is supply constrained to memory which if you haven’t seen the news recently…
  - Ma8ee 10mo ago
    The energy demand doesn’t decrease.
mkozlows 10mo ago
... and people kept using Uber.

magic_hamster 10mo ago

There's a lot in this comment that doesn't exactly fit. First of all, there could be other solutions, such as B2B subsidizing individual user plans, or more fine grained model tiering per cost. Also, yes you can get some access for free, but even today the higher tiers of proprietary models is around $200/mo for individual users, which might still be subsidized but is definitely not free, and is quite a chunk of money at $2400 a year! I don't know what your setup is at the moment, but it's possible more efficient hardware and stack are available that you're not utilizing. Of course this depends on what models you're trying to run. I think that smaller models will become a lot better, and hardware will become more optimized as well. We're starting to see this with NPUs and TPUs. All this means running models will cost less, and maybe upgrading the power grid will also reduce cost of energy, making it more affordable. I don't see any way that AI will go away because it "hits a wall". We have long passed the point of no return.

nosianu 10mo ago
You are looking at it from the individual's PoV, but the OP is using the bird view from high above. It is the total amount of effort deployed today already to provide all the existing AI services, which is enormous. Data centers, electricity, planning/attention (entities focused on AI have less time to work on something else), components (Nvidia shortage, RAM shortage), etc. This is not about finance, but about the real economy and how much of it, and/or its growth, is diverted to AI. The real economy is being reshaped, influencing a lot of other sectors independent of AI use itself. AI heavily competes with other uses for many kinds of actual real resources - without having equally much to show for it yet. Just an example: https://www.technologyreview.com/2025/05/20/1116327/ai-energy-usage-climate-footprint-big-tech/ https://www.technologyreview.com/2025/05/20/1116327/ai-energ... (it is worth skip-reading it all, the headline on its own is useless)

anthonypasq 10mo ago

if you wrote this comment 70 years ago when computers were the size of rooms, it would make a lot of sense, and yet we know how history played out where everyone has a super computer in their pocket. for some reason it feels like people are under the assumption that hardware isnt going to improve or something?

Imustaskforhelp 10mo ago

Writing my comment on this post, I kind of feel like LLM's are like similar to wordpress/drag and drop tool although its more inconsistent too perhaps not sure I 100% share the codependent path too and had written a similar comment some 2-3 days ago but these AI companies which provide are either seriously negative/loss making subsidizing or they are barely net zero. I doubt that it will continue and so the bubble will burst I guess and prices of these will rise perhaps We will see perhaps something like google which can feed on advertising can perhaps still provide such subsidies for a longer time but the fact of the matter is that I have no alleigance to any model as some might have and I will simply shift to the cheapest thing which can still provide me / be enough for my queries or prototypes mostly I suppose.

robotnikman 10mo ago

>I run local LLMs infrequently, and every single prompt makes my beefy PC sounds like a jet engine taking off. It's a great reminder to not become codependent. I would try setting the GPU to run at a lower power level. I set my GPU power level to 80% and it becomes much quieter, and only runs maybe 5% slower at most. Also I 100% agree with the rest of your comment. We can only power the current growth we are seeing for so long.

akkad33 10mo ago

So what'll happen to all these companies building on top of openai license. I don't hear these warnings in professional circles, only online

12345hn6789 10mo ago

LLMs cannot generate coherent sentences LLMs writing prose is too robotic LLMs output is too dependent on prompts to be interesting LLMs take too much RAM to run effectively LLMs take too much electricity to run locally LLMs work locally but are a bit too slow for my taste LLMs output mostly correct code but it isn't applicable to my codebase LLMs make tool calls to pull in additional context LLMs outputted code works for most developers but not my codebase <---- you are currently here

fuy 10mo ago
isn't this template supposed to mean that all the previous considerations are now obsolete?
- 12345hn6789 10mo ago
  I guess you could argue that the standard LLM sentence structure is too robotic but prompting mostly fixes that. The rest is no longer true, indeed

fnord77 10mo ago

The cost is coming down fast. You can get a $2000 desktop machine (AMD 395) that can run effectively chatGPT 3.5 levels of LLMs at over 100 tokens per second.