5 ms·

Something is changing in the unit economics of software

73 points by coconido 2mo ago

smalltorch 2mo ago
I think part of the new equation may also become; "Why even pay for the SaaS in the first place if you can just forge the service exactly how you want it?" The benefits of unlimited access to the tool you forge are still there, its just a lot easier to make whatever tool you want. Are there any examples of products containing ai inference that are successful? Products that are beyond just direct access to frontier LLM's, I mean.
- coconido 2mo ago
  [dead]
- esafak 2mo ago
  As if making all the decisions that go into that product, hosting it, and maintaining it are free! Now do that for all the SaaS you subscribe to. Would you even get any real work done?
  - cwmoore 2mo ago
    So many pivot opportunities, we’ll live remotely from office.
  - nostrademons 2mo ago
    There wouldn't be a product. Consumers would just use the underlying foundational models directly to solve their problem. ChatGPT and Claude and Gemini are the products, not your business. Products are only viable when you do a lot of work to solve a problem, which is then shared by many potential customers. It makes sense to amortize the high costs of solving the problem across all the different customers to reap economies of scale. Businesses then pay for product design, hosting, and maintenance because those costs can also be amortized, and they are cheaper than a bespoke solution for each customer. But if the bespoke solution becomes cheaper than that, because it's generated by an LLM that doesn't need to be paid a living wage, there's no reason to have the product in the first place. Just solve your damn problem and let other people solve theirs. This is an underrated factor in the market structure of the AI bubble going on now. Anecdotally, we're not seeing new AI-based products other than the foundational models and coding assistants gain traction. Why? Because AI makes it so easy to customize the solution to your particular needs that everybody is just solving their particular needs directly. It's the opposite of the Internet boom, where the network greatly expanded the potential market, reduced the cost of reaching them, and made it economical to spend large amounts of money building a software product that had a TAM of billions. The AI boom instead enables extremely cheap customization, which shrinks the market to a single customer who uses AI to directly solve their problem rather than building a product that's generally applicable.
    - esafak 2mo ago
      > Consumers would just use the underlying foundational models directly to solve their problem. ChatGPT and Claude and Gemini are the products, not your business. How am I going to use ChatGPT instead of, say, Figma or an office suite with my coworkers?? It's no use for it to spin up a clone for me if my coworkers can't collaborate. And if they can, what it will serve is a product. How can it be cheaper for every company to re-invent the wheel? > Products are only viable when you do a lot of work to solve a problem, which is then shared by many potential customers. Yes, that's what businesses pay services for.
      - DangitBobby 2mo ago
        > How can it be cheaper for every company to re-invent the wheel? Companies not matching their prices to current reality, mostly.
        esafak 2mo ago
        There are already free, open source clones of numerous things but companies still pay for commercial versions. Do you think LLMs that you have to pay for are going to change that?
      - nostrademons 2mo ago
        Actual example: my wife wanted a logo for the facepainting business she's starting with our 5yo, so she asked Gemini "I want a logo with these objects in it, drawn in these colors, with this business name." And it spits back a perfectly passable logo in 5 seconds. Done. No designer needed. No Figma needed. It cut out the entire value chain. Is it as good as a professional could've done with Photoshop? No. But it's about 90-95% of the way there, which is good enough that neither her nor any customers would care. A great deal of business is precisely this, tasks that need to be done but where you only care about "good enough" solutions. After all, a key principle of business is you don't outsource your competitive advantages. You pay people for the table stakes, the things that everybody needs but that they only need to be "good enough". LLMs can generate "good enough" facsimiles of a wide variety of fields.
  - samrus 2mo ago
    Hosting is cheap if its abstracted away. Same with the technical aspect of maintainance About design and the product side of maintainance, what i have felt is that, deep down inside, the user does know exactly what they need. If they can learn to communicate that in a way another person or agent can understand, then design and maintainance will be fine. Thats a big if, but if the economics align for people to benefit from developing that skill, maybe they do.
- anal_reactor 2mo ago
  If my manager prefers to pay for SaaS while I sit on my ass rather than have me develop an in-house solution, how do LLMs drastically change the equation?
- tonyedgecombe 2mo ago
  >"Why even pay for the SaaS in the first place if you can just forge the service exactly how you want it?" This is the part I'm least convinced by. Corporates used to develop their own tools and they moved away from it for many reasons. Cost of development was only one of those reasons.
euazOn 2mo ago
> Meeting that expectation means making LLM calls, and LLM calls cost money. Of course, and so does everything in the software world. The point is getting the cost so low that it’s basically free. The new DS V4 Flash or the smaller Qwen3.6 models are still really expensive compared to what we were used to in the economics of software, but it’s not unreasonable to expect these costs to continue falling down. Rough chatgpt estimate says 3-5 orders of magnitude of difference compared to a typical user interaction with a SPA (db/cache lookup, CDN…)
- throwaway27448 2mo ago
  > and so does everything in the software world. Well, no. Copying is free, or so near free it makes zero sense to charge. LLMs are just papering over the damage caused by profit.
  - euazOn 2mo ago
    Yeah, my point is that LLMs can reach that point too. Especially if you do it clientside. Copying was also way more expensive back in the 60s (accounting just for machine time and electricity, not storage cost, about 100 million times more expensive than today). Everything has a cost.
    - deleted 2mo ago
      [deleted]
    - throwaway27448 2mo ago
      LLMs will never be as cheap or reliable as copying.
      - euazOn 2mo ago
        Definitely, of course. But I think the logic of the author’s article is based on there being a huge difference between the two, or rather a high cost of inference in absolute terms. And that can change and we have seen it change. Which breaks the entire premise of the article going forward, no? Inference will be always more expensive than db operations or copying, sure. But how much more expensive is the question.
zmmmmm 2mo ago
I don't think it's at all certain this won't land back on the same unit economics as the old way. The cost of serving a user doesn't have to be free - it never has been - it just has to not be the dominating factor in your costs. I'm guessing there are still quite a lot of per-user costs that aren't easily visible. Like how many of your users are logging support requests, or suing you, or demanding bug fixes or custom integrations or a myriad of other things. And how much are you having to invest in security updates, regulatory compliance, marketing etc. Not to mention, users are getting well acclimatised to the idea of quotas and paying for increased limits.
SwellJoe 2mo ago
This is one reason I've been trying to figure out tasks (and products based on those tasks) that can be pushed to the edge, either via small specialized models or small general purpose open models. I suspect the same desire to keep unit costs low is part of why Google is falling behind on the "frontier", but seemingly at the lead, or near it, on models that run on-device. I think they're just focused on making models for tasks that don't require boiling the ocean. But, it's a hard problem. The models that run locally on normal computers/phones are pretty terrible compared to the frontier, without specialization and fine-tuning. And, even with specialization and fine-tuning, often a high-end general purpose model is going to do a better job and people don't need a bunch of local tools installed to do their various tasks.
- skinfaxi 2mo ago
  > And, even with specialization and fine-tuning, often a high-end general purpose model is going to do a better job and people don't need a bunch of local tools installed to do their various tasks. I think this is the critical point that would be interesting to see if it holds. Technology seemingly tends towards increased specialization.
  - antonvs 2mo ago
    > Technology seemingly tends towards increased specialization. The bitter lesson says the exact opposite.
    - ahartmetz 2mo ago
      The bitter lesson is rather specific and IMO vastly over-applied. It basically says that you can beat hand-tuning by waiting a few years and using "more of the same" generic method. It's not all that different from saying "Don't optimize software, just wait for faster hardware". Yet highly optimized software exist, and performance on currently available stacks is a competitive advantage. What gives? I say: Staying ahead of the "Don't optimize, just wait" curve can absolutely make sense. At worst, your advantage decays after a few years. At best, you stay ahead by n number of years and keep increasing the gap as you invest more.
chr15m 2mo ago
"Inference" is just software running. It has always cost money to run software, it's just that it is generally too cheap to matter. If a client makes a regular API call to your server, you pay for that compute, probably in the form of a flat hosting fee. If too many calls come in and workload goes up, you pay for a more expensive hosting tier to handle it (or do dynamic scaling which is per-unit of compute). Right now the "hosting" cost for inference is per-unit because it's new and expensive, but that won't last. There is a lot of inefficiency right now keeping prices elevated. That will change very fast and soon paying for inference will likely resemble paying for hosting your app. The bigger problem for SaaS is that the floor has risen - people can build their own solutions for things that they used to buy SaaS for. So the industry needs to level up and solve harder problems.
- euazOn 2mo ago
  Exactly my point in another comment. Just to illustrate this further: a rough ballpark of how the cost of intelligence fell since 2022 could be about 1000x, and continues to fall. Unfortunately, it’s really hard to measure. It’s so cheap that companies choose to spend more on AI inference (more reasoning, more capabilities, longer context), not less - see Jevons paradox.
- margalabargala 2mo ago
  > There is a lot of inefficiency right now keeping prices elevated. That will change very fast and soon paying for inference will likely resemble paying for hosting your app. I don't know about "very fast" or "soon" unless you're speaking in geological terms. SOTA models like Kimi 3 require thousands of GB of RAM/VRAM to run at speeds that are real-time useful. Manufacturing the memory necessary for that quantity to be available at app-hosting prices will take decades. Software efficiency solutions might drop needed memory by an order of magnitude in that time...but a tenth of an enormous amount is still pretty darn big so won't get us there "soon".
  - dalenw 2mo ago
    Speaking of soon, tangentially related this just got announced: https://standardcode.ai/ https://standardcode.ai/ They claim to support 24/7 agent coding with no VC subsidizations or money lost on a subscription, instead relying on optimizations on agent selection. I think it'll shift sooner rather than later.
Spooky23 2mo ago
It depends on the solution. If AI is generating value, you can charge for the value. Most SaaS already works this way. M365 or Adobe Creative Cloud are great examples. They value it like a life insurance policy and find ways to make you sticky. It’s easier to just buy it. The first round of AI products suck because they are not well defined. Copilot only makes sense if you do shit in office and SharePoint isn’t a dumpster fire. In my large O365 environment the bottom 50% of users use less storage than the top 2%. So why would i buy copilot for my janitor? When M365 E9 reconciles invoices automatically with Excel, I’ll pay $150/mo and fire a bunch of people.
carlosjobim 2mo ago
"Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond." Absolutely not. Customers want systems for sales, reservations, accounting, and taking stock. That's where almost all the SaaS money is and none of it benefits from AI - and never will.
horticulturist 2mo ago
Wonder how much this cost the author to write, as it’s just AI slop…
phendrenad2 2mo ago
Customers expect more, so they'll pay more. It really isn't more complicated than that.
roncesvalles 2mo ago
>Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond. Not really. >Every inference call costs money. Not really, either. If you buy your own GPU, rack it, and run an open model, there is no unit cost. This is just expensive hosting infra. You also pay unit costs for SaaS that your software uses (things like SMS etc).
- euazOn 2mo ago
  > If you buy your own GPU, rack it, and run an open model, there is no unit cost. No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.
  - nostrademons 2mo ago
    These are basically all fixed costs, not unit costs. You buy the GPU once and use it for as many calls as you have traffic for, and depreciate it over a fixed lifetime. You have to power it regardless of whether it's fully utilized or not. You have to maintain it by virtue of owning it, not really based on how many queries it has served. Bandwidth is the only one that really scales as a unit cost. Open question whether this model is actually more economical than using the cloud AI service. The whole reason the industry moved to cloud computing in the first place was because computing had very high fixed costs, and the more these could be amortized over a fully-loaded query stream, the lower the unit costs.
    - euazOn 2mo ago
      Good points. Speaking from experience, it’s really hard to make it more economical than using a cloud AI service - even if you utilize the GPU to its fullest. There are, of course, other benefits, such as privacy/control/compliance/security, which should be the real reasons to do this, not cost.
    - noosphr 2mo ago
Ozzie-D 2mo ago
[flagged]
- cleandreams 2mo ago
  I don't think the bottleneck was engineering time alone. There is context, brand, customer support, etc. I am not sure big complex products are particularly vulnerable in the way you state. However I have been thinking of a one smallish problem in a niche space that I could easily code for. There will be more of that.
mullingitover 2mo ago
If I'm any kind of indicator of where Youtube users are headed, their AI chatbots in the video pages are going to kill their business model. There are so many videos with hooks/teasers/'you won't believe what we discovered!!1', and now I just pause the video in the first second, ask "what's the tldr" and get the value from the video without a single ad impression (and likely racking up far more opex for Youtube than if I just streamed the video).
- jolmg 2mo ago
  Seems it's only available to users on YouTube Premium. I imagine most viewers are not.
jrm4 2mo ago
I strongly predict this article is mostly pointless very soon. Much as people may not want to like it, "software" as a product to buy and sell, even as a subscription, is probably going away, and will make about as much sense as "math" as a product. We were already headed in this direction, but AI's going to rapidly accelerate this.
matchagaucho 2mo ago
Fortunately, SaaS vendors have already conditioned users to accept usage limits within seat-based plans: “Upgrade to Pro for 50 GB of storage.” How do we make subscribers become equally comfortable paying for AI usage? Tokens, credits, inference calls?
gofreddygo 2mo ago
> Build the product once, distribute it to a million users for roughly the same cost as distributing it to one. Every incremental customer flowed largely to the bottom line. common misconception about software unit economics. With enterprise software (one that costs real $$$) there always more costs attached post shipping. Before client/server it was support. Then it was security and the constant threat of cyber attacks. Distributing software is nothing like distributing books.
WorldMaker 2mo ago
There's an interesting bellwether in mobile games right now. There are two easy statistics to track when trying random free-to-start games: time to first ad (TTFA), time to paywall (TTP). Most LLM-using mobile games are already at incredibly weak TTP scores and perhaps the only current stratum of mobile games where TTP is almost always ahead of TTFA. TTFA before TTP is almost nonexistent because they claim to need a monthly subscription as soon as immediately after install, despite being advertised as free to start. It's also one of the few types of games where the paywall explicitly does not include "no ads". Some of these games running on monthly subscriptions still need ads for unit costs. (The bulk of mobile games try for a sweet spot of TTFA in the order of hours of gameplay and TTP in the order of days of gameplay. Easier to get people hooked on your game if you can give them a few hours of uninterrupted fun up front.) It seems pretty condemning of software economics with LLMs involved.
alun 2mo ago
The author misses one potential future of AI software where people use use their own subscriptions / API keys for their AI usage. In this scenario the user would either sign into the platform via their Anthropic / OpenAI / Gemini account or use their API keys, and any of their usage would be billed to them directly. The company then doesn't have to worry about the increasing costs from the AI usage. Of course, in this future, AI providers become the new "Facebooks" of the world.
shirinlf 2mo ago
Interesting read

73 points by coconido 2mo ago

I think part of the new equation may also become; "Why even pay for the SaaS in the first place if you can just forge the service exactly how you want it?" The benefits of unlimited access to the tool you forge are still there, its just a lot easier to make whatever tool you want. Are there any examples of products containing ai inference that are successful? Products that are beyond just direct access to frontier LLM's, I mean.

coconido 2mo ago
[dead]
esafak 2mo ago
As if making all the decisions that go into that product, hosting it, and maintaining it are free! Now do that for all the SaaS you subscribe to. Would you even get any real work done?
- cwmoore 2mo ago
  So many pivot opportunities, we’ll live remotely from office.
- nostrademons 2mo ago
  There wouldn't be a product. Consumers would just use the underlying foundational models directly to solve their problem. ChatGPT and Claude and Gemini are the products, not your business. Products are only viable when you do a lot of work to solve a problem, which is then shared by many potential customers. It makes sense to amortize the high costs of solving the problem across all the different customers to reap economies of scale. Businesses then pay for product design, hosting, and maintenance because those costs can also be amortized, and they are cheaper than a bespoke solution for each customer. But if the bespoke solution becomes cheaper than that, because it's generated by an LLM that doesn't need to be paid a living wage, there's no reason to have the product in the first place. Just solve your damn problem and let other people solve theirs. This is an underrated factor in the market structure of the AI bubble going on now. Anecdotally, we're not seeing new AI-based products other than the foundational models and coding assistants gain traction. Why? Because AI makes it so easy to customize the solution to your particular needs that everybody is just solving their particular needs directly. It's the opposite of the Internet boom, where the network greatly expanded the potential market, reduced the cost of reaching them, and made it economical to spend large amounts of money building a software product that had a TAM of billions. The AI boom instead enables extremely cheap customization, which shrinks the market to a single customer who uses AI to directly solve their problem rather than building a product that's generally applicable.
  - esafak 2mo ago
    > Consumers would just use the underlying foundational models directly to solve their problem. ChatGPT and Claude and Gemini are the products, not your business. How am I going to use ChatGPT instead of, say, Figma or an office suite with my coworkers?? It's no use for it to spin up a clone for me if my coworkers can't collaborate. And if they can, what it will serve is a product. How can it be cheaper for every company to re-invent the wheel? > Products are only viable when you do a lot of work to solve a problem, which is then shared by many potential customers. Yes, that's what businesses pay services for.
    - DangitBobby 2mo ago
      > How can it be cheaper for every company to re-invent the wheel? Companies not matching their prices to current reality, mostly.
      - esafak 2mo ago
        There are already free, open source clones of numerous things but companies still pay for commercial versions. Do you think LLMs that you have to pay for are going to change that?
    - nostrademons 2mo ago
      Actual example: my wife wanted a logo for the facepainting business she's starting with our 5yo, so she asked Gemini "I want a logo with these objects in it, drawn in these colors, with this business name." And it spits back a perfectly passable logo in 5 seconds. Done. No designer needed. No Figma needed. It cut out the entire value chain. Is it as good as a professional could've done with Photoshop? No. But it's about 90-95% of the way there, which is good enough that neither her nor any customers would care. A great deal of business is precisely this, tasks that need to be done but where you only care about "good enough" solutions. After all, a key principle of business is you don't outsource your competitive advantages. You pay people for the table stakes, the things that everybody needs but that they only need to be "good enough". LLMs can generate "good enough" facsimiles of a wide variety of fields.
- samrus 2mo ago
  Hosting is cheap if its abstracted away. Same with the technical aspect of maintainance About design and the product side of maintainance, what i have felt is that, deep down inside, the user does know exactly what they need. If they can learn to communicate that in a way another person or agent can understand, then design and maintainance will be fine. Thats a big if, but if the economics align for people to benefit from developing that skill, maybe they do.
anal_reactor 2mo ago
If my manager prefers to pay for SaaS while I sit on my ass rather than have me develop an in-house solution, how do LLMs drastically change the equation?
tonyedgecombe 2mo ago
>"Why even pay for the SaaS in the first place if you can just forge the service exactly how you want it?" This is the part I'm least convinced by. Corporates used to develop their own tools and they moved away from it for many reasons. Cost of development was only one of those reasons.

euazOn 2mo ago

> Meeting that expectation means making LLM calls, and LLM calls cost money. Of course, and so does everything in the software world. The point is getting the cost so low that it’s basically free. The new DS V4 Flash or the smaller Qwen3.6 models are still really expensive compared to what we were used to in the economics of software, but it’s not unreasonable to expect these costs to continue falling down. Rough chatgpt estimate says 3-5 orders of magnitude of difference compared to a typical user interaction with a SPA (db/cache lookup, CDN…)

throwaway27448 2mo ago
> and so does everything in the software world. Well, no. Copying is free, or so near free it makes zero sense to charge. LLMs are just papering over the damage caused by profit.
- euazOn 2mo ago
  Yeah, my point is that LLMs can reach that point too. Especially if you do it clientside. Copying was also way more expensive back in the 60s (accounting just for machine time and electricity, not storage cost, about 100 million times more expensive than today). Everything has a cost.
  - deleted 2mo ago
    [deleted]
  - throwaway27448 2mo ago
    LLMs will never be as cheap or reliable as copying.
    - euazOn 2mo ago
      Definitely, of course. But I think the logic of the author’s article is based on there being a huge difference between the two, or rather a high cost of inference in absolute terms. And that can change and we have seen it change. Which breaks the entire premise of the article going forward, no? Inference will be always more expensive than db operations or copying, sure. But how much more expensive is the question.

zmmmmm 2mo ago

I don't think it's at all certain this won't land back on the same unit economics as the old way. The cost of serving a user doesn't have to be free - it never has been - it just has to not be the dominating factor in your costs. I'm guessing there are still quite a lot of per-user costs that aren't easily visible. Like how many of your users are logging support requests, or suing you, or demanding bug fixes or custom integrations or a myriad of other things. And how much are you having to invest in security updates, regulatory compliance, marketing etc. Not to mention, users are getting well acclimatised to the idea of quotas and paying for increased limits.

SwellJoe 2mo ago

This is one reason I've been trying to figure out tasks (and products based on those tasks) that can be pushed to the edge, either via small specialized models or small general purpose open models. I suspect the same desire to keep unit costs low is part of why Google is falling behind on the "frontier", but seemingly at the lead, or near it, on models that run on-device. I think they're just focused on making models for tasks that don't require boiling the ocean. But, it's a hard problem. The models that run locally on normal computers/phones are pretty terrible compared to the frontier, without specialization and fine-tuning. And, even with specialization and fine-tuning, often a high-end general purpose model is going to do a better job and people don't need a bunch of local tools installed to do their various tasks.

skinfaxi 2mo ago
> And, even with specialization and fine-tuning, often a high-end general purpose model is going to do a better job and people don't need a bunch of local tools installed to do their various tasks. I think this is the critical point that would be interesting to see if it holds. Technology seemingly tends towards increased specialization.
- antonvs 2mo ago
  > Technology seemingly tends towards increased specialization. The bitter lesson says the exact opposite.
  - ahartmetz 2mo ago
    The bitter lesson is rather specific and IMO vastly over-applied. It basically says that you can beat hand-tuning by waiting a few years and using "more of the same" generic method. It's not all that different from saying "Don't optimize software, just wait for faster hardware". Yet highly optimized software exist, and performance on currently available stacks is a competitive advantage. What gives? I say: Staying ahead of the "Don't optimize, just wait" curve can absolutely make sense. At worst, your advantage decays after a few years. At best, you stay ahead by n number of years and keep increasing the gap as you invest more.

chr15m 2mo ago

"Inference" is just software running. It has always cost money to run software, it's just that it is generally too cheap to matter. If a client makes a regular API call to your server, you pay for that compute, probably in the form of a flat hosting fee. If too many calls come in and workload goes up, you pay for a more expensive hosting tier to handle it (or do dynamic scaling which is per-unit of compute). Right now the "hosting" cost for inference is per-unit because it's new and expensive, but that won't last. There is a lot of inefficiency right now keeping prices elevated. That will change very fast and soon paying for inference will likely resemble paying for hosting your app. The bigger problem for SaaS is that the floor has risen - people can build their own solutions for things that they used to buy SaaS for. So the industry needs to level up and solve harder problems.

euazOn 2mo ago
Exactly my point in another comment. Just to illustrate this further: a rough ballpark of how the cost of intelligence fell since 2022 could be about 1000x, and continues to fall. Unfortunately, it’s really hard to measure. It’s so cheap that companies choose to spend more on AI inference (more reasoning, more capabilities, longer context), not less - see Jevons paradox.
margalabargala 2mo ago
> There is a lot of inefficiency right now keeping prices elevated. That will change very fast and soon paying for inference will likely resemble paying for hosting your app. I don't know about "very fast" or "soon" unless you're speaking in geological terms. SOTA models like Kimi 3 require thousands of GB of RAM/VRAM to run at speeds that are real-time useful. Manufacturing the memory necessary for that quantity to be available at app-hosting prices will take decades. Software efficiency solutions might drop needed memory by an order of magnitude in that time...but a tenth of an enormous amount is still pretty darn big so won't get us there "soon".
- dalenw 2mo ago
  Speaking of soon, tangentially related this just got announced: https://standardcode.ai/ https://standardcode.ai/ They claim to support 24/7 agent coding with no VC subsidizations or money lost on a subscription, instead relying on optimizations on agent selection. I think it'll shift sooner rather than later.

Spooky23 2mo ago

It depends on the solution. If AI is generating value, you can charge for the value. Most SaaS already works this way. M365 or Adobe Creative Cloud are great examples. They value it like a life insurance policy and find ways to make you sticky. It’s easier to just buy it. The first round of AI products suck because they are not well defined. Copilot only makes sense if you do shit in office and SharePoint isn’t a dumpster fire. In my large O365 environment the bottom 50% of users use less storage than the top 2%. So why would i buy copilot for my janitor? When M365 E9 reconciles invoices automatically with Excel, I’ll pay $150/mo and fire a bunch of people.

carlosjobim 2mo ago

"Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond." Absolutely not. Customers want systems for sales, reservations, accounting, and taking stock. That's where almost all the SaaS money is and none of it benefits from AI - and never will.

horticulturist 2mo ago

Wonder how much this cost the author to write, as it’s just AI slop…

phendrenad2 2mo ago

Customers expect more, so they'll pay more. It really isn't more complicated than that.

roncesvalles 2mo ago

>Users began expecting something fundamentally different from software: not just tools that store and retrieve, but products that reason, generate, and respond. Not really. >Every inference call costs money. Not really, either. If you buy your own GPU, rack it, and run an open model, there is no unit cost. This is just expensive hosting infra. You also pay unit costs for SaaS that your software uses (things like SMS etc).

euazOn 2mo ago
> If you buy your own GPU, rack it, and run an open model, there is no unit cost. No. There is economic opportunity cost (borrowing), energy cost, infra cost, depreciation / risk of failure with each unit of work, bandwidth, maintenance, and lots more. Small, but not zero, and often overlooked - especially the opportunity cost.
- nostrademons 2mo ago
  These are basically all fixed costs, not unit costs. You buy the GPU once and use it for as many calls as you have traffic for, and depreciate it over a fixed lifetime. You have to power it regardless of whether it's fully utilized or not. You have to maintain it by virtue of owning it, not really based on how many queries it has served. Bandwidth is the only one that really scales as a unit cost. Open question whether this model is actually more economical than using the cloud AI service. The whole reason the industry moved to cloud computing in the first place was because computing had very high fixed costs, and the more these could be amortized over a fully-loaded query stream, the lower the unit costs.
  - euazOn 2mo ago
    Good points. Speaking from experience, it’s really hard to make it more economical than using a cloud AI service - even if you utilize the GPU to its fullest. There are, of course, other benefits, such as privacy/control/compliance/security, which should be the real reasons to do this, not cost.
  - noosphr 2mo ago

Ozzie-D 2mo ago

[flagged]

cleandreams 2mo ago
I don't think the bottleneck was engineering time alone. There is context, brand, customer support, etc. I am not sure big complex products are particularly vulnerable in the way you state. However I have been thinking of a one smallish problem in a niche space that I could easily code for. There will be more of that.

mullingitover 2mo ago

If I'm any kind of indicator of where Youtube users are headed, their AI chatbots in the video pages are going to kill their business model. There are so many videos with hooks/teasers/'you won't believe what we discovered!!1', and now I just pause the video in the first second, ask "what's the tldr" and get the value from the video without a single ad impression (and likely racking up far more opex for Youtube than if I just streamed the video).

jolmg 2mo ago
Seems it's only available to users on YouTube Premium. I imagine most viewers are not.

jrm4 2mo ago

I strongly predict this article is mostly pointless very soon. Much as people may not want to like it, "software" as a product to buy and sell, even as a subscription, is probably going away, and will make about as much sense as "math" as a product. We were already headed in this direction, but AI's going to rapidly accelerate this.

matchagaucho 2mo ago

Fortunately, SaaS vendors have already conditioned users to accept usage limits within seat-based plans: “Upgrade to Pro for 50 GB of storage.” How do we make subscribers become equally comfortable paying for AI usage? Tokens, credits, inference calls?

gofreddygo 2mo ago

> Build the product once, distribute it to a million users for roughly the same cost as distributing it to one. Every incremental customer flowed largely to the bottom line. common misconception about software unit economics. With enterprise software (one that costs real $$$) there always more costs attached post shipping. Before client/server it was support. Then it was security and the constant threat of cyber attacks. Distributing software is nothing like distributing books.

WorldMaker 2mo ago

There's an interesting bellwether in mobile games right now. There are two easy statistics to track when trying random free-to-start games: time to first ad (TTFA), time to paywall (TTP). Most LLM-using mobile games are already at incredibly weak TTP scores and perhaps the only current stratum of mobile games where TTP is almost always ahead of TTFA. TTFA before TTP is almost nonexistent because they claim to need a monthly subscription as soon as immediately after install, despite being advertised as free to start. It's also one of the few types of games where the paywall explicitly does not include "no ads". Some of these games running on monthly subscriptions still need ads for unit costs. (The bulk of mobile games try for a sweet spot of TTFA in the order of hours of gameplay and TTP in the order of days of gameplay. Easier to get people hooked on your game if you can give them a few hours of uninterrupted fun up front.) It seems pretty condemning of software economics with LLMs involved.

alun 2mo ago

The author misses one potential future of AI software where people use use their own subscriptions / API keys for their AI usage. In this scenario the user would either sign into the platform via their Anthropic / OpenAI / Gemini account or use their API keys, and any of their usage would be billed to them directly. The company then doesn't have to worry about the increasing costs from the AI usage. Of course, in this future, AI providers become the new "Facebooks" of the world.

shirinlf 2mo ago

Interesting read