4 ms·

The “Better Offline” podcast has been warning that this was coming for months now. It’s incredible to me that AI evangelists have managed to focus the entire co

The “Better Offline” podcast has been warning that this was coming for months now. It’s incredible to me that AI evangelists have managed to focus the entire conversation on LLM “usefulness” without even mentioning costs. It allows them to play a game of smoke and mirrors where no one actually knows the true cost of an API call and there is no predictable cost model. Most AI companies can be safely assumed to be running at a loss and if they can’t figure out how to make these models a lot more efficient, dreams of replacing software engineers will face the same reality as dreams of self driving semi trucks. Im looking forward to the AI winter myself… https://m.youtube.com/@BetterOfflinePod/videos https://m.youtube.com/@BetterOfflinePod/videos

moduspol 1y ago
I agree, though honestly at this point I'm a little more cynical. I'm open to seeing the best that anyone can get these things to do regardless of cost.
- deleted 1y ago
  [deleted]
jauntywundrkind 1y ago
I both love Better Offline, but also very much see it's faults. Ed's angle is sharp and pointy, which is lovely, but having to take it as entertainment rather than informed journalism is a bit hard. His anti-AI basis is both rewarding, but also not well informed, is trying to go as hard as possible. Honestly I liked some of the non-AI content a lot more (but AI seems to be more of a focus lately). He also had such an amazing run of fantastic guests: its nonsense to say this, but he's running through the list of awesome people to talk to fast, and I hope he's not afraid to invite many of these fantastic people back again!
- luma 1y ago
  Ed Zitron is a PR man who runs a PR company who found widespread traction when he started posting AI shittakes on twitter. He now claims to be a tech guy but has zero tech background. His entire schtick is saying smarmy things that gets clicks, and with AI skepticism he's found a very wide audience. If he's right about something in this field, it's by accident and not a result of experience or research. He's a social media creation and one should consider his spicy takes in that light.
  - ath3nd 1y ago
    > If he's right about something in this field, it's by accident and not a result of experience or research. I could say the same thing about Elon Musk. /s In all honesty though, there has to be some counterbalance to the faceless mass of AI enthusiast fanboys. In Ed's articles he is rightfully pointing out that these companies are burning a lot of cash and are extremely unprofitable at this very moment. Seeing that OpenAI's recent announcements were an office suite, a "study" mode, and ChatGPT4.001 while also claiming that AGI is just behind the corner, I think he smells bullshit, and I, for one am happy that he is calling it out. Adam Neumann from WeWork was also a visionary genius, and look how it turned out. The current AI diehards might have the habit of outsourcing some critical thinking to LLMs, but even they should be able to spot the red flags: unprofitabiliy with no clear profit window in sight, questionable launches, quick to squeeze the user base, no moat, and plateauing performance. My money is on Ed on this one.
    - tptacek 1y ago
      Not that I want to start this particular kudzu subthread but: lots and lots of people say exactly that about Elon Musk.
    - fragmede 1y ago
      Charging people to access their system is the obvious route to profitability, the only question is under what time window. Uber shows that money is able to be patient. There will be losers and there will be winners, but discounting the entire sector seems illogical.
      - ath3nd 1y ago
        > Charging people to access their system is the obvious route to profitability Ed argues, and I agree with him, that there is simply no route to profitablility that makes sense numbers wise. Considering this is the only sector propping up the otherwise very bleak US economy currently, if there are indeed losers, the consequences would far surpass the 2008 financial crisis. And it will spread. Closing our eyes to the numbers, especially if they are that bad, and relying on vibes and promises of leaders that failed to deliver again and again (Musk's literally every promise recently, to Zuck's metaverse and superintelligence, to sama's ChatGPT recent releases and false promises of AGI). I know investors often invest based on vibes and hype (see Softbank and WeWork) but not paying attention to the numbers and hoping the balance sheets magically resolve themselves somehow has reached such a predominance that it starts resembling a cult.
- mvdtnz 1y ago
  Ed Zitron is a professional cynic. You don't need to listen to anything he says because it's easy enough to predict. Whatever it is, he won't like it. Boring.
mns 1y ago
Reading the article and the linked Github post, as well as the original pricing announcement and the clarification post afterwards, this whole thing seems like some sort of Monty Python sketch. I can't believe that an actual enterprise targeted product comes up with something like: > "AWS now defines two types of Kiro AI request. Spec requests are those started from tasks, while vibe requests are general chat responses. Executing a sub-task consumes at least one spec request plus a vibe request for "coordination"". I still don't understand why pricing can't be as simple as it was initially and presented in a clear and understandable way: token cost this much, you used this many tokens, and that is it. Probably because if people would see how much they actually consume for real tasks, they would realize that the "vibes" cost more than an actual developer.
- ranie93 1y ago
  Just give me dollar amounts, I feel like I'm paying these companies with vbucks at this point
  - sshine 1y ago
    With open-weights models reaching a level where they can sufficiently be used for agentic coding, the price can be directly compared to the price of GPU rentals: https://vast.ai/ https://vast.ai/ has an H100 at $1.65/hr. which can support ~40 concurrent sessions at 40 tok/s. Depending on your agentic workload, you can stretch that any way you like, but let's say it might support 10 active developers at a speed comparable to Claude Code with Sonnet 4 (which I've read is 90 tok/s) who aren't going crazy with sub-agents. Let's scale that up: $1.65/hr. is ~$1188/mo. (assuming you're renting the GPU 24/7 which you can speculate to not do, which is probably fine as long as there's not scarcity), and divided by ten is ~$119/mo. per user for 10 users. Add the service layer on top in order to make this a convenient software service, I think $100/mo. is a bargain for unhindered (as far as it goes) access to a high-quality (as far as they come) agentic coding framework. Feel free to correct my napkin math, it's done very quickly.
- kermatt 1y ago
  Clear pricing makes it easy for you to control costs. Vibe pricing makes it easy for the vendor to maximize revenue. They have a little incentive to make pricing transparent.
motorest 1y ago
> It’s incredible to me that AI evangelists have managed to focus the entire conversation on LLM “usefulness” without even mentioning costs. LLMs are highly useful. AI-assisted services are in general costly. There are LLMs you can run locally on your own hardware. You pay AI-assisted services to use their computational resources.
torben-friis 1y ago
I think it’s clear that these tools are going to get more expensive. What is not clear to me is that they’ll get expensive enough to not be worth it for a company. A good engineer costs a lot of money and comes with limitations derived from being human. Let’s say AI manages to be a 2x multiplier in productivity. Prices for that multiplier can rise a lot before they reach 120k/year for 40 hours a week, the point at which you’re better off hiring someone.
- beefnugs 1y ago
  Too bad companies love to smash that authoritarian hammer down as quick as possible, and try to dictate everything based on baseless bullshit from other ceos at the country club. Instead of just making sure their employees have access to the tools and generously encourage them to explore their usefulness
- ethbr1 1y ago
  There are 3 functional futures for AI coding assistants, depending on what they provide, and all of which support order of magnitude different pricing: 1. Vibe coder's wet dream = generates working technical product direct from product owner requirements = can be priced up to 99% of senior developer compensation (incl benefits / taxes) 2. Productivity multiplier = fills in gaps, but requires technical architecture and planning knowledge = can be priced up to 99% of junior developer compensation (incl benefits / taxes) 3. Macro on steroids = ignorantly expands code or performs basic tasks, but screws things up frequently enough, both obviously and subtly, to still benefit from even a junior dev driving it = can be priced up to 99% of team size reduction Technically, even if we're only in #3 (and I think there's a strong case to be made that we're there), these solutions are drastically underpriced according to human labor decreasing (e.g. axing the equivalent FTE hours it would have taken to write test coverage). * And don't forget the number they're replacing from the company perspective isn't {salary} but {salary+benefits+taxes}, which can be substantially greater
  - torben-friis 1y ago
    Exactly. Pretty much our only hope is that these tools get cheap enough to provide that competition drives the price down - ideally a locally run open source solution that gets you a significant part of the way there. And don’t forget that in your case one, they can pull the supermarket trick - keep prices low until the industry is destroyed, then price back up to a higher point than the crushed competition used to demand.
mvieira38 1y ago
GPT-5 released just a few weeks ago and it's drastically cheaper than similar models. I think efficiency might not be that big of an issue long-term
- theappsecguy 1y ago
  They could be simply under cutting the market, there’s no guarantee that it’s actually that much cheaper to operate for OpenAI
  - grim_io 1y ago
    They wouldn't be so eager to remove access to their other models if 5 were more costly to run.
    - therobots927 1y ago
      You’re operating under the assumption that they can deploy multiple models at scale with no additional overhead. I wouldn’t be so sure of that…
      - grim_io 1y ago
        Why did they keep hosting old models before gpt5?
        therobots927 1y ago
        Maybe they spent so much money making got-5 they can’t afford to run older models anymore

moduspol 1y ago

I agree, though honestly at this point I'm a little more cynical. I'm open to seeing the best that anyone can get these things to do regardless of cost.

deleted 1y ago
[deleted]

jauntywundrkind 1y ago

I both love Better Offline, but also very much see it's faults. Ed's angle is sharp and pointy, which is lovely, but having to take it as entertainment rather than informed journalism is a bit hard. His anti-AI basis is both rewarding, but also not well informed, is trying to go as hard as possible. Honestly I liked some of the non-AI content a lot more (but AI seems to be more of a focus lately). He also had such an amazing run of fantastic guests: its nonsense to say this, but he's running through the list of awesome people to talk to fast, and I hope he's not afraid to invite many of these fantastic people back again!

luma 1y ago
Ed Zitron is a PR man who runs a PR company who found widespread traction when he started posting AI shittakes on twitter. He now claims to be a tech guy but has zero tech background. His entire schtick is saying smarmy things that gets clicks, and with AI skepticism he's found a very wide audience. If he's right about something in this field, it's by accident and not a result of experience or research. He's a social media creation and one should consider his spicy takes in that light.
- ath3nd 1y ago
  > If he's right about something in this field, it's by accident and not a result of experience or research. I could say the same thing about Elon Musk. /s In all honesty though, there has to be some counterbalance to the faceless mass of AI enthusiast fanboys. In Ed's articles he is rightfully pointing out that these companies are burning a lot of cash and are extremely unprofitable at this very moment. Seeing that OpenAI's recent announcements were an office suite, a "study" mode, and ChatGPT4.001 while also claiming that AGI is just behind the corner, I think he smells bullshit, and I, for one am happy that he is calling it out. Adam Neumann from WeWork was also a visionary genius, and look how it turned out. The current AI diehards might have the habit of outsourcing some critical thinking to LLMs, but even they should be able to spot the red flags: unprofitabiliy with no clear profit window in sight, questionable launches, quick to squeeze the user base, no moat, and plateauing performance. My money is on Ed on this one.
  - tptacek 1y ago
    Not that I want to start this particular kudzu subthread but: lots and lots of people say exactly that about Elon Musk.
  - fragmede 1y ago
    Charging people to access their system is the obvious route to profitability, the only question is under what time window. Uber shows that money is able to be patient. There will be losers and there will be winners, but discounting the entire sector seems illogical.
    - ath3nd 1y ago
      > Charging people to access their system is the obvious route to profitability Ed argues, and I agree with him, that there is simply no route to profitablility that makes sense numbers wise. Considering this is the only sector propping up the otherwise very bleak US economy currently, if there are indeed losers, the consequences would far surpass the 2008 financial crisis. And it will spread. Closing our eyes to the numbers, especially if they are that bad, and relying on vibes and promises of leaders that failed to deliver again and again (Musk's literally every promise recently, to Zuck's metaverse and superintelligence, to sama's ChatGPT recent releases and false promises of AGI). I know investors often invest based on vibes and hype (see Softbank and WeWork) but not paying attention to the numbers and hoping the balance sheets magically resolve themselves somehow has reached such a predominance that it starts resembling a cult.
mvdtnz 1y ago
Ed Zitron is a professional cynic. You don't need to listen to anything he says because it's easy enough to predict. Whatever it is, he won't like it. Boring.

mns 1y ago

Reading the article and the linked Github post, as well as the original pricing announcement and the clarification post afterwards, this whole thing seems like some sort of Monty Python sketch. I can't believe that an actual enterprise targeted product comes up with something like: > "AWS now defines two types of Kiro AI request. Spec requests are those started from tasks, while vibe requests are general chat responses. Executing a sub-task consumes at least one spec request plus a vibe request for "coordination"". I still don't understand why pricing can't be as simple as it was initially and presented in a clear and understandable way: token cost this much, you used this many tokens, and that is it. Probably because if people would see how much they actually consume for real tasks, they would realize that the "vibes" cost more than an actual developer.

ranie93 1y ago
Just give me dollar amounts, I feel like I'm paying these companies with vbucks at this point
- sshine 1y ago
  With open-weights models reaching a level where they can sufficiently be used for agentic coding, the price can be directly compared to the price of GPU rentals: https://vast.ai/ https://vast.ai/ has an H100 at $1.65/hr. which can support ~40 concurrent sessions at 40 tok/s. Depending on your agentic workload, you can stretch that any way you like, but let's say it might support 10 active developers at a speed comparable to Claude Code with Sonnet 4 (which I've read is 90 tok/s) who aren't going crazy with sub-agents. Let's scale that up: $1.65/hr. is ~$1188/mo. (assuming you're renting the GPU 24/7 which you can speculate to not do, which is probably fine as long as there's not scarcity), and divided by ten is ~$119/mo. per user for 10 users. Add the service layer on top in order to make this a convenient software service, I think $100/mo. is a bargain for unhindered (as far as it goes) access to a high-quality (as far as they come) agentic coding framework. Feel free to correct my napkin math, it's done very quickly.
kermatt 1y ago
Clear pricing makes it easy for you to control costs. Vibe pricing makes it easy for the vendor to maximize revenue. They have a little incentive to make pricing transparent.

motorest 1y ago

> It’s incredible to me that AI evangelists have managed to focus the entire conversation on LLM “usefulness” without even mentioning costs. LLMs are highly useful. AI-assisted services are in general costly. There are LLMs you can run locally on your own hardware. You pay AI-assisted services to use their computational resources.

torben-friis 1y ago

I think it’s clear that these tools are going to get more expensive. What is not clear to me is that they’ll get expensive enough to not be worth it for a company. A good engineer costs a lot of money and comes with limitations derived from being human. Let’s say AI manages to be a 2x multiplier in productivity. Prices for that multiplier can rise a lot before they reach 120k/year for 40 hours a week, the point at which you’re better off hiring someone.

beefnugs 1y ago
Too bad companies love to smash that authoritarian hammer down as quick as possible, and try to dictate everything based on baseless bullshit from other ceos at the country club. Instead of just making sure their employees have access to the tools and generously encourage them to explore their usefulness
ethbr1 1y ago
There are 3 functional futures for AI coding assistants, depending on what they provide, and all of which support order of magnitude different pricing: 1. Vibe coder's wet dream = generates working technical product direct from product owner requirements = can be priced up to 99% of senior developer compensation (incl benefits / taxes) 2. Productivity multiplier = fills in gaps, but requires technical architecture and planning knowledge = can be priced up to 99% of junior developer compensation (incl benefits / taxes) 3. Macro on steroids = ignorantly expands code or performs basic tasks, but screws things up frequently enough, both obviously and subtly, to still benefit from even a junior dev driving it = can be priced up to 99% of team size reduction Technically, even if we're only in #3 (and I think there's a strong case to be made that we're there), these solutions are drastically underpriced according to human labor decreasing (e.g. axing the equivalent FTE hours it would have taken to write test coverage). * And don't forget the number they're replacing from the company perspective isn't {salary} but {salary+benefits+taxes}, which can be substantially greater
- torben-friis 1y ago
  Exactly. Pretty much our only hope is that these tools get cheap enough to provide that competition drives the price down - ideally a locally run open source solution that gets you a significant part of the way there. And don’t forget that in your case one, they can pull the supermarket trick - keep prices low until the industry is destroyed, then price back up to a higher point than the crushed competition used to demand.

mvieira38 1y ago

GPT-5 released just a few weeks ago and it's drastically cheaper than similar models. I think efficiency might not be that big of an issue long-term

theappsecguy 1y ago
They could be simply under cutting the market, there’s no guarantee that it’s actually that much cheaper to operate for OpenAI
- grim_io 1y ago
  They wouldn't be so eager to remove access to their other models if 5 were more costly to run.
  - therobots927 1y ago
    You’re operating under the assumption that they can deploy multiple models at scale with no additional overhead. I wouldn’t be so sure of that…
    - grim_io 1y ago
      Why did they keep hosting old models before gpt5?
      - therobots927 1y ago
        Maybe they spent so much money making got-5 they can’t afford to run older models anymore