3 ms·

> I also ended up blowing through $15 of LLM tokens in a single evening. This is a feature, not a bug. LLMs are going to be the next "OMG my AWS bill" phenome

> I also ended up blowing through $15 of LLM tokens in a single evening. This is a feature, not a bug. LLMs are going to be the next "OMG my AWS bill" phenomenon.

Scene_Cast2 1y ago
Cline very visibly displays the ongoing cost of the task. Light edits are about 10 cents, and heavy stuff can run a couple of bucks. It's just that the tab accumulates faster than I expect.
- PretzelPirate 1y ago
  > Cline very visibly displays the ongoing cost of the task LLMs are now being positioned as "let them work autonomously in the background" which means no one will be watching the cost in real time. Perhaps I can set limits on how much money each task is worth, but very few would estimate that properly.
  - Aurornis 1y ago
    > LLMs are now being positioned as "let them work autonomously in the background" The only people who believe this level of AI marketing are the people who haven't yet used the tools. > which means no one will be watching the cost in real time. Maybe some day there's an agentic coding tool that goes off into the weeds and runs for days doing meaningless tasks until someone catches it and does a Ctrl-C, but the tools I've used are more likely to stop short of the goal than to continue crunching indefinitely. Regardless, it seems like a common experience for first-timers to try a light task and then realize they've spent $3, instantly setting expectations for how easy it is to run up a large bill if you're not careful.
- eterm 1y ago
  > Light edits are about 10 cents Some well-paid developers will excuse this with, "Well if it saved me 5 minutes, it's worth an order of magnitude than 10 cents". Which is true, however there's a big caveat: Time saved isn't time gained. You can "Save" 1,000 hours every night, but you don't actuall get those 1,000 hours back.
  - grepfru_it 1y ago
    Hourly_rate / 12 = 5min_rate If light_edit_cost < 5min_rate then savings=true
    - spicypete 1y ago
      (from a companies perspective, this is true). As a developer, you may not be paid by the task -- If I finish something early, I start work on the next thing.
  - shepherdjerred 1y ago
    > You can "Save" 1,000 hours every night, but you don't actuall get those 1,000 hours back. What do you mean? If I have some task that requires 1000 hours, and I'm able to shave it down to one hour, then I did just "save" 999 hours -- just in the same way that if something costs $5 and I pay $4, I saved $
    - eterm 1y ago
      My point is that saving 1,000 hours each day doesn't actually give you 1,000 hours a day to do things with. You still get your 24 hours, no matter how much time you save. What actually matters is the value of what is delivered, not how much time it actually saves you. Justifying costs by "time saved" is a good way to eat up your money on time-saving devices.
      - leptons 1y ago
        If I "save 1000 hours" then that could be distributed over 41.666 days, so no task would need to be performed during that period because "I saved 1000 hours". You could also say you saved 41.666 people an entire 24 hour day, by "saving 1000 hours", or some other fractional way. How you're trying to explain it as "saving 1000 hours each day" is really not making any sense without further context. And I'm sure if I hadn't written this comment I would be saving 1000 hours on a stupid comment thread.
        1y ago
philkuz 1y ago
I think that models are gonna commoditize, if they haven't already. The cost of switching over is rather small, especially when you have good evals on what you want done. Also there's no way you can build a business without providing value in this space. Buyers are not that dumb.
- raincole 1y ago
  They are already quite commoditized. Commoditization doesn't mean "cheap", and it doesn't mean you won't spend $15 a night like the GP did.
Cthulhu_ 1y ago
Especially at companies (hence this github one), where the employees don't care about cost because it's the boss' credit card.

> I also ended up blowing through $15 of LLM tokens in a single evening. This is a feature, not a bug. LLMs are going to be the next "OMG my AWS bill" phenomenon.

Scene_Cast2 1y ago

Cline very visibly displays the ongoing cost of the task. Light edits are about 10 cents, and heavy stuff can run a couple of bucks. It's just that the tab accumulates faster than I expect.

PretzelPirate 1y ago
> Cline very visibly displays the ongoing cost of the task LLMs are now being positioned as "let them work autonomously in the background" which means no one will be watching the cost in real time. Perhaps I can set limits on how much money each task is worth, but very few would estimate that properly.
- Aurornis 1y ago
  > LLMs are now being positioned as "let them work autonomously in the background" The only people who believe this level of AI marketing are the people who haven't yet used the tools. > which means no one will be watching the cost in real time. Maybe some day there's an agentic coding tool that goes off into the weeds and runs for days doing meaningless tasks until someone catches it and does a Ctrl-C, but the tools I've used are more likely to stop short of the goal than to continue crunching indefinitely. Regardless, it seems like a common experience for first-timers to try a light task and then realize they've spent $3, instantly setting expectations for how easy it is to run up a large bill if you're not careful.
eterm 1y ago
> Light edits are about 10 cents Some well-paid developers will excuse this with, "Well if it saved me 5 minutes, it's worth an order of magnitude than 10 cents". Which is true, however there's a big caveat: Time saved isn't time gained. You can "Save" 1,000 hours every night, but you don't actuall get those 1,000 hours back.
- grepfru_it 1y ago
  Hourly_rate / 12 = 5min_rate If light_edit_cost < 5min_rate then savings=true
  - spicypete 1y ago
    (from a companies perspective, this is true). As a developer, you may not be paid by the task -- If I finish something early, I start work on the next thing.
- shepherdjerred 1y ago
  > You can "Save" 1,000 hours every night, but you don't actuall get those 1,000 hours back. What do you mean? If I have some task that requires 1000 hours, and I'm able to shave it down to one hour, then I did just "save" 999 hours -- just in the same way that if something costs $5 and I pay $4, I saved $
  - eterm 1y ago
    My point is that saving 1,000 hours each day doesn't actually give you 1,000 hours a day to do things with. You still get your 24 hours, no matter how much time you save. What actually matters is the value of what is delivered, not how much time it actually saves you. Justifying costs by "time saved" is a good way to eat up your money on time-saving devices.
    - leptons 1y ago
      If I "save 1000 hours" then that could be distributed over 41.666 days, so no task would need to be performed during that period because "I saved 1000 hours". You could also say you saved 41.666 people an entire 24 hour day, by "saving 1000 hours", or some other fractional way. How you're trying to explain it as "saving 1000 hours each day" is really not making any sense without further context. And I'm sure if I hadn't written this comment I would be saving 1000 hours on a stupid comment thread.
      - 1y ago

philkuz 1y ago

I think that models are gonna commoditize, if they haven't already. The cost of switching over is rather small, especially when you have good evals on what you want done. Also there's no way you can build a business without providing value in this space. Buyers are not that dumb.

raincole 1y ago
They are already quite commoditized. Commoditization doesn't mean "cheap", and it doesn't mean you won't spend $15 a night like the GP did.

Cthulhu_ 1y ago

Especially at companies (hence this github one), where the employees don't care about cost because it's the boss' credit card.