4 ms·

I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many

by cebert 2mo ago

I have similar concerns about this. The past year or so, software engineers have been encouraged by employers to adopt AI tooling and agentic coding. Now, many enterprises, including mine, are starting to crack down on token spend. I’ve learned how to fully embrace agentic tooling to do tasks like keeping up with vulnerability reports, initially triaging defects that come in, etc. It would be hard for me to do things the old way at this point when I know tools are available that could make me more productive for a particular category of tasks. My employer is considered capping all engineers at $200 or $500/mo of token spend depending on level. I regularly spend over $1k/mo today, but believe I can make a strong business justification for the value those tokens are creating. At this point, I think engineers may be asking what token budgets are when considering new roles.

alainrk 2mo ago
Indeed, but I also think structural considerations must be done at the company level on the work model overall
theChris-in 2mo ago
I think eventually we will converge on a new AI + old AI (ILP, ASP) way of working.
neaden 2mo ago
It's also kind of odd because even a junior engineer has to cost a company around 10 grand a month in salary, taxes, and other costs. So if the company thinks it makes you 10% more effective it seems like it should be an easy choice. So either the companies are shooting themselves in the foot limiting spending or they aren't seeing the productivity boost.
- andriy_koval 2mo ago
  there is likely curve of diminishing productivity return
- pdhborges 2mo ago
  10 grand a month for a junior is absolutely ridiculous in most of the world including large swats of the western world.
  - neaden 2mo ago
    Between all the benefits, taxes, and the infrastructure the company needs costs rise pretty quickly.
    - izacus 2mo ago
      No, they really don't, it's a ridiculous claim.
  - jdrek1 2mo ago
    They were talking about what it costs the company, not just the salary. There are taxes, insurances, equipment costs, etc. 10k might be a bit too high, but it's far from "absolutely ridiculous" amounts of being too high. An employee with a 5k salary can easily cost the employer 7-8k
    - pdhborges 2mo ago
      I took into account all of that. Still ridiculous.
      - cebert 2mo ago
        In the US, it’s really difficult to live a middle class lifestyle if you’re not making at least low to mid 100k. I don’t see any starting software salaries at good companies below 100k even in the Midwest.
        pdhborges 2mo ago
        Sure in the US. But people are throwing token caps in 500, 1000, 2000 dollar range here. That is a very significant expense on top of salary, taxes and benefis. It's only justifiable if they observe 20% 30% or more productivity increases across the board. If you have huge salaries, yeah sure 500 dollars its okay even if you only get 5 10% more productive.
the__alchemist 2mo ago
Maybe a dumb question, but what are you doing/seeing others do that uses that many tokens? I find myself pressing the weekly caps on the $20/month plan only when I'm really having the LLMs go wild with the Xtra high effort on the newest/biggest models on abstract problems or where I don't really understand the problem well. (E.g. improving ML models, planning creative molecule synthesis pipelines, identifying subtle performance problems etc) Generic CRUD/Config stuff I do for work (and assume most software jobs entail? Maybe not correct) doesn't use a significant amount of tokens.
- unified101 2mo ago
  The retail account is about 200 to 300 dollars of api equivalent. Large companies disallow retail.
  - the__alchemist 2mo ago
    Ah! That may explain the entirety of my disconnect.
- goolz 2mo ago
  At least for myself personally: Opus 5 Ultra spreading out hundreds of subagents on specific tasks can gobble tokens like mad. Recently I did a custom OCR pipeline in a day and some Revit automation, extremely large datasets, but I definitely spent good money to make it happen so fast. I almost certainly wasted tokens, but right now things are subsidized, so it works.
- ygjb 2mo ago
  I routinely hit the limits on a Claude pro max 20 account for personal projects. For context I have 26 years of professional experience and have been programming for around 40 years. In addition I have virtually unconstrained usage at work (for now). Tasks that I do on my personal account: - building stuff for my wife's business - analyzing and experimentation and building harnesses to test other models/services - processing interesting research papers that publish without code or sufficient data to replicate the work And that's besides simple stuff like exploring new topics I am interested in, and building learning assistants and personal tooling, and eliminating technical chores.
monkeydust 2mo ago
On business side orgs are now absorbing costs of tokens to their own local budgets...so the calculus soon will be:1 extra engineer or extra token spend for existing ones. Tbh, when the productivity payoff becomes more obvious I would happily take the latter.

by cebert 2mo ago

alainrk 2mo ago

Indeed, but I also think structural considerations must be done at the company level on the work model overall

theChris-in 2mo ago

I think eventually we will converge on a new AI + old AI (ILP, ASP) way of working.

neaden 2mo ago

It's also kind of odd because even a junior engineer has to cost a company around 10 grand a month in salary, taxes, and other costs. So if the company thinks it makes you 10% more effective it seems like it should be an easy choice. So either the companies are shooting themselves in the foot limiting spending or they aren't seeing the productivity boost.

andriy_koval 2mo ago
there is likely curve of diminishing productivity return
pdhborges 2mo ago
10 grand a month for a junior is absolutely ridiculous in most of the world including large swats of the western world.
- neaden 2mo ago
  Between all the benefits, taxes, and the infrastructure the company needs costs rise pretty quickly.
  - izacus 2mo ago
    No, they really don't, it's a ridiculous claim.
- jdrek1 2mo ago
  They were talking about what it costs the company, not just the salary. There are taxes, insurances, equipment costs, etc. 10k might be a bit too high, but it's far from "absolutely ridiculous" amounts of being too high. An employee with a 5k salary can easily cost the employer 7-8k
  - pdhborges 2mo ago
    I took into account all of that. Still ridiculous.
    - cebert 2mo ago
      In the US, it’s really difficult to live a middle class lifestyle if you’re not making at least low to mid 100k. I don’t see any starting software salaries at good companies below 100k even in the Midwest.
      - pdhborges 2mo ago
        Sure in the US. But people are throwing token caps in 500, 1000, 2000 dollar range here. That is a very significant expense on top of salary, taxes and benefis. It's only justifiable if they observe 20% 30% or more productivity increases across the board. If you have huge salaries, yeah sure 500 dollars its okay even if you only get 5 10% more productive.

the__alchemist 2mo ago

Maybe a dumb question, but what are you doing/seeing others do that uses that many tokens? I find myself pressing the weekly caps on the $20/month plan only when I'm really having the LLMs go wild with the Xtra high effort on the newest/biggest models on abstract problems or where I don't really understand the problem well. (E.g. improving ML models, planning creative molecule synthesis pipelines, identifying subtle performance problems etc) Generic CRUD/Config stuff I do for work (and assume most software jobs entail? Maybe not correct) doesn't use a significant amount of tokens.

unified101 2mo ago
The retail account is about 200 to 300 dollars of api equivalent. Large companies disallow retail.
- the__alchemist 2mo ago
  Ah! That may explain the entirety of my disconnect.
goolz 2mo ago
At least for myself personally: Opus 5 Ultra spreading out hundreds of subagents on specific tasks can gobble tokens like mad. Recently I did a custom OCR pipeline in a day and some Revit automation, extremely large datasets, but I definitely spent good money to make it happen so fast. I almost certainly wasted tokens, but right now things are subsidized, so it works.
ygjb 2mo ago
I routinely hit the limits on a Claude pro max 20 account for personal projects. For context I have 26 years of professional experience and have been programming for around 40 years. In addition I have virtually unconstrained usage at work (for now). Tasks that I do on my personal account: - building stuff for my wife's business - analyzing and experimentation and building harnesses to test other models/services - processing interesting research papers that publish without code or sufficient data to replicate the work And that's besides simple stuff like exploring new topics I am interested in, and building learning assistants and personal tooling, and eliminating technical chores.

monkeydust 2mo ago

On business side orgs are now absorbing costs of tokens to their own local budgets...so the calculus soon will be:1 extra engineer or extra token spend for existing ones. Tbh, when the productivity payoff becomes more obvious I would happily take the latter.