4 ms·

Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or eve

Rationing tokens on a spend level or count makes no sense to me. Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? I’m not an AI-maximalist, but “work a few days with AI and the rest of the month without because of cost” sounds literally crazy to me. (If you think AI is low/zero/negative net value, don’t do the first half; if it has net value, don’t do the second half.)

loki-ai 2mo ago
this not making any sense to you is exactly why companies are implementing these caps: you lack the business acumen to understand how financial resources should actually be deployed.
- iugtmkbdfil834 2mo ago
  Please consider rewriting this in a way that does not seem to unnecessarily attack parent. You are not wrong about the incentives, but you may not be right about the execution of the restrictions. I should know. I am living through it.
  - loki-ai 2mo ago
    you're fully correct and I apologize for my tone. similarly to many here, I got hit by the caps and frustrated by how many times coworkers were unintentionally or not burning through these resources. Claiming you have no idea why the managers had to do this felt to me like avoiding any responsibility
    - sokoloff 2mo ago
      I am the manager in this case and the one who sets the AI policy for my low-8-figure/year product development teams. Our policy is quite permissive for exactly the reasons I conveyed above. It takes an extremely finely-tuned sense of value and high assurance that you’re on the diminishing return portion of the curve to conclude “SWEs should have this precisely metered amount (rather than zero or ‘as much as they don’t waste’)” My claiming to have no idea was not an abdication of responsibility but rather a claim that it was likely an error (assuming a non-trivial team size and a business that can continue to grow).
Avicebron 2mo ago
I think in this future they don't think a software engineer is making that much, maybe a nice 50k a year, 65K after 10+ years on the job
- iugtmkbdfil834 2mo ago
  I honestly don't know how 100k is not effectively the new 40k from late 90s atm in US. 50K is barely enough to do anything.
  - sarchertech 2mo ago
    I mean median household income in 1999 was $40k and median household income now is $84k.
- sarchertech 2mo ago
  Still doesn’t make sense if you believe that AI is making a meaningful boost to productivity. If you are the kind of person who thinks AI will drive down the wages of software engineers that much, you have to think that it is at least a 2x multiplier. $65k salary all in for a company is easily $100k. So spending anything less than $100k would be what you would aim for.
abuani 2mo ago
What about a $10k to $20k bill? Or better, 100%-200% of the employees salary? At a certain point in time, finance is going to care greatly about the ability to predict costs. Pre AI, predicting the cost an engineer was quite easy. Salary + benefits + licenses for software. CI systems that were unbounded were rolled up into opex and treated as a separate line item. Where does token spend land in this now? Finance teams above all else value predictability. Token spend is the opposite of predictable. Eventually these two forces are going to collide and some part of the system will break. My guess is it's token spend, not finance allowing unbound spend on a previously predictable part of the balance sheet.
- gibolt 2mo ago
  Set a limit, train people to use tokens efficiently. For anyone getting high value from it, raise their limit.
  - delusional 2mo ago
    How do you figure out they are "getting high value from it" What if they feel like they are getting value from it, but are actually not?
    - bot403 2mo ago
      Assumedly between the dev and their boss they can make the case they are getting value from it and how much. That's part of the basic job description of both of them. Explain how your work relates to your company's goals both at a large scale and at a localized team level.
izacus 2mo ago
> Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? No idea, but companies refusing to buy a 2000$ laptop once every 3 years, a new mouse, keyboard, or screen for < 500$ are very common, more common than the other type. So I'd expect those to not suddenly change into "let's waste money" companies.
verbify 2mo ago
> Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? Spending $10k and then balking at an additional $2k can make sense to me based on the circumstances. It's an additional 20% that hasn't been budgeted for. The question is partly about what that 20% gives the employer - for some businesses, you might see diminishing returns from additional productivity of a software developer. E.g. I did some work creating BI reports and warehousing for a manufacturing company. Being more productive just meant faster reports or more reports. But faster/more reports didn't necessarily equate to more money for the business. Sometimes it's a question of a business being penny wise and pound foolish, but sometimes it does make sense to me to not spend this additional money.
delusional 2mo ago
If I had to pay $10k salary for a certain job previously, but I now have to pay $10k + $2k for the same job. Why would I not care about that? You were doing your job perfectly well previously, why do you want more budget?
krab 2mo ago
Because there are diminishing returns somewhere for sure. Will the developer be more effective when he gets $2,000 more in tokens? I can't max the $100 subscription. The same for any other employee-related expense. It's a no-brainer to give the developer a solid laptop. But if you'd pay extra $2,000/month in computer and office supplies costs, will it save the employee a few hours? I don't think so. Their salary doesn't matter that much.
cfiggers 2mo ago
This sounds roughly similar to, "If eating steak is nutritious and tastes good, why would you not eat an 8oz sirloin for breakfast, lunch, and dinner for an entire month?" "Too much of a good thing" does exist. Someone might feel that it takes time for their organization to "metabolize" the rapid additions made by coding agents. That seems quite reasonable to me. Your comment seems to argue a false dichotomy.
veeti 2mo ago
Forward thinking employers should use these limits with a holistic perspective. It's obviously time to take a deserved coffee break if you've token maxxed too much. Maybe take Friday off, AI now allows us to accomplish in four working days what used to take five.

loki-ai 2mo ago

this not making any sense to you is exactly why companies are implementing these caps: you lack the business acumen to understand how financial resources should actually be deployed.

iugtmkbdfil834 2mo ago
Please consider rewriting this in a way that does not seem to unnecessarily attack parent. You are not wrong about the incentives, but you may not be right about the execution of the restrictions. I should know. I am living through it.
- loki-ai 2mo ago
  you're fully correct and I apologize for my tone. similarly to many here, I got hit by the caps and frustrated by how many times coworkers were unintentionally or not burning through these resources. Claiming you have no idea why the managers had to do this felt to me like avoiding any responsibility
  - sokoloff 2mo ago
    I am the manager in this case and the one who sets the AI policy for my low-8-figure/year product development teams. Our policy is quite permissive for exactly the reasons I conveyed above. It takes an extremely finely-tuned sense of value and high assurance that you’re on the diminishing return portion of the curve to conclude “SWEs should have this precisely metered amount (rather than zero or ‘as much as they don’t waste’)” My claiming to have no idea was not an abdication of responsibility but rather a claim that it was likely an error (assuming a non-trivial team size and a business that can continue to grow).

Avicebron 2mo ago

I think in this future they don't think a software engineer is making that much, maybe a nice 50k a year, 65K after 10+ years on the job

iugtmkbdfil834 2mo ago
I honestly don't know how 100k is not effectively the new 40k from late 90s atm in US. 50K is barely enough to do anything.
- sarchertech 2mo ago
  I mean median household income in 1999 was $40k and median household income now is $84k.
sarchertech 2mo ago
Still doesn’t make sense if you believe that AI is making a meaningful boost to productivity. If you are the kind of person who thinks AI will drive down the wages of software engineers that much, you have to think that it is at least a 2x multiplier. $65k salary all in for a company is easily $100k. So spending anything less than $100k would be what you would aim for.

abuani 2mo ago

What about a $10k to $20k bill? Or better, 100%-200% of the employees salary? At a certain point in time, finance is going to care greatly about the ability to predict costs. Pre AI, predicting the cost an engineer was quite easy. Salary + benefits + licenses for software. CI systems that were unbounded were rolled up into opex and treated as a separate line item. Where does token spend land in this now? Finance teams above all else value predictability. Token spend is the opposite of predictable. Eventually these two forces are going to collide and some part of the system will break. My guess is it's token spend, not finance allowing unbound spend on a previously predictable part of the balance sheet.

gibolt 2mo ago
Set a limit, train people to use tokens efficiently. For anyone getting high value from it, raise their limit.
- delusional 2mo ago
  How do you figure out they are "getting high value from it" What if they feel like they are getting value from it, but are actually not?
  - bot403 2mo ago
    Assumedly between the dev and their boss they can make the case they are getting value from it and how much. That's part of the basic job description of both of them. Explain how your work relates to your company's goals both at a large scale and at a localized team level.

izacus 2mo ago

> Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? No idea, but companies refusing to buy a 2000$ laptop once every 3 years, a new mouse, keyboard, or screen for < 500$ are very common, more common than the other type. So I'd expect those to not suddenly change into "let's waste money" companies.

verbify 2mo ago

> Why would I pay $10-20K/mo to employ a software engineer and then balk at a $500, $1000, or even $2000 monthly AI bill, assuming they were even vaguely trying to use the tokens productively? Spending $10k and then balking at an additional $2k can make sense to me based on the circumstances. It's an additional 20% that hasn't been budgeted for. The question is partly about what that 20% gives the employer - for some businesses, you might see diminishing returns from additional productivity of a software developer. E.g. I did some work creating BI reports and warehousing for a manufacturing company. Being more productive just meant faster reports or more reports. But faster/more reports didn't necessarily equate to more money for the business. Sometimes it's a question of a business being penny wise and pound foolish, but sometimes it does make sense to me to not spend this additional money.

delusional 2mo ago

If I had to pay $10k salary for a certain job previously, but I now have to pay $10k + $2k for the same job. Why would I not care about that? You were doing your job perfectly well previously, why do you want more budget?

krab 2mo ago

Because there are diminishing returns somewhere for sure. Will the developer be more effective when he gets $2,000 more in tokens? I can't max the $100 subscription. The same for any other employee-related expense. It's a no-brainer to give the developer a solid laptop. But if you'd pay extra $2,000/month in computer and office supplies costs, will it save the employee a few hours? I don't think so. Their salary doesn't matter that much.

cfiggers 2mo ago

This sounds roughly similar to, "If eating steak is nutritious and tastes good, why would you not eat an 8oz sirloin for breakfast, lunch, and dinner for an entire month?" "Too much of a good thing" does exist. Someone might feel that it takes time for their organization to "metabolize" the rapid additions made by coding agents. That seems quite reasonable to me. Your comment seems to argue a false dichotomy.

veeti 2mo ago

Forward thinking employers should use these limits with a holistic perspective. It's obviously time to take a deserved coffee break if you've token maxxed too much. Maybe take Friday off, AI now allows us to accomplish in four working days what used to take five.