3 ms·

I know I'm responding to AI right now, but > which means figuring out if the company can afford this level of productivity at scale. If it was actually produc

I know I'm responding to AI right now, but > which means figuring out if the company can afford this level of productivity at scale. If it was actually productive, then the revenue would increase and affordability wouldn't be a question.

sonofhans 5mo ago
Yes, my thoughts exactly. Productivity by definition creates things, hopefully valuable things. Is all the extra burn on chatbots worth the cost? Has Uber somehow gotten dramatically more efficient and effective due to this massive budget overrun? Or have they just given people shiny and expensive ways to push the same work around?
orf 5mo ago
Not every change a developer makes increases revenue, and the changes that do often have a lag time.
- guywithahat 5mo ago
  This is my thought too. The eggheads in accounting set budgets, and we produce products within that budget. I could be twice as productive with twice as many people, and maybe 50% more productive with good AI, but if it's not budgeted for it's an issue (especially short-term before the product is released).
- fg137 5mo ago
  I'd argue it's often the contrary -- since it's easy to ship features and fixes, people often ship things without questioning if it makes business sense to support a use case, or if the design is solid. Now you have exactly the same revenge but more things to maintain
  - fragmede 5mo ago
    What if you're the SRE and the code fixes mean the site goes from 99% uptime to 99.9% up? How do you measure the revenue from that?
    - linkregister 5mo ago
      Depending if the site has a direct competitor and non-sticky customers, you can often get accurate loss estimates from outages. For example, friends of mine at Doordash would know when UberEats was down by the corresponding spike in traffic to their app. The competitor captures all the lost traffic. Most enterprises will have a harder time quantifying losses, as some percentage of customers will come back later. To understand that, you need to look for a drop in completed purchase rates compared to site visits. For a SaaS, it's even more difficult, as customers are often held captive by long contracts and might tolerate SLA breaches up to a certain point. A reasonable, though fictional, proxy would be the revenue for the contract pro-rated against the uptime during that period.
      - fragmede 5mo ago
        Seems like an unscrupulous operator would take action to take down their competitor's site with a DDoS in order to drive business to themselves.
    - kevin_nisbet 5mo ago
      On this side of the equation I think you start pulling in customer context and risk analysis on the downside. What is the churn risk for operation at 99% vs 99.9% availability. If your site is for B2B and impacts customers own operations or revenue, you'll likely be wanting to chase the 99.9%, customers won't tolerate the 1.5 hours per week of downtime and will churn. However, if the value you're site creates is tolerant to those sorts of disruptions, someone is just inconvenienced and can come back later, a large investment to move from 99% to 99.9% wouldn't be justified. There is literally no impact from the investment. The harder part will be the reality, most investments will be somewhere in the middle with ambiguity on the impact. IIRC, SRE principles do talk about this when setting SLOs in different terms. I've heard some companies refer to the concept as economical thinking, which is I think a great way to think about it. Doesn't mean you'll always get it right, more so that we embed being conscious about the ROI in our work. I also believe this is an area that I've observed several engineers really struggle with, especially when moving from big tech to startups, where it's really easy to import culture from another company, and in earlier stages of startup life... if you don't have product-market-fit, it doesn't matter how good you're availability is. Attention is a resource, make sure it's allocated to what creates value for the customer.
solenoid0937 5mo ago
> If it was actually productive, then the revenue would increase and affordability wouldn't be a question. Revenue has increased. Have you seen Meta's latest earnings? +33% revenue - in this economy. Affordability is not a question. There is a reason companies like Meta have no issue with their engineers spending $1k/day on tokens. It's just not that much compared to how much they make per employee.
- louiereederson 5mo ago
  This article is about Uber, not Meta
- joshuastuden 5mo ago
  How can that be attributed to any code an LLM wrote? >$8 billion of net income was the result of a tax benefit the company realized in the first quarter of the year. So exactly how much of their revenue is because of any code LLMs wrote vs. just structural tail winds?
  - solenoid0937 5mo ago
    You can always say "it's not because of LLMs", that's nearly unfalsifiable. But if all of your peers are saying LLMs are more productive, if you're building things faster than ever before, the macro picture speaks for itself.
    - jmcqk6 5mo ago
      It sounds like this has a pretty falsifiable claim here - is the revenue attributed to a tax thing? Then it's clearly not attributable to code. I agree that the macro picture would speak for itself. Can you point to any macro level detail that is indeed cleanly showing benefits from increased productivity from LLMs?
      - solenoid0937 5mo ago
        Not a macro detail, but my peers and I are shipping features at at least 5-10x the speed we used to.
leptons 5mo ago
That is not true at all. No matter how "productive" a company is means nothing if people aren't buying your product. And using LLMs to be more productive will not convince anyone to buy your product. Human creativity and intuition to make a product that people want to use is what sells. Productivity for productivity's sake doesn't really move the needle at all, and can make things worse.
nothrabannosir 5mo ago
Steelmanning the other side: a counter example would be if competitors use the same tools to achieve the same productivity gains.
alecco 5mo ago
> If it was actually productive They are extremely productive if you use them right. To the point it worries me how clever these pseudo-AI models can get in the next year.
deleted 5mo ago
[deleted]

sonofhans 5mo ago

Yes, my thoughts exactly. Productivity by definition creates things, hopefully valuable things. Is all the extra burn on chatbots worth the cost? Has Uber somehow gotten dramatically more efficient and effective due to this massive budget overrun? Or have they just given people shiny and expensive ways to push the same work around?

orf 5mo ago

Not every change a developer makes increases revenue, and the changes that do often have a lag time.

guywithahat 5mo ago
This is my thought too. The eggheads in accounting set budgets, and we produce products within that budget. I could be twice as productive with twice as many people, and maybe 50% more productive with good AI, but if it's not budgeted for it's an issue (especially short-term before the product is released).
fg137 5mo ago
I'd argue it's often the contrary -- since it's easy to ship features and fixes, people often ship things without questioning if it makes business sense to support a use case, or if the design is solid. Now you have exactly the same revenge but more things to maintain
- fragmede 5mo ago
  What if you're the SRE and the code fixes mean the site goes from 99% uptime to 99.9% up? How do you measure the revenue from that?
  - linkregister 5mo ago
    Depending if the site has a direct competitor and non-sticky customers, you can often get accurate loss estimates from outages. For example, friends of mine at Doordash would know when UberEats was down by the corresponding spike in traffic to their app. The competitor captures all the lost traffic. Most enterprises will have a harder time quantifying losses, as some percentage of customers will come back later. To understand that, you need to look for a drop in completed purchase rates compared to site visits. For a SaaS, it's even more difficult, as customers are often held captive by long contracts and might tolerate SLA breaches up to a certain point. A reasonable, though fictional, proxy would be the revenue for the contract pro-rated against the uptime during that period.
    - fragmede 5mo ago
      Seems like an unscrupulous operator would take action to take down their competitor's site with a DDoS in order to drive business to themselves.
  - kevin_nisbet 5mo ago
    On this side of the equation I think you start pulling in customer context and risk analysis on the downside. What is the churn risk for operation at 99% vs 99.9% availability. If your site is for B2B and impacts customers own operations or revenue, you'll likely be wanting to chase the 99.9%, customers won't tolerate the 1.5 hours per week of downtime and will churn. However, if the value you're site creates is tolerant to those sorts of disruptions, someone is just inconvenienced and can come back later, a large investment to move from 99% to 99.9% wouldn't be justified. There is literally no impact from the investment. The harder part will be the reality, most investments will be somewhere in the middle with ambiguity on the impact. IIRC, SRE principles do talk about this when setting SLOs in different terms. I've heard some companies refer to the concept as economical thinking, which is I think a great way to think about it. Doesn't mean you'll always get it right, more so that we embed being conscious about the ROI in our work. I also believe this is an area that I've observed several engineers really struggle with, especially when moving from big tech to startups, where it's really easy to import culture from another company, and in earlier stages of startup life... if you don't have product-market-fit, it doesn't matter how good you're availability is. Attention is a resource, make sure it's allocated to what creates value for the customer.

solenoid0937 5mo ago

> If it was actually productive, then the revenue would increase and affordability wouldn't be a question. Revenue has increased. Have you seen Meta's latest earnings? +33% revenue - in this economy. Affordability is not a question. There is a reason companies like Meta have no issue with their engineers spending $1k/day on tokens. It's just not that much compared to how much they make per employee.

louiereederson 5mo ago
This article is about Uber, not Meta
joshuastuden 5mo ago
How can that be attributed to any code an LLM wrote? >$8 billion of net income was the result of a tax benefit the company realized in the first quarter of the year. So exactly how much of their revenue is because of any code LLMs wrote vs. just structural tail winds?
- solenoid0937 5mo ago
  You can always say "it's not because of LLMs", that's nearly unfalsifiable. But if all of your peers are saying LLMs are more productive, if you're building things faster than ever before, the macro picture speaks for itself.
  - jmcqk6 5mo ago
    It sounds like this has a pretty falsifiable claim here - is the revenue attributed to a tax thing? Then it's clearly not attributable to code. I agree that the macro picture would speak for itself. Can you point to any macro level detail that is indeed cleanly showing benefits from increased productivity from LLMs?
    - solenoid0937 5mo ago
      Not a macro detail, but my peers and I are shipping features at at least 5-10x the speed we used to.

leptons 5mo ago

That is not true at all. No matter how "productive" a company is means nothing if people aren't buying your product. And using LLMs to be more productive will not convince anyone to buy your product. Human creativity and intuition to make a product that people want to use is what sells. Productivity for productivity's sake doesn't really move the needle at all, and can make things worse.

nothrabannosir 5mo ago

Steelmanning the other side: a counter example would be if competitors use the same tools to achieve the same productivity gains.

alecco 5mo ago

> If it was actually productive They are extremely productive if you use them right. To the point it worries me how clever these pseudo-AI models can get in the next year.

deleted 5mo ago

[deleted]