4 ms·

A friend of mine added some pretty extensive iOS UI tests to a keystone feature hit by millions every month. They'd been kicking the can down the road for years

A friend of mine added some pretty extensive iOS UI tests to a keystone feature hit by millions every month. They'd been kicking the can down the road for years, trying to fit it in their roadmap, and with Claude running overnight they were able to bang out the whole suite in a week. I'm not sure how it would show up in quarterly results.

dotancohen 5mo ago
> I'm not sure how it would show up in quarterly results. Technical debt is famously difficult to express in either layman's terms or financial terms.
- PearlRiver 5mo ago
  Everything can be expressed in financial terms- it is one of the guiding principles of the universe. Anyone who thinks they are above or beyond it will rue the day.
  - dotancohen 5mo ago
    I hope that you never loose a child and remain so naive.
  - ironmagma 5mo ago
    Yes but that representation can be over-applied and misleading. For example you can roughly estimate the price of the entire earth. What does that number mean though? Well, nothing, really.
smelendez 5mo ago
I see these kinds of stories here a lot, and I'm curious whether they reflect a steady stream of need for AI coding, or whether a lot of companies have a burst of AI-appropriate coding work now that the technology is available and then will have a smaller need going forward. Is it like the stereotypical dad who rents a power washer, powerwashes every exposed surface on his property, and then doesn't need to do any powerwashing for a few years; his neighbor who gets an Instant Pot and uses it for every meal for a month, then sees it gathering dust when the family gets tired of pressure-cooked stews; or like their neighbor who gets a microwave oven and uses it multiple times a day for decades? I guess only time will tell.
- JSR_FDED 5mo ago
  That’s been my theory - there’s some low hanging fruit in every environment where AI knocks it out of the park. Then complex brownfield reality (coupled with non-technical factors) rears its head and the stunning productivity gains are nowhere near to be seen. That’s the explanation how you can have both the anecdotes of amazing AI productivity and rigorous studies showing anything from actual loss of productivity to single-digit gains.
  - joenot443 5mo ago
    I think this is directionally right. The code AI produces is not created equally, not even close.
  - blensor 5mo ago
    I also think in addition to that the increased speed compounds the problems much quicker. And I don't mean bugs. I mean that duplicated code here, that additional state variable to keep everything going there. Not removing things that should be removed because we can work around that, etc. It's like building a super tall Jenga tower very quickly but laying the bricks much worse than a careful player.
- thewebguyd 5mo ago
  So far where I work its the Instant Pot, at least for the non-devs. We rolled out Claude & Cowork to the masses after a brief pilot. It was about a solid month and a half of heavy usage and then suddenly usage fell off a cliff. Once it stopped being a cool new toy, people just didn't find a use for it. A few mundane things got automated, but these were just back office admin type work. Nothing that's going to show on the P&L. Yeah those people now have a little more time for other things, but those other things are also not revenue generating. No FTE got replaced by it so in the end they just paid for a bunch of administrative positions to be a little less busy. Great for the workers who are now less stressed, but almost no impact on the business financials except there's now yet another subscription.
  - palmotea 5mo ago
rtkwe 5mo ago
In my limited experience with using agents to create tests it tends to code the tests to the existing code instead of ensuring the correctness from a spec. Great for regression testing but still limited in effectiveness for catching existing issues.
levkk 5mo ago
It wouldn't, at least not directly. That's why it wasn't done pre-AI.
ElFitz 5mo ago
Over here our CTO replaced Intercom with an internal equivalent that costs less than $20 / month to run, haiku and sonnet support agent costs included. In less than a few weeks, in his spare time.

dotancohen 5mo ago

> I'm not sure how it would show up in quarterly results. Technical debt is famously difficult to express in either layman's terms or financial terms.

PearlRiver 5mo ago
Everything can be expressed in financial terms- it is one of the guiding principles of the universe. Anyone who thinks they are above or beyond it will rue the day.
- dotancohen 5mo ago
  I hope that you never loose a child and remain so naive.
- ironmagma 5mo ago
  Yes but that representation can be over-applied and misleading. For example you can roughly estimate the price of the entire earth. What does that number mean though? Well, nothing, really.

smelendez 5mo ago

I see these kinds of stories here a lot, and I'm curious whether they reflect a steady stream of need for AI coding, or whether a lot of companies have a burst of AI-appropriate coding work now that the technology is available and then will have a smaller need going forward. Is it like the stereotypical dad who rents a power washer, powerwashes every exposed surface on his property, and then doesn't need to do any powerwashing for a few years; his neighbor who gets an Instant Pot and uses it for every meal for a month, then sees it gathering dust when the family gets tired of pressure-cooked stews; or like their neighbor who gets a microwave oven and uses it multiple times a day for decades? I guess only time will tell.

JSR_FDED 5mo ago
That’s been my theory - there’s some low hanging fruit in every environment where AI knocks it out of the park. Then complex brownfield reality (coupled with non-technical factors) rears its head and the stunning productivity gains are nowhere near to be seen. That’s the explanation how you can have both the anecdotes of amazing AI productivity and rigorous studies showing anything from actual loss of productivity to single-digit gains.
- joenot443 5mo ago
  I think this is directionally right. The code AI produces is not created equally, not even close.
- blensor 5mo ago
  I also think in addition to that the increased speed compounds the problems much quicker. And I don't mean bugs. I mean that duplicated code here, that additional state variable to keep everything going there. Not removing things that should be removed because we can work around that, etc. It's like building a super tall Jenga tower very quickly but laying the bricks much worse than a careful player.
thewebguyd 5mo ago
So far where I work its the Instant Pot, at least for the non-devs. We rolled out Claude & Cowork to the masses after a brief pilot. It was about a solid month and a half of heavy usage and then suddenly usage fell off a cliff. Once it stopped being a cool new toy, people just didn't find a use for it. A few mundane things got automated, but these were just back office admin type work. Nothing that's going to show on the P&L. Yeah those people now have a little more time for other things, but those other things are also not revenue generating. No FTE got replaced by it so in the end they just paid for a bunch of administrative positions to be a little less busy. Great for the workers who are now less stressed, but almost no impact on the business financials except there's now yet another subscription.
- palmotea 5mo ago

rtkwe 5mo ago

In my limited experience with using agents to create tests it tends to code the tests to the existing code instead of ensuring the correctness from a spec. Great for regression testing but still limited in effectiveness for catching existing issues.

levkk 5mo ago

It wouldn't, at least not directly. That's why it wasn't done pre-AI.

ElFitz 5mo ago

Over here our CTO replaced Intercom with an internal equivalent that costs less than $20 / month to run, haiku and sonnet support agent costs included. In less than a few weeks, in his spare time.