4 ms·
> Coding agents really did change everything. These are tools which burn vastly more tokens The assumption here is that this is a positive thing. But this ver
by cj 5mo ago
> Coding agents really did change everything. These are tools which burn vastly more tokens
The assumption here is that this is a positive thing.
But this very well could end up being a major negative long term by increasing the cost per user, reducing margins.
More usage = more cost = less profit.
It's not obvious that more usage is good. It's only good if revenue per user increases more than cost does. I'm skeptical about that.
- simonw 5mo ago> It's only good if revenue per user increases more than cost does. That's why it's so important for these labs that they're selling API tokens for more than the compute+energy costs needed to generate them. Every indicator I've seen is that they do have a positive margin on that. If they don't, they're screwed.
- mattas 5mo agoWhat's an example of an indicator? Genuinely curious!
- simonw 5mo agoInsider tips from Google and AWS telling me that they run inference at a profit (though that was over a year ago now). Dario telling Dwarkesh three months ago that they have a margin on inference: https://www.dwarkesh.com/p/dario-amodei-2?timestamp=3528.0 https://www.dwarkesh.com/p/dario-amodei-2?timestamp=3528.0
- infinitezest 5mo agoAre these sources not incentivized to say exactly this, regardless of whether it's true?
- simonw 5mo agoThe insiders tips? I don't think so, they were people I trust. They had all the incentive in the world to say "I'm not going to talk about that."
- Denzel 5mo agoUnless your insider is the CFO, I wouldn’t trust these sources to have the access, knowledge, or insight to determine whether they’re running inference at a profit. Simple test: can they get their hands on a data center contracts and financials? Search isn’t anywhere near as high profile as the profitability of AI inference, and yet, even aspects of the Search org were walled off from the rest of the company such that other employees couldn’t see what we see.
- simonw 5mo agoI don't think the fact that inference runs at a profit is a particularly deeply kept secret inside these companies. It might be worth keeping it secret from the staff if they were running at a loss.
- grttq 5mo agoNo this is not what matters. The customers of these tokens need to see returns on their projects that exceed the cost of financing. Laying people off only goes so far. If enough said firms don’t see enough value given the price of frontiers they will cancel and consume open source. This is the risk the frontier labs are exposed to.