4 ms·
The thing that makes AI investment hard to reason about for individuals is that our expectations are mostly driven by a single person’s usage, just like many of
by iambateman 10mo ago
The thing that makes AI investment hard to reason about for individuals is that our expectations are mostly driven by a single person’s usage, just like many of the numbers reported in the article.
But the AI providers are betting, correctly in my opinion, that many companies will find uses for LLM’s which are in the trillions of tokens per day.
Think less of “a bunch of people want to get recipe ideas.”
Think more of “a pharma lab wants to explore all possible interactions for a particular drug” or “an airline wants its front-line customer service fully managed by LLM.”
It’s unusual that individuals and industry get access to basically similar tools at the same time, but we should think of tools like ChatGPT and similar as “foot in the door” products which create appetite and room to explore exponentially larger token use in industry.
- gradus_ad 10mo agoWhen I'm building out a new feature, I can churn through millions of tokens in Claude code. And that's just me... Now think about Claude code but integrated with Excel or datadog, or whatever app could be improved through LLM integration. Think about the millions of office workers, beyond just software engineers, who will be running hundreds of thousands or millions of tokens per day through these tools. Let's estimate 200 million office workers globally as TAM running an average of 250k tokens. That's 50 trillion tokens DAILY. Not sure what model provider profit per token is, but let's say it's .001 cents. Thats $500M per day in profit.
- not_the_fda 10mo agoCurrently there is no profit per token, quite a bit of loss per token, that's the problem. Your not going to make it up in volume.
- danielbln 10mo agoDo you have a source for that? I'm especially interested in a source for Anthropic.
- iambateman 10mo agohttps://www.wsj.com/tech/ai/openai-anthropic-profitability-e9f5bcd6?gaa_at=eafs&gaa_n=AWEtsqdr-DK6KUZO6hTO_MrlH5cySYfOPGQ3ybBJODtMM1yImpBDZqfUdHjC&gaa_ts=6930b648&gaa_sig=DioK8l-GqZDq-VWeAfAqxdEHLks3eQN2Ek06zgJzBVoy3sKsnUVDQLCX2eWnWDR1z5MS44DQQADX8OrmM0-ZdQ%3D%3D https://www.wsj.com/tech/ai/openai-anthropic-profitability-e... Anthropic expects to break even in 2028. They’re all unprofitable now.
- grim_io 10mo agopaywalled. Are they unprofitable because they don't profit on inference, or because they reinvest all of the profit into scaling up? Remember how long Amazon was unprofitable, by choice.
- lelanthran 10mo ago> Are they unprofitable because they don't profit on inference, or because they reinvest all of the profit into scaling up? They are scaling up using VC money, not revenue. As far as profit on inference goes, it's hard to separate it out from training: they cannot, at any given time, simply stop training because that would kill any advantage they have 6 months down the line. For all practical purposes, you can't look at their inference costs independent of the training cost; they need to keep spending on both if they want to continue doing inference. > Remember how long Amazon was unprofitable, by choice. That was a very different scenario - AMZ was not spending their revenue on land-grabbing, they were spending their revenue on long-lived infra, while AI companies now are spending VC investment, not revenue, on land-grabbing. The difference between spending your revenue on short-lived infra (training a new model, acquiring GPUs) and long-lived infra is that with long-lived infra, at any time, even after 10+ years, you can stop expanding your infra and keep the resulting revenue as profit. With short-lived infra (models, GPUs), you can't simply stop infra spending and collect profit from the revenue, because the infra reached end-of-life and needs to be replaced anyway.
- iambateman 10mo agoI’m with you on the Claude Code example —- it matches my experience. But I do think the important thing to look forward to is AI work which is totally detached from human intervention.
- watwut 10mo agoI find it absurd to pay for tokens I cant control, predict or even check in any reasonable way. It is literally amounts to "pay whatever random money the company asks you to pay" kind of contract.
- iambateman 10mo agoI pay $100/mo for CC and have functionally unlimited tokens. I find it irreplaceable.
- pickledoyster 10mo ago>When I'm building out a new feature, I can churn through millions of tokens in Claude code. + >Not sure what model provider profit per token is, but let's say it's .001 cents. So you'd be willing to pay thousands for a new feature, right?
- epistasis 10mo ago> Think more of “a pharma lab wants to explore all possible interactions for a particular drug” Pharma does not trust OpenAI with their data, and they don't work on tokens for any of the protein or chemical modeling. There will undoubtedly be tons of deep nets used by pharma, with many $1-10k buys replacing more expensive physical assays, but it won't be through OpenAI, and it won't be as big as a consumer business. Of course there may be other new markets opened up but current pharma is not big enough to move the needle in a major way for a company with an OpenAI valuation.
- iambateman 10mo agoMy claim is that there will exist some company which pharma is willing to trust for AI research…they presumably trust Microsoft with their email today. But my bigger claim is that ~half the Fortune 500 will be able to profitably deploy AI with spends in the tens or hundreds of millions per year quite soon. Not that pharma itself is a major contributor to that effect.
- nickff 10mo agoBut for 'AI' to be a winner-take-all market, it seems that the winner would have to be using customer data to improve the 'AI'. Not only do you have to believe that one of these (relatively) under-capitalized upstarts can corral the money, but also that they can convince (enterprise) customers to 'fork over' their proprietary data to only one provider, and also that the provider can then charge a monopoly rent. Those all seem possible, but I wouldn't assign greater than a 50% probability to any of them, and the valuations seem to imply near-certainty.
- s_ting765 10mo ago> “an airline wants its front-line customer service fully managed by LLM.” This has been experimented on before by many companies over the recent years, most notably Klarna which was among the earliest guinea pigs for it and had to later on backtrack on this "novel" idea when the results came out.
- danielbln 10mo agoDoes neither mean it can't work nor that it can't work with LLMs. Just that hacking together some RAG chatbot is probably not it.
- malfist 10mo agoBut if I'm a pharma lab, I don't want to rely on a statistical engine that makes mistakes to answer those questions, I want to query a database that is deterministic.
- hn_acc1 10mo agoThis. LLM is NOT the tool for a pharma lab - properly trained ML is the right tool. Heck, English is probably not even the right LANGUAGE to use for discussing chemical interactions.
- lenerdenator 10mo agoWould the proper language be math, or another human language? I could see things like "nitrate" and "nitrite" possibly being a stumbling block for an LLM.
- iambateman 10mo agoSam Altman says that he thinks scientific research is a huge opportunity for AI to contribute as it more fully develops and I think he’s right. Since I’m not a scientific researcher, I have no idea if he’s just blowing smoke but I think it’s reasonable to think of a purpose-built system which has an LLM component being used by a team to do something useful.
- bluefirebrand 10mo ago> Sam Altman says that he thinks scientific research is a huge opportunity for AI to contribute as it more fully develops and I think he’s right Sam Altman will say anything if he thinks it will increase people's investment in his company. Don't believe a word that slimeball speaks
- Gazoche 10mo ago> “a pharma lab wants to explore all possible interactions for a particular drug” How would an LLM be any good at this?