4 ms·

I keep seeing the "Productivity Paradox" highlighted over an over again. I think one thing people are missing with this specific technology is that unlike many

I keep seeing the "Productivity Paradox" highlighted over an over again. I think one thing people are missing with this specific technology is that unlike many of the comparisons (computers, internet, broadband, etc), AI in particular doesn't have a high requirement at the consumer side. Everyone already has everything they need to use it. There will be a period like we are in now where dramatic capability gain (like recent coding gains) take a while for people to adapt to, however, I think the change will be much faster. Even the speed of uptake in coding tools over the last 3 months has been faster than I predicted. I think we'll see other shifts like this in different sectors where it changes almost over a series of a few months.

afavour 8mo ago
> AI in particular doesn't have a high requirement at the consumer side. Everyone already has everything they need to use it. That isn’t actually true though, right now everyone has a hard dependency on a cloud service. That is currently sold to them at deep discount by companies that are losing billions. When the market eventually corrects it’ll be interesting to see how much AI ends up costing. At the very least it will be comparable to the broadband internet connection you mentioned. Possibly a whole lot more.
- slg 8mo ago
  >That isn’t actually true though, right now everyone has a hard dependency on a cloud service. That is currently sold to them at deep discount by companies that are losing billions. Isn't that a huge red flag? If customers are being given this product at a discount and it still isn't showing a positive ROI for them, what makes people think it will improve once we're charged full price?
  - weevil 8mo ago
    I think most people just assume it's magic, and are too awestruck by the hype to think critically. Financially this feels similar to Uber's business plan in the 2010s; undercut the market with unsound pricing propped up by venture capital (PE was literally subsidising taxi fares; they admitted this and their intention to readjust, but no one seemed to care) then stop manipulating the market and allow fares to even out at (gasp) what it cost to get a cab before Uber. The difference here is that the LLM market is human productivity; enormous subsidies are afforded to Anthropic, OpenAI etc. in the form of VC or compute credit, but eventually those debts will be called in, the free-to-use aspect will vanish because it's simply not profitable, and we'll be left with several premium products that only a few people will actually pay for, and even then that may not be enough to cover their costs. That's when the bubble will burst.
    - great_psy 8mo ago
      Actually I think there’s another option. There’s the scenario where LLMs get more efficient in size, and to get 2026 SOTA performance you will be able to get it from consumer grade laptop. Sure with a 1000B parameter you will get better performance but the average person will have it write some python script, not derive new physics equations. So in a sense the demand for LLM intelligence with reach a plateau (arguably we are there today for avg person) so there will not be any subsidy required, because the avg person will not need the latest and greatest. There’s not the same demand pattern for something like uber.
      - palmotea 8mo ago
        > There’s the scenario where LLMs get more efficient in size, and to get 2026 SOTA performance you will be able to get it from consumer grade laptop. But isn't that bad for the AI companies, too? Because then people just run an ~2026 SOTA performance open source model on their laptop for free and not pay any subscription.
        great_psy 8mo ago
        Yes and no. Regular folks will not pay Anthropic, but NSA, NASA or research labs might. I’m not implying this will be a good time for AI companies. I am saying AI as a technology can provide value without it being controlled by only 3 companies.
        jononor 8mo ago
        In a hypothetical future with 2026 level LLMs on a (high end) consumer laptop, I still think that majority of buyers would prefer to pay 20 USD/month for a service. Just for the convenience and flexibility.
        palmotea 8mo ago
        > In a hypothetical future with 2026 level LLMs on a (high end) consumer laptop, I still think that majority of buyers would prefer to pay 20 USD/month for a service. Just for the convenience and flexibility. $20 a month is a lot of money, I don't think the "convenience and flexibility" you get would actually be worth it, unless you've 1) got money to burn, 2) lack the skills to install software, 3) the open source community totally fails to develop a reasonable installer. The LLM service would probably be akin to a scam preying on ignorance, like those companies that will rent you a water softener for like $100/month.
- BobbyJo 8mo ago
  Is it actually being sold at a steep discount? Anthropic CEO has stated they have high margins on inference, so training is the big cost center.
  - ambicapter 8mo ago
    Well, training isn't going to end soon if these companies keep on competing with one another whilst being neck-and-neck, so I'm not sure why you would ignore the cost of training in the ROI calculation.
  - numbsafari 8mo ago
    Does the cumulative earnings from inference on a single model exceed its training costs? That’s.. kinda the question.
    - ainch 8mo ago
      Amodei says yes - each model pays for its training. But they're scaling up investment for each new run, so they're still happily in the red. And also that may be the case for Anthropic who have fewer free users, a large enterprise business, and less generous rate limits on their subscriptions. I don't know if OpenAI or Google have commented. I suspect OpenAI is in a worse position given their massive non-paying consumer base.
  - itsmenick 8mo ago
    Then why are they stopping people from having multiple max plans? If they are making such good margins on inference.
    - lehmacdj 8mo ago
      They have good margins on inference at API costs, i.e. $5/$25 per mtok input/output. They are almost certainly making losses on subscriptions, at least if people max out rate limits. In the past 30 days I have burned $78.19 in API token costs with my $20/month Claude Pro subscription. In January I burnt over $300 in API token costs.
- fooster 8mo ago
  >That is currently sold to them at deep discount by companies that are losing billions. They're not losing billions on inference, they're losing billions in the arms race of training.
deleted 8mo ago
[deleted]
sifar 8mo ago
> AI in particular doesn't have a high requirement at the consumer side Effective use of these AI tools need high critical thinking skills which are in short supply.
fallinditch 8mo ago
At the large insurance company I'm doing some work for the big capability gains have yet to materialize. There are some pockets of workflow innovation but big institutions can carry a kind of inertia and are slow to adapt. But as the organization slowly learns and adapts I'm sure the capability gains will materialize.

afavour 8mo ago

> AI in particular doesn't have a high requirement at the consumer side. Everyone already has everything they need to use it. That isn’t actually true though, right now everyone has a hard dependency on a cloud service. That is currently sold to them at deep discount by companies that are losing billions. When the market eventually corrects it’ll be interesting to see how much AI ends up costing. At the very least it will be comparable to the broadband internet connection you mentioned. Possibly a whole lot more.

slg 8mo ago
>That isn’t actually true though, right now everyone has a hard dependency on a cloud service. That is currently sold to them at deep discount by companies that are losing billions. Isn't that a huge red flag? If customers are being given this product at a discount and it still isn't showing a positive ROI for them, what makes people think it will improve once we're charged full price?
- weevil 8mo ago
  I think most people just assume it's magic, and are too awestruck by the hype to think critically. Financially this feels similar to Uber's business plan in the 2010s; undercut the market with unsound pricing propped up by venture capital (PE was literally subsidising taxi fares; they admitted this and their intention to readjust, but no one seemed to care) then stop manipulating the market and allow fares to even out at (gasp) what it cost to get a cab before Uber. The difference here is that the LLM market is human productivity; enormous subsidies are afforded to Anthropic, OpenAI etc. in the form of VC or compute credit, but eventually those debts will be called in, the free-to-use aspect will vanish because it's simply not profitable, and we'll be left with several premium products that only a few people will actually pay for, and even then that may not be enough to cover their costs. That's when the bubble will burst.
  - great_psy 8mo ago
    Actually I think there’s another option. There’s the scenario where LLMs get more efficient in size, and to get 2026 SOTA performance you will be able to get it from consumer grade laptop. Sure with a 1000B parameter you will get better performance but the average person will have it write some python script, not derive new physics equations. So in a sense the demand for LLM intelligence with reach a plateau (arguably we are there today for avg person) so there will not be any subsidy required, because the avg person will not need the latest and greatest. There’s not the same demand pattern for something like uber.
    - palmotea 8mo ago
      > There’s the scenario where LLMs get more efficient in size, and to get 2026 SOTA performance you will be able to get it from consumer grade laptop. But isn't that bad for the AI companies, too? Because then people just run an ~2026 SOTA performance open source model on their laptop for free and not pay any subscription.
      - great_psy 8mo ago
        Yes and no. Regular folks will not pay Anthropic, but NSA, NASA or research labs might. I’m not implying this will be a good time for AI companies. I am saying AI as a technology can provide value without it being controlled by only 3 companies.
        jononor 8mo ago
        In a hypothetical future with 2026 level LLMs on a (high end) consumer laptop, I still think that majority of buyers would prefer to pay 20 USD/month for a service. Just for the convenience and flexibility.
        palmotea 8mo ago
        > In a hypothetical future with 2026 level LLMs on a (high end) consumer laptop, I still think that majority of buyers would prefer to pay 20 USD/month for a service. Just for the convenience and flexibility. $20 a month is a lot of money, I don't think the "convenience and flexibility" you get would actually be worth it, unless you've 1) got money to burn, 2) lack the skills to install software, 3) the open source community totally fails to develop a reasonable installer. The LLM service would probably be akin to a scam preying on ignorance, like those companies that will rent you a water softener for like $100/month.
BobbyJo 8mo ago
Is it actually being sold at a steep discount? Anthropic CEO has stated they have high margins on inference, so training is the big cost center.
- ambicapter 8mo ago
  Well, training isn't going to end soon if these companies keep on competing with one another whilst being neck-and-neck, so I'm not sure why you would ignore the cost of training in the ROI calculation.
- numbsafari 8mo ago
  Does the cumulative earnings from inference on a single model exceed its training costs? That’s.. kinda the question.
  - ainch 8mo ago
    Amodei says yes - each model pays for its training. But they're scaling up investment for each new run, so they're still happily in the red. And also that may be the case for Anthropic who have fewer free users, a large enterprise business, and less generous rate limits on their subscriptions. I don't know if OpenAI or Google have commented. I suspect OpenAI is in a worse position given their massive non-paying consumer base.
- itsmenick 8mo ago
  Then why are they stopping people from having multiple max plans? If they are making such good margins on inference.
  - lehmacdj 8mo ago
    They have good margins on inference at API costs, i.e. $5/$25 per mtok input/output. They are almost certainly making losses on subscriptions, at least if people max out rate limits. In the past 30 days I have burned $78.19 in API token costs with my $20/month Claude Pro subscription. In January I burnt over $300 in API token costs.
fooster 8mo ago
>That is currently sold to them at deep discount by companies that are losing billions. They're not losing billions on inference, they're losing billions in the arms race of training.

deleted 8mo ago

[deleted]

sifar 8mo ago

> AI in particular doesn't have a high requirement at the consumer side Effective use of these AI tools need high critical thinking skills which are in short supply.

fallinditch 8mo ago

At the large insurance company I'm doing some work for the big capability gains have yet to materialize. There are some pockets of workflow innovation but big institutions can carry a kind of inertia and are slow to adapt. But as the organization slowly learns and adapts I'm sure the capability gains will materialize.