9 ms·

The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services? A car when in neutral wi

by nuc1e0n 11mo ago

The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services? A car when in neutral will rev up easily if the accelerator pedal is pushed even very slightly, because there's no load on the engine. But in gear the same engine will rev up much less when the accelerator is pushed the same amount. Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising? I doubt it. And what if the technology to locally run these systems without reliance on the cloud becomes commonplace, as it now is with open source models? The expensive part is in the training of these models more than the inference.

aitchnyu 11mo ago
Will the OpenRouter marketplace of M clouds X N models die if the investor money stops? I believe its a free and profitable service, offered completely pay as you go.
an0malous 11mo ago
> The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services? Absolutely, not only are most AI services free but even the paid portion is coming from executives mandating that their employees use AI services. It's a heavily distorted market.
- keeda 11mo ago
  On the flip side, there are many indications that a huge part of corporate AI usage is "Shadow AI", i.e. where workers use GenAI without their employer's blessing or knowledge. E.g. https://cacm.acm.org/news/the-real-significant-threat-of-shadow-ai/ https://cacm.acm.org/news/the-real-significant-threat-of-sha... And a majority of those workers do not reveal their AI usage, so they either take credit for the faster work or use the extra time for other activities, which further confounds the impact of AI. This is also distorting the market, but in other ways.
treis 11mo ago
We're talking miraculous level of improvement for a SOA LLM to run on a phone without crushing battery life this decade. People are missing the forest for the trees here. Being the go to consumer Gen AI is a trillion+ dollar business. How many 10s of billions you waste on building unnecessary data centers is a rounding error. The important number is your odds of becoming that default provider in the minds of consumers.
- nuc1e0n 11mo ago
  How big does an LLM need to be to support natural language queries with RAG?
  - mattnewton 11mo ago
    My hot (maybe just warm these days) take is, the problem with voice assistants on phones is they have to be able to have reasonable responses to a long tail or users will learn not to use them, since the use cases aren’t discoverable and the primarily value is talking to it like a person. So voice assistants backed by very large LLMs over the network are going to win even if we solve the (substantial) battery usage issue.
    - nuc1e0n 11mo ago
      Why even bother with the text generation then? You could just make a phone call to an LLM with a TTS frontend. Like with directory enquiries back in the day. Which can be set up as easily as a BBS if you have a home server rack like Jeff Geerling makes youtube videos about.
      - jazzyjackson 11mo ago
        1 800 CHAT GPT exists, I use it often enough to put it on speed dial on my flip phone
- bayarearefugee 11mo ago
  > The important number is your odds of becoming that default provider in the minds of consumers. I haven't seen any evidence that any Gen AI provider will be able to build a moat that allows for this. Some are better than others at certain things over certain time periods, but they are all relatively interchangeable for most practical uses and the small differences are becoming less pronounced, not more. I use LLMs fairly frequently now and I just bounce around between them to stay within their free tiers. Short of some actual large breakthrough I never need to commit to one, and I can take advantage of their own massive spends and wait it out a couple of years until I'm running a local model self-hosted with a cloudflare tunnel if I need to access it on my phone. And yes, most people won't do that, but there will be a lot of opportunity for cheap providers to offer that as a service with some data center spend, but nowhere near the massive amounts OpenAI, Google, Meta, et al are burning now.
mNovak 11mo ago
Of all the players, I'd argue Google certainly knows how to give away a product for free and still make money. The local open source argument doesn't hold water for me -- why does anyone buy Windows, Dropbox, etc when there's free alternatives?
- steveBK123 11mo ago
  Yes, it seems to me their strategy is to watch the OpenAI, Anthropics, etc of the world bleed themselves to death.
- bee_rider 11mo ago
  Not many people buy Windows, they buy laptops that happen to have Windows installed. IMO this is a worthwhile distinction because most people don’t really care about operating systems anyway, and would happily (I suspect, at least) use an Open Source one if it came installed and configured on a device that they got in a store. Installing an OS is seen as a hard/technical task still. Installing a local program, not so much. I suspect people install LLM programs from app stores without knowing if they are calling out to the internet or running locally.
- lelanthran 11mo ago
  > why does anyone buy Windows, Dropbox, etc when there's free alternatives? No one buys Windows - it comes with the PC. If people were shipped blank computers and told to order the OS separately, they wouldn't be buying Windows at the current price point.
Anon1096 11mo ago
Besides the fact that this article is obviously AI generated (and not even well, why is there mismatches in british/american english? I can only assume that the few parts in british english are the human author's writing or edits), yes "overutilization" is not a real thing. There is a level of utilization at every price point. If something is "overutilizated" that actually means it's just being offered at a low price, which is good for consumers. It's a nice scare word though and there's endless appetite at the moment for ai-doomer articles.
- bobmcnamara 11mo ago
  > why is there mismatches in british/american english You sometimes see this with real live humans who have lived in multiple counties.
  - delusional 11mo ago
    I do this because I'm a non-native english speaker. My preference varies from word to word. I write color, but i also write aliminium.
  - SJC_Hacker 11mo ago
    > multiple counties Pay no attention to those fopheads from Kent. We speak proper British English here in Essex
  - nucleardog 11mo ago
    > You sometimes see this with real live humans who have lived in multiple counties. Also very common with... most Canadians. We officially use an English closer to British English (Zed not zee, honour not honor) however geographically and culturally we're very close to the US. At school you learn "X, Y, Zed". The toy you buy your toddler is made for the US and Canadian market and sings "X, Y, Zee" as does practically every show on TV. The dictionary says it's spelled "colour" but most of the books you read will spell it "color". Most people we communicate with are either from Canada or the US, so much of our personal communication is with US English. But also there are a number of British shows that air here, so some particularly British phrases do sneak in to a lot of people's lexicon. See a similar thing in the way we measure things. We use celsius for temperature but most of our thermostats default to Fahrenheit and most cookbooks are primarily in imperial measures and units because they're from the US. The store sells everything in grams and kilograms, but most recipes are still in tablespoons/cups/etc. Most things are sold in metric, but when you buy lumber it's sold in feet, and any construction site is going to be working primarily in feet and inches. If anything I expect any AI-written content would be more consistent about this than I usually am.
btilly 11mo ago
Yes, over-utilization is a natural response to being undercharged. And being undercharged is a natural result when investors are throwing money at you. During bubbles, Silicon Valley often goes to "lose money, make it up with scale". With the vague idea that after you get to scale, THEN you can figure out how to make money. And fairly consistently, their idea for how to make money is "sell ads". Past successes like Google encourage hope in this strategy. Sure, it mostly doesn't work. Most of of everything that VCs do doesn't work. Returns follow a power law, and a handful of successes in the tail drive the whole portfolio. The key problem here doesn't lie in the fact that this strategy is being pursued. The key problem is that it is rare for first mover advantages to last with new technologies. That's why Netscape and Yahoo! aren't among the FAANGs today. The long-term wins go to whoever successfully create a sufficient moat for themselves to protect lasting excess returns. And the capabilities of each generation of AI leapfrogs the last so well that nobody has figured out how to create such a moat. Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997, 3 years after launching Netscape Navigator. I see no reason to expect that 30 years from now OpenAI will be any more dominant than Netscape is today. Right now companies are pouring money into their candidates to win the AI race. But if the history of browsers repeats itself, the company that wins in the long-term would launch in about a year from now, focused on applications on top of AI. And its entrant into the AI wars wouldn't get launched until a decade after that! (Yes, that is the right timeline for the launch of Google, and Google's launch of Chrome.) Investing in silicon valley is like buying a positive EV lottery ticket. An awful lot of people are going to be reminded the hard way that it is wiser to buy a lot of lottery tickets, than it is to sink a fortune into a single big one.
- rhetocj23 11mo ago
  [dead]
- signatoremo 11mo ago
  > Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997. Incorrect. There were about 150 millions Internet users in 1998, or 3.5% of total population. The number grew 10 times by 2008 [0]. Netwcape had about 50% of the browser market at the time [1]. In other words, Netscape dominated a small base and couldn’t keep it up. ChatGPT has about 800 millions monthly users, or already 10% of total current population. Granted, not exclusively. ChatGPT is already a household name. Outside of early internet adopters, very few people knew who Netscape or what Navigator was. [0] https://archive.globalpolicy.org/component/content/article/109-tables-and-charts/27519-internet-users.html?utm_source=chatgpt.com https://archive.globalpolicy.org/component/content/article/1... [1] https://www.wired.com/1999/06/microsoft-leading-browser-war/?utm_source=chatgpt.com https://www.wired.com/1999/06/microsoft-leading-browser-war/...
deleted 11mo ago
[deleted]
burnte 11mo ago
> Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising? > I doubt it. I agree. Right now a lot of AI tools are underpriced to get customers hooked, then they'll jack up the prices later. The flaw is that AI does not have the ubiquitous utility internet access has, and a lot of people are not happy with the performance per dollar TODAY, much less when prices rise 80%. We already see companies like Google raising prices stating it's for "AI" and we customers can't opt out of AI and not pay the fee. At my company we've already decided to leave Google Workspace in the spring. GW is a terrible product with no advanced features, garbage admin tools, uncompetitive pricing, and now AI shoved in everywhere and no way to granularly opt out of a lot of it. Want spell check? Guess what, you need to leave Gemini enabled! Shove off, Google.
- gboss 11mo ago
  Where are you moving to?
  - hotpaper75 11mo ago
    Best Comment
- LMSolar 11mo ago
  Yes, where are you moving to from GW? We're looking for options...
- beepbopboopp 11mo ago
  > Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising? > I doubt it. Yea, I think this is wrong. The analogy is more like the App Store, in that there is very little to do currently other than a better Google Search with the product. The bet is that over time (short time) there are much more financially valuable use cases with a more mature ecosystem and tech.
  - echelon 11mo ago
keeda 11mo ago
> I doubt it. I don't. This is simply the "drug dealer" model where the first hit is free. They know that once people are addicted, they will keep coming back. The question of course is, will they keep coming back? I think they very much will. There are indications that GenAI adoption is already increasing labor producitivity labor improvements at a national scale, which is quite astounding for a technology just 3 years old: https://news.ycombinator.com/item?id=46061369 https://news.ycombinator.com/item?id=46061369 Imagine a magic box where you put in some money and get more productivity back. There is no chance Capitalism (with a capital "C") is going to let such a powerful growth machine wither on the vine. This mad AI rush is all about that.

by nuc1e0n 11mo ago

aitchnyu 11mo ago

Will the OpenRouter marketplace of M clouds X N models die if the investor money stops? I believe its a free and profitable service, offered completely pay as you go.

an0malous 11mo ago

> The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services? Absolutely, not only are most AI services free but even the paid portion is coming from executives mandating that their employees use AI services. It's a heavily distorted market.

keeda 11mo ago
On the flip side, there are many indications that a huge part of corporate AI usage is "Shadow AI", i.e. where workers use GenAI without their employer's blessing or knowledge. E.g. https://cacm.acm.org/news/the-real-significant-threat-of-shadow-ai/ https://cacm.acm.org/news/the-real-significant-threat-of-sha... And a majority of those workers do not reveal their AI usage, so they either take credit for the faster work or use the extra time for other activities, which further confounds the impact of AI. This is also distorting the market, but in other ways.

treis 11mo ago

We're talking miraculous level of improvement for a SOA LLM to run on a phone without crushing battery life this decade. People are missing the forest for the trees here. Being the go to consumer Gen AI is a trillion+ dollar business. How many 10s of billions you waste on building unnecessary data centers is a rounding error. The important number is your odds of becoming that default provider in the minds of consumers.

nuc1e0n 11mo ago
How big does an LLM need to be to support natural language queries with RAG?
- mattnewton 11mo ago
  My hot (maybe just warm these days) take is, the problem with voice assistants on phones is they have to be able to have reasonable responses to a long tail or users will learn not to use them, since the use cases aren’t discoverable and the primarily value is talking to it like a person. So voice assistants backed by very large LLMs over the network are going to win even if we solve the (substantial) battery usage issue.
  - nuc1e0n 11mo ago
    Why even bother with the text generation then? You could just make a phone call to an LLM with a TTS frontend. Like with directory enquiries back in the day. Which can be set up as easily as a BBS if you have a home server rack like Jeff Geerling makes youtube videos about.
    - jazzyjackson 11mo ago
      1 800 CHAT GPT exists, I use it often enough to put it on speed dial on my flip phone
bayarearefugee 11mo ago
> The important number is your odds of becoming that default provider in the minds of consumers. I haven't seen any evidence that any Gen AI provider will be able to build a moat that allows for this. Some are better than others at certain things over certain time periods, but they are all relatively interchangeable for most practical uses and the small differences are becoming less pronounced, not more. I use LLMs fairly frequently now and I just bounce around between them to stay within their free tiers. Short of some actual large breakthrough I never need to commit to one, and I can take advantage of their own massive spends and wait it out a couple of years until I'm running a local model self-hosted with a cloudflare tunnel if I need to access it on my phone. And yes, most people won't do that, but there will be a lot of opportunity for cheap providers to offer that as a service with some data center spend, but nowhere near the massive amounts OpenAI, Google, Meta, et al are burning now.

mNovak 11mo ago

Of all the players, I'd argue Google certainly knows how to give away a product for free and still make money. The local open source argument doesn't hold water for me -- why does anyone buy Windows, Dropbox, etc when there's free alternatives?

steveBK123 11mo ago
Yes, it seems to me their strategy is to watch the OpenAI, Anthropics, etc of the world bleed themselves to death.
bee_rider 11mo ago
Not many people buy Windows, they buy laptops that happen to have Windows installed. IMO this is a worthwhile distinction because most people don’t really care about operating systems anyway, and would happily (I suspect, at least) use an Open Source one if it came installed and configured on a device that they got in a store. Installing an OS is seen as a hard/technical task still. Installing a local program, not so much. I suspect people install LLM programs from app stores without knowing if they are calling out to the internet or running locally.
lelanthran 11mo ago
> why does anyone buy Windows, Dropbox, etc when there's free alternatives? No one buys Windows - it comes with the PC. If people were shipped blank computers and told to order the OS separately, they wouldn't be buying Windows at the current price point.

Anon1096 11mo ago

Besides the fact that this article is obviously AI generated (and not even well, why is there mismatches in british/american english? I can only assume that the few parts in british english are the human author's writing or edits), yes "overutilization" is not a real thing. There is a level of utilization at every price point. If something is "overutilizated" that actually means it's just being offered at a low price, which is good for consumers. It's a nice scare word though and there's endless appetite at the moment for ai-doomer articles.

bobmcnamara 11mo ago
> why is there mismatches in british/american english You sometimes see this with real live humans who have lived in multiple counties.
- delusional 11mo ago
  I do this because I'm a non-native english speaker. My preference varies from word to word. I write color, but i also write aliminium.
- SJC_Hacker 11mo ago
  > multiple counties Pay no attention to those fopheads from Kent. We speak proper British English here in Essex
- nucleardog 11mo ago
  > You sometimes see this with real live humans who have lived in multiple counties. Also very common with... most Canadians. We officially use an English closer to British English (Zed not zee, honour not honor) however geographically and culturally we're very close to the US. At school you learn "X, Y, Zed". The toy you buy your toddler is made for the US and Canadian market and sings "X, Y, Zee" as does practically every show on TV. The dictionary says it's spelled "colour" but most of the books you read will spell it "color". Most people we communicate with are either from Canada or the US, so much of our personal communication is with US English. But also there are a number of British shows that air here, so some particularly British phrases do sneak in to a lot of people's lexicon. See a similar thing in the way we measure things. We use celsius for temperature but most of our thermostats default to Fahrenheit and most cookbooks are primarily in imperial measures and units because they're from the US. The store sells everything in grams and kilograms, but most recipes are still in tablespoons/cups/etc. Most things are sold in metric, but when you buy lumber it's sold in feet, and any construction site is going to be working primarily in feet and inches. If anything I expect any AI-written content would be more consistent about this than I usually am.

btilly 11mo ago

Yes, over-utilization is a natural response to being undercharged. And being undercharged is a natural result when investors are throwing money at you. During bubbles, Silicon Valley often goes to "lose money, make it up with scale". With the vague idea that after you get to scale, THEN you can figure out how to make money. And fairly consistently, their idea for how to make money is "sell ads". Past successes like Google encourage hope in this strategy. Sure, it mostly doesn't work. Most of of everything that VCs do doesn't work. Returns follow a power law, and a handful of successes in the tail drive the whole portfolio. The key problem here doesn't lie in the fact that this strategy is being pursued. The key problem is that it is rare for first mover advantages to last with new technologies. That's why Netscape and Yahoo! aren't among the FAANGs today. The long-term wins go to whoever successfully create a sufficient moat for themselves to protect lasting excess returns. And the capabilities of each generation of AI leapfrogs the last so well that nobody has figured out how to create such a moat. Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997, 3 years after launching Netscape Navigator. I see no reason to expect that 30 years from now OpenAI will be any more dominant than Netscape is today. Right now companies are pouring money into their candidates to win the AI race. But if the history of browsers repeats itself, the company that wins in the long-term would launch in about a year from now, focused on applications on top of AI. And its entrant into the AI wars wouldn't get launched until a decade after that! (Yes, that is the right timeline for the launch of Google, and Google's launch of Chrome.) Investing in silicon valley is like buying a positive EV lottery ticket. An awful lot of people are going to be reminded the hard way that it is wiser to buy a lot of lottery tickets, than it is to sink a fortune into a single big one.

rhetocj23 11mo ago
[dead]
signatoremo 11mo ago
> Today, 3 years after launching the first LLM chatbot, OpenAI is nowhere near as dominant as Netscape was in late 1997. Incorrect. There were about 150 millions Internet users in 1998, or 3.5% of total population. The number grew 10 times by 2008 [0]. Netwcape had about 50% of the browser market at the time [1]. In other words, Netscape dominated a small base and couldn’t keep it up. ChatGPT has about 800 millions monthly users, or already 10% of total current population. Granted, not exclusively. ChatGPT is already a household name. Outside of early internet adopters, very few people knew who Netscape or what Navigator was. [0] https://archive.globalpolicy.org/component/content/article/109-tables-and-charts/27519-internet-users.html?utm_source=chatgpt.com https://archive.globalpolicy.org/component/content/article/1... [1] https://www.wired.com/1999/06/microsoft-leading-browser-war/?utm_source=chatgpt.com https://www.wired.com/1999/06/microsoft-leading-browser-war/...

deleted 11mo ago

[deleted]

burnte 11mo ago

> Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising? > I doubt it. I agree. Right now a lot of AI tools are underpriced to get customers hooked, then they'll jack up the prices later. The flaw is that AI does not have the ubiquitous utility internet access has, and a lot of people are not happy with the performance per dollar TODAY, much less when prices rise 80%. We already see companies like Google raising prices stating it's for "AI" and we customers can't opt out of AI and not pay the fee. At my company we've already decided to leave Google Workspace in the spring. GW is a terrible product with no advanced features, garbage admin tools, uncompetitive pricing, and now AI shoved in everywhere and no way to granularly opt out of a lot of it. Want spell check? Guess what, you need to leave Gemini enabled! Shove off, Google.

gboss 11mo ago
Where are you moving to?
- hotpaper75 11mo ago
  Best Comment
LMSolar 11mo ago
Yes, where are you moving to from GW? We're looking for options...
beepbopboopp 11mo ago
> Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising? > I doubt it. Yea, I think this is wrong. The analogy is more like the App Store, in that there is very little to do currently other than a better Google Search with the product. The bet is that over time (short time) there are much more financially valuable use cases with a more mature ecosystem and tech.
- echelon 11mo ago

keeda 11mo ago

> I doubt it. I don't. This is simply the "drug dealer" model where the first hit is free. They know that once people are addicted, they will keep coming back. The question of course is, will they keep coming back? I think they very much will. There are indications that GenAI adoption is already increasing labor producitivity labor improvements at a national scale, which is quite astounding for a technology just 3 years old: https://news.ycombinator.com/item?id=46061369 https://news.ycombinator.com/item?id=46061369 Imagine a magic box where you put in some money and get more productivity back. There is no chance Capitalism (with a capital "C") is going to let such a powerful growth machine wither on the vine. This mad AI rush is all about that.