4 ms·

Does anyone have data on the relative cost of running AI-enhanced experiences vs. traditional ones? For example, search: what's the increase in cost to Microsof

Does anyone have data on the relative cost of running AI-enhanced experiences vs. traditional ones? For example, search: what's the increase in cost to Microsoft when you do a Copilot search vs. a plain Bing search? I have a vague hunch that it's a lot more expensive and is being subsidized by the companies in question as they try to grab market share, but I don't have anything to back up this intuition. My worry is that we're in a bubble where AI is being applied to cases where it's not actually cost-effective. This might distort the market by suppressing non-AI competitors and then lead to some pretty drastic monetization when the bills start to come due.

bawolff 3y ago
My intuition is that traditional search probably takes a lot of energy too. Its too bad we don't have any actual numbers
vmfunction 3y ago
From my limited understanding of this AI field, that it cost over million $ to train the model, after that the the query against the model is fast and doesn't cost a lot money. So with this AI/LLM it is more about the upfront cost, and how well the model is train using that millions dollars, and according to some data expert seems to be an art and a science [1]. Some AI experts here please correct me if I'm wrong. 1: https://www.amorphousdata.com/blog/time-series-vs-regression https://www.amorphousdata.com/blog/time-series-vs-regression
- sureglymop 3y ago
  But then isn't the model out of date very quickly? It would need to be constantly retrained with current information...
  - Legend2440 3y ago
    Yes, but for now you just accept that - ChatGPT had a knowledge cutoff in 2022 until relatively recently.
- saurik 3y ago
  Just because it doesn't cost that much to use the model vs. to train the model doesn't mean it isn't still quite a bit more expensive per query than a keyword search.
makz 3y ago
So, layoffs are happening not because AI is replacing those workers, they are doing it to subsidise AI.
- Legend2440 3y ago
  Neither. Tech companies are wildly profitable and can afford all these GPUs. Layoffs are because they just don't think they need that many workers.
  - beretguy 3y ago
    Or they are switching to cheaper labor overseas.

bawolff 3y ago

My intuition is that traditional search probably takes a lot of energy too. Its too bad we don't have any actual numbers

vmfunction 3y ago

From my limited understanding of this AI field, that it cost over million $ to train the model, after that the the query against the model is fast and doesn't cost a lot money. So with this AI/LLM it is more about the upfront cost, and how well the model is train using that millions dollars, and according to some data expert seems to be an art and a science [1]. Some AI experts here please correct me if I'm wrong. 1: https://www.amorphousdata.com/blog/time-series-vs-regression https://www.amorphousdata.com/blog/time-series-vs-regression

sureglymop 3y ago
But then isn't the model out of date very quickly? It would need to be constantly retrained with current information...
- Legend2440 3y ago
  Yes, but for now you just accept that - ChatGPT had a knowledge cutoff in 2022 until relatively recently.
saurik 3y ago
Just because it doesn't cost that much to use the model vs. to train the model doesn't mean it isn't still quite a bit more expensive per query than a keyword search.

makz 3y ago

So, layoffs are happening not because AI is replacing those workers, they are doing it to subsidise AI.

Legend2440 3y ago
Neither. Tech companies are wildly profitable and can afford all these GPUs. Layoffs are because they just don't think they need that many workers.
- beretguy 3y ago
  Or they are switching to cheaper labor overseas.