4 ms·

Well from my point of view. When they talk about gigawatt datacenters, then yes it is economically nonviable. You just need to know the scale of a gigawatt to r

Well from my point of view. When they talk about gigawatt datacenters, then yes it is economically nonviable. You just need to know the scale of a gigawatt to realize that we need to start building power plants and fortifying the power grid to ship a gigawatt of power to a single location. Until the build out which takes years mind you, it is competing with other consumers of power. Lets take another huge consumer of power like a large steel mills use 100 megawatt. So if that power becomes more expensive because of datacenters, then the price of steel will go up. And if the price of steel goes up it affects a lot of things in the economy. We are facing a situation that the short term effects are on memory and storage prices going up and lack of jet engines. Long term we wont be able to build actual buildings and ships without financing it with even more debt than today and everyone in the economy is going to service that debt through the price.

simianwords 7mo ago
but the costs of inference have been going down 20x to 30x over the years. so how can you tell it is nonviable? unless you are saying they are not paying market rate for the inference
- JanneVee 7mo ago
  So, they still booked up all the ram and ssd in the world and still going to use gigawatts of power. The price of energy production is not going to go down 20x and 30x it just means that they can cram in more inference on the same energy consumption if the cost goes down. But they aren't paying the market rate for inference because everything is subsidized with debt and investors money to scale as fast as possibly. They are flushed with money and that is why they can book up all silicon production.
- surgical_fire 7mo ago
  I have no idea if costs indeed came down 20x-30x. This claim sounds extremely fancy when AI companies bleed money, and will keep bleeding money in the foreseeable future. I don't pretend to know the future. Maybe LLMs become economically viable and are the future, maybe not. I don't really care either way, to be frank. And I use LLMs, btw. I pay for a ChatGPT account, but I find it only moderately useful. I always sort of question myself upon renewal date if it is worth the 20 bucks I spend monthly on it. In no small part I keep using it to keep myself up to date on the best practices of using them in case it becomes standard.
  - simianwords 7mo ago
    https://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends Do you have any reason to not believe it? It’s expected for costs to come down
    - surgical_fire 7mo ago
      The graph you linked seems to compare different OpenAI models in terms of "price per million tokens". I am very skeptical of any financial information that comes from OpenAI. I have no idea how truthful those numbers are, or how creatively they can be collected to paint a rosier future for them. Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not? Also, I don't know this "epoch.ai" website, I don't know their stance. The website name itself does not inspire my confidence on their reporting of anything related to AI. "Eat meat, says the butcher" vibes and all. You can claim that the AI bleeds money because training is expensive, but inference is cheap. So it will only be financially viable when they stop training models? So they would need to stop improving their capabilities entirely for it to make any sense, is that your claim? Even if I take this claim at face value (and that would take a lot of faith I don't have to give), it doesn't sound as good as you think it does.
      - simianwords 7mo ago
        >To analyze the decline in LLM prices over time, we focused on the most cost-effective LLMs above a certain performance threshold at each point in time. To identify these models, we iterated through models sorted by release date. In each iteration, we added a model to the set of cheapest models if it had a lower price than all previous models that scored at or above the threshold. Can you look at the analysis? It will make it clear. I mean its so obvious because GPT 4 costs way more than GPT 5.2-mini but much worse performance. >Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not? Do you think they are subsidising 900x or simply that the costs have gone down? Overall you have shown what I feel is extreme skepticism in something that is obvious. You can literally run a model in your laptop that matches an older closed model. Costs are obviously going down, I have shown data. Use your own anecdotes and report. Extreme skepticism in such a way doesn't do any help.

simianwords 7mo ago

but the costs of inference have been going down 20x to 30x over the years. so how can you tell it is nonviable? unless you are saying they are not paying market rate for the inference

JanneVee 7mo ago
So, they still booked up all the ram and ssd in the world and still going to use gigawatts of power. The price of energy production is not going to go down 20x and 30x it just means that they can cram in more inference on the same energy consumption if the cost goes down. But they aren't paying the market rate for inference because everything is subsidized with debt and investors money to scale as fast as possibly. They are flushed with money and that is why they can book up all silicon production.
surgical_fire 7mo ago
I have no idea if costs indeed came down 20x-30x. This claim sounds extremely fancy when AI companies bleed money, and will keep bleeding money in the foreseeable future. I don't pretend to know the future. Maybe LLMs become economically viable and are the future, maybe not. I don't really care either way, to be frank. And I use LLMs, btw. I pay for a ChatGPT account, but I find it only moderately useful. I always sort of question myself upon renewal date if it is worth the 20 bucks I spend monthly on it. In no small part I keep using it to keep myself up to date on the best practices of using them in case it becomes standard.
- simianwords 7mo ago
  https://epoch.ai/data-insights/llm-inference-price-trends https://epoch.ai/data-insights/llm-inference-price-trends Do you have any reason to not believe it? It’s expected for costs to come down
  - surgical_fire 7mo ago
    The graph you linked seems to compare different OpenAI models in terms of "price per million tokens". I am very skeptical of any financial information that comes from OpenAI. I have no idea how truthful those numbers are, or how creatively they can be collected to paint a rosier future for them. Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not? Also, I don't know this "epoch.ai" website, I don't know their stance. The website name itself does not inspire my confidence on their reporting of anything related to AI. "Eat meat, says the butcher" vibes and all. You can claim that the AI bleeds money because training is expensive, but inference is cheap. So it will only be financially viable when they stop training models? So they would need to stop improving their capabilities entirely for it to make any sense, is that your claim? Even if I take this claim at face value (and that would take a lot of faith I don't have to give), it doesn't sound as good as you think it does.
    - simianwords 7mo ago
      >To analyze the decline in LLM prices over time, we focused on the most cost-effective LLMs above a certain performance threshold at each point in time. To identify these models, we iterated through models sorted by release date. In each iteration, we added a model to the set of cheapest models if it had a lower price than all previous models that scored at or above the threshold. Can you look at the analysis? It will make it clear. I mean its so obvious because GPT 4 costs way more than GPT 5.2-mini but much worse performance. >Even if the numbers are truthful, I have no idea how the calculate price there. Is it in terms of cost of compute they rent? Is this cost subsidized or not? Do you think they are subsidising 900x or simply that the costs have gone down? Overall you have shown what I feel is extreme skepticism in something that is obvious. You can literally run a model in your laptop that matches an older closed model. Costs are obviously going down, I have shown data. Use your own anecdotes and report. Extreme skepticism in such a way doesn't do any help.