2 ms·

Only if you pay a flat rate for electricity and cooling.

deleted 7d ago
[deleted]
tristanj 7d ago
But Anthropic isn't paying for the electricity and cooling. They don't run their own data centers, they rent compute from providers who cover those costs. That's entirely why they can blow compute on the fun projects like this. If they had to pay extra for the electricity, they wouldn't do it.
- Barbing 7d ago
  Is the electricity cost far greater than the marketing value?
  - rightnutwingjob 7d ago
    The first is a physical quantity that can be written down. The second is approximately no better than astrology.
    - ehe78qhe 7d ago
      The second point is, sadly, true of quite a lot of aspects of software, including "design" and "quality"
      - nullsanity 7d ago
        [dead]
  - adastra22 7d ago
    The marginal electricity cost is zero.
    - lazide 6d ago
      Or specifically, electricity was already paid for with the pre-paid capacity. Not using it would not save them any money, they already paid for it.
      - Barbing 6d ago
        I was thinking if they owned their own data centers, how expensive might this project have been.
- londons_explore 6d ago
  But training LLM's is also a task one can do whenever you have a spare GPU-minutes. I wonder why they don't have some kind of scheduler which makes sure there are never any idle minutes. One would imagine they at least would have autoscaling on their production serving workload and use the freed compute capacity for model training for example.
  - esseph 6d ago
    I doubt they're inferencing on their training hardware
odo1242 7d ago
Most of the GPU cost is in the GPUs themselves (and in the space and maintenance costs of the building). Electricity is a small fraction, and it's not like datacenters are just going to shut down their servers when they're not in use. There is cost, but the cost is mostly the opportunity cost of not being able to do something else.
- toast0 7d ago
  > Electricity is a small fraction, and it's not like datacenters are just going to shut down their servers when they're not in use. I don't have any insight on modern GPU datacenters, but in decades past, some owned and operated datacenters didn put effort into making sure power management worked because the cost savings were worth it. I'm pretty sure I saw plans to shed load and power off servers if a utility made a demand response request or in case of loss of cooling. I wouldn't be surprised if some owned and operated data centers do regular full shutdowns at off peak... WOL, IPMI or RTC wakeup can bring them back when needed and if you already have a dynamic service orchestrator and setup times are acceptable, why not shut down if there's no actual priority work and there's also no idle priority opportunistic load either...
  - huslage 6d ago
    No one does this. There is zero value, economic or otherwise, in turning off machines.
  - esseph 6d ago
    I've never heard of anybody shitting a DC down off peak. Been around 30yr or so. > why not shut down if there's no actual priority work and there's also no idle priority opportunistic load either... Full shutdown and startup often kills capacitors and used to be dangerous for rotational HDD. Sometimes once you turn things off, they simply don't come back on. It happens.
    - esseph 6d ago
      Lol *shutting, sorry

deleted 7d ago

[deleted]

But Anthropic isn't paying for the electricity and cooling. They don't run their own data centers, they rent compute from providers who cover those costs. That's entirely why they can blow compute on the fun projects like this. If they had to pay extra for the electricity, they wouldn't do it.

Barbing 7d ago
Is the electricity cost far greater than the marketing value?
- rightnutwingjob 7d ago
  The first is a physical quantity that can be written down. The second is approximately no better than astrology.
  - ehe78qhe 7d ago
    The second point is, sadly, true of quite a lot of aspects of software, including "design" and "quality"
    - nullsanity 7d ago
      [dead]
- adastra22 7d ago
  The marginal electricity cost is zero.
  - lazide 6d ago
    Or specifically, electricity was already paid for with the pre-paid capacity. Not using it would not save them any money, they already paid for it.
    - Barbing 6d ago
      I was thinking if they owned their own data centers, how expensive might this project have been.
londons_explore 6d ago
But training LLM's is also a task one can do whenever you have a spare GPU-minutes. I wonder why they don't have some kind of scheduler which makes sure there are never any idle minutes. One would imagine they at least would have autoscaling on their production serving workload and use the freed compute capacity for model training for example.
- esseph 6d ago
  I doubt they're inferencing on their training hardware

odo1242 7d ago

Most of the GPU cost is in the GPUs themselves (and in the space and maintenance costs of the building). Electricity is a small fraction, and it's not like datacenters are just going to shut down their servers when they're not in use. There is cost, but the cost is mostly the opportunity cost of not being able to do something else.

toast0 7d ago
> Electricity is a small fraction, and it's not like datacenters are just going to shut down their servers when they're not in use. I don't have any insight on modern GPU datacenters, but in decades past, some owned and operated datacenters didn put effort into making sure power management worked because the cost savings were worth it. I'm pretty sure I saw plans to shed load and power off servers if a utility made a demand response request or in case of loss of cooling. I wouldn't be surprised if some owned and operated data centers do regular full shutdowns at off peak... WOL, IPMI or RTC wakeup can bring them back when needed and if you already have a dynamic service orchestrator and setup times are acceptable, why not shut down if there's no actual priority work and there's also no idle priority opportunistic load either...
- huslage 6d ago
  No one does this. There is zero value, economic or otherwise, in turning off machines.
- esseph 6d ago
  I've never heard of anybody shitting a DC down off peak. Been around 30yr or so. > why not shut down if there's no actual priority work and there's also no idle priority opportunistic load either... Full shutdown and startup often kills capacitors and used to be dangerous for rotational HDD. Sometimes once you turn things off, they simply don't come back on. It happens.
  - esseph 6d ago
    Lol *shutting, sorry