4 ms·
Can someone explain why everything is being marketed in terms of power consumption?
by Eufrat 6mo ago
Can someone explain why everything is being marketed in terms of power consumption?
- teaearlgraycold 6mo agoBecause that’s the limiting factor
- Animats 6mo agoSomehow we must be doing this wrong. "Do you realize that the human brain has been liken to an electronic brain? Someone said and I don't know whether he is right or not, but he said, if the human brain were put together on the basis of an IBM electronic brain, it would take 7 buildings the size of the Empire State Building to house it, it would take all the water of the Niagara River to cool it, and all of the power generated by the Niagara River to operate it." (Sermon by Paris Reidhead, circa 1950s.[1]) We're there on size and power. Is there some more efficient way to do this? [1] https://www.sermonindex.net/speakers/paris-reidhead/the-tragedy-of-third-generation-religion/ https://www.sermonindex.net/speakers/paris-reidhead/the-trag...
- whimsicalism 6mo agopretty sure evolution spent more time and energy getting there then we ultimately will
- huflungdung 6mo ago[dead]
- brianjlogan 6mo agoI'd imagine one day there will be a limiting factor of cash to burn as well.
- serf 6mo agokinda complicated though when you consider it fully. Power consumption only measures the environmental impact really, we come up with more clever ways to use the same amount of power daily. it's kind of like an electrical motor that exists before the strong understanding of lorentz/ohm's law. We don't really know how inefficient the thing is because we don't really know where the ceiling is aside from some loosey theoretical computational efficiency concepts that don't strongly apply to practical LLMs. to be clear, I don't disagree that it's the limiting factor, just that 'limits' is nuanced here between effort/ability and raw power use.
- zozbot234 6mo agoThere's at least a decent argument to be made that the limiting factor is actually the physical silicon itself (at least at cutting-edge nodes) not really the power. This actually gives AI labs an incentive to run those specific chips somewhat cooler, because high device temperatures and high input voltages (which you need to push frequencies higher) might severely impact a modern chip's reliability over time.
- aurareturn 6mo agoPower is the limiting layer above physical chips. You can add more chips and them at lower clock or add more efficient chips later on, but you can't really change the power of a data center easily.
- zozbot234 6mo agoIt will nonetheless be vastly cheaper to build a new datacenter and arrange for powering it than to fab the amount of leading-edge chips and compute systems that are going to ultimately eat that power. So the chips themselves are still the meaningful constraint.
- Eufrat 6mo agoI feel like that’s a bit glib? Surely, there should be some more critical questions posed by why just buying a bunch of GPUs is a good idea? It just feels like a cheap way to show that growth is happening. It feels a bit much like FOMO. It feels like nobody with the capital is questioning whether this is actually a good idea or even a desirable way to improve AI models or even if that is money well spent. 1 GW is a lot of power. My understanding is that it is the equivalent to the instantaneous demand of a city like Seattle. This is absurd. It feels like there is some awareness that asking for gigawatts if not terrawatts of compute probably needs more justification than has been proffered and the big banks are already trying to CYA themselves by publishing reports saying AI has not contributed meaningfully to the economy like Goldman Sachs recently did.
- NoahZuniga 6mo agoIt's more meaningful to most people than FLOPS/other measures of actual computing power.
- jeffbee 6mo agoIt's easy to think about. Google reported a global average power consumption of 3.7GW in 2024, so you can think of this deal as representing an expansion of something like 10-15% of that 2024 baseline, if you assume 50% capacity utilization.
- kumarvvr 6mo agoBecause all the variables that go into performance / efficiency measurement of a model (processing power, algorithm efficiency, parallelization, etc) boil down to cost per token input and token output. And the tangible cost for a datacenter is power consumed. Of course, amortized capex costs are also part of the game.
- levocardia 6mo agoSome of it might be market-signaling to the broader energy industry: "hey would you PLEASE build more power plants and power lines? Look at all this money we have, we will pay for it!"
- hgoel 6mo agoMaybe it's just because the specifics on FLOPs are more complicated, especially given how many different floating point formats are floating around in ML. Even NVIDIA has like 6 different FLOPs numbers on their GPUs nowadays.
- OliverGuy 6mo agoAnd you know Nvidia can't be constent with one format for FLOPs within a single graph, 1,000,000x faster but comparing FP32 to FP8 or NVFP4 and acting like it's the same.