3 ms·

Because that's the main constraint for building them - how much power can you get to the site, and the cooling involved. Also the workloads completely change o

Because that's the main constraint for building them - how much power can you get to the site, and the cooling involved. Also the workloads completely change over time as racks get retired and replaced, so it doesn't mean much. But you can basically assume with GB200s right now 1GW is ~5exaflops of compute depending on precision type and my maths being correct!

sedawkgrep 1y ago
[flagged]
jauntywundrkind 1y ago
Yes! The varying precisions and maths feels like just the start! Look at next gen Rubin with it's CPX co-processor chip to see things getting much weirder & more specialized. There for prefilling long contexts, which is compute intensive: > Something has to give, and that something in the Nvidia product line is now called the "Rubin" CPX GPU accelerator, which is aimed specifically at parts of the inference workload that do not require high bandwidth memory but do need lots of compute and, increasingly, the ability to process video formats for both input and output as part of the AI workflow. https://www.nextplatform.com/2025/09/11/nvidia-disaggregates-long-context-inference-to-drive-bang-for-the-buck/ https://www.nextplatform.com/2025/09/11/nvidia-disaggregates... To confirm what you are saying, there is no coherent unifying way to measure what's getting built other than by power consumption. Some of that budget will go to memory, some to compute (some to interconnect, some to storage), and it's too early to say what ratio each may have, to even know what ratios of compute:memory we're heading towards (and one size won't fit all problems). Perhaps we end up abandoning HBM & dram! Maybe the future belongs to high bandwidth flash! Maybe with it's own Computational Storage! Trying to use figures like flops or bandwidth is applying today's answers to a future that might get weirder on us. https://www.tomshardware.com/tech-industry/sandisk-and-sk-hynix-join-forces-to-standardize-high-bandwidth-flash-memory-a-nand-based-alternative-to-hbm-for-ai-gpus-move-could-enable-8-16x-higher-capacity-compared-to-dram https://www.tomshardware.com/tech-industry/sandisk-and-sk-hy...
udkl 1y ago
As a reference for anyone interested - the cost is estimated to be $10 billion for EACH 500MW data center - this includes the cost of the chips and the data center infra.
- ponector 1y ago
  With such price tag the power plant should be included.

sedawkgrep 1y ago

[flagged]

jauntywundrkind 1y ago

Yes! The varying precisions and maths feels like just the start! Look at next gen Rubin with it's CPX co-processor chip to see things getting much weirder & more specialized. There for prefilling long contexts, which is compute intensive: > Something has to give, and that something in the Nvidia product line is now called the "Rubin" CPX GPU accelerator, which is aimed specifically at parts of the inference workload that do not require high bandwidth memory but do need lots of compute and, increasingly, the ability to process video formats for both input and output as part of the AI workflow. https://www.nextplatform.com/2025/09/11/nvidia-disaggregates-long-context-inference-to-drive-bang-for-the-buck/ https://www.nextplatform.com/2025/09/11/nvidia-disaggregates... To confirm what you are saying, there is no coherent unifying way to measure what's getting built other than by power consumption. Some of that budget will go to memory, some to compute (some to interconnect, some to storage), and it's too early to say what ratio each may have, to even know what ratios of compute:memory we're heading towards (and one size won't fit all problems). Perhaps we end up abandoning HBM & dram! Maybe the future belongs to high bandwidth flash! Maybe with it's own Computational Storage! Trying to use figures like flops or bandwidth is applying today's answers to a future that might get weirder on us. https://www.tomshardware.com/tech-industry/sandisk-and-sk-hynix-join-forces-to-standardize-high-bandwidth-flash-memory-a-nand-based-alternative-to-hbm-for-ai-gpus-move-could-enable-8-16x-higher-capacity-compared-to-dram https://www.tomshardware.com/tech-industry/sandisk-and-sk-hy...

udkl 1y ago

As a reference for anyone interested - the cost is estimated to be $10 billion for EACH 500MW data center - this includes the cost of the chips and the data center infra.

ponector 1y ago
With such price tag the power plant should be included.