3 ms·
252GB of HBM3e at 100k vs. a multi A6000 96GB setup. GB300 seem expensive in comparison at ~$100k. Am I missing something?
by dsrtslnd23 21d ago
252GB of HBM3e at 100k vs. a multi A6000 96GB setup.
GB300 seem expensive in comparison at ~$100k. Am I missing something?
- _diyar 21d agoWithout looking it up and doing the math, I bet GB300 has higher Tflops and memory bandwidth, especially when used with e.g. NVFP4.
- hgoel 21d agoPlus lower peak power draw
- rbanffy 21d agoHow long until the higher power draw nullifies the higher acquisition price of the GB300 machine?
- cyanydeez 21d agoAre you in a world where energy prices are goimg down?
- rbanffy 21d agoQuite possibly, depending on grid-scale renewable deployment. I also am about to install a set of solar panels, so a base level of power will cost me only the depreciation of the PV hardware. We could condition datacenter installs to providing power to the grid - you want to build a datacenter, you also need to build a wind or solar farm that can fully power its peak load.
- cyanydeez 21d agoAmerica has decided the answer is no.
- nutjob2 21d agoYes, which planet are you on? Australia has recently lowered power prices due to renewables, probably other places will too for similar reasons as the rollout continues.
- cyanydeez 21d agoCountry: America, where we pay people not to build windmills.
- LargoLasskhyfv 21d agoHP’s spec sheet fills in details the platform announcements skipped. The CPU memory is four 128GB SOCAMM modules delivering 396GB/s, and the Grace CPU is soldered to the host processor module rather than socketed. The two pools add up to the 748GB coherent space that lets the GPU address CPU memory directly, which is what makes trillion-parameter inference and fine-tuning of models in the 100 billion parameter class possible on a single box. HP’s footnote on those model sizes is that the harness quantizes at FP4.
- cmrdporcupine 21d agoThere's a lot more going on here because the host machine has a boatload (496GB) of expensive LPDDR5x (which is stupidly expensive) that can also be used as unified (but slower) memory to the GPU, 72 ARM64 cores, stupid fast NVlink/QSFP networking, the PSU to support all that etc. etc. Basically it's the same as a tray in a GB300 NVL72, but in workstation/desktop form. Niche would be AI researchers. They will... not sell a lot of these. But what a beast. I am too lazy to price out 496GB of DDR5 but, um, mostly because it is terrifying to see what today's prices look like. You can't build such a machine out yourself but if you could I suspect the price point would come out about the same.
- rbanffy 21d ago> You can't build such a machine out yourself but if you could I suspect the price point would come out about the same. That's kind of the nature of capitalism and price elasticity - the manufacture price only limited the minimum sale price, and sale price usually reflects how much is the market willing to pay for the good. Where you might save a lot of money is on building something that's very targeted to your needs that matches them better than a GB300 workstation, for a lower price.
- cmrdporcupine 21d agoBut the point of such a machine generally is to have the equivalent of a GB300 NVL72 tray on your desk. It's so that you can do the work that belongs in a production DC eventually. Or at least that's now NVIDIA would like you to use it. You could build out a complicated multi GPU setup of your own, but the work you do on inference tuning for your kernels etc would not necessarily translate to the real world. But yes, if you just want to run GLM 5.3 on your own machine, that's a whole other story.
- rbanffy 20d ago> Or at least that's now NVIDIA would like you to use it. Exactly. It's tailor-built to be a GB300 tray under your desk. Unless you really need a GB300 tray under your desk, you would be better served looking elsewhere.
- segmondy 21d agohave you priced ddr5 memory?