5 ms·
Also, Datacenter GPUs only have passive cooling, presumably to allow for more Cuda cores. I get that what I have is older tech but for the cost of one H100 GPU
by dougSF70 3y ago
Also, Datacenter GPUs only have passive cooling, presumably to allow for more Cuda cores. I get that what I have is older tech but for the cost of one H100 GPU I can have 60 of these servers (600 GPUs) plus some change to pay for CoLo fees.
- crabbone 3y agoI'm not trying to discourage, and I don't have concrete numbers on hand, but there are other factors, beside the price of h/w. Like, obviously, electricity use, data locality / moving it around (with more smaller units you'd have to move it more, also, not sure if consumer-grade GPUs support NVLink, and even if they do, then at what bandwidth?) For some of these, you could obviously pay with your time. Sometimes that time is very valuable, and sometimes you have a lot to spare. Also, the amount of VRAM (3x)... Sometimes having too little of it means having to re-write the program, or it could dramatically impact the speed. Similarly for bandwidth (8x). But, again, if the kind of workload you have isn't constrained by either, then you could probably win by running on more smaller / older GPUs.