3 ms·
> anyone got a guide on setting up and running the business-class stuff (70B models over multiple A100, etc)? If you need a guide, the A100 class cards are cer
by Aurornis 2y ago
> anyone got a guide on setting up and running the business-class stuff (70B models over multiple A100, etc)?
If you need a guide, the A100 class cards are certainly not what you want. Running data center level clusters in the home is no walk in the park. You wouldn’t even be able to power a decent sized cluster without getting an electrician involved for proper power. Networking is expensive. Even second-hand it all adds up fast. Don’t underestimate how much it would cost.
Getting some 3090s in a case with a big gaming PSU is the way to go for home use. If you really want A100 level, rent it from a cloud provider. Trying to build your own cluster would be like lighting money on fire for a rapidly depreciating asset.