You can parallelize inference with much cheaper GPUs as well! I’ve got a 4060 ti 16gb, and I’m thinking about getting another. I previously specced out a clust

You can parallelize inference with much cheaper GPUs as well! I’ve got a 4060 ti 16gb, and I’m thinking about getting another. I previously specced out a cluster using multiple 3090s. At the time, the 3090s were going for $700 on eBay. They’re more than that now, but there’s no need to spend $4k per GPU at all.