If it's really on-prem (i.e. not in the "datacenter" as per NVIDIA EULA), you could spend a lot less than $100K+ for a lot more throughput by purchasing consume

If it's really on-prem (i.e. not in the "datacenter" as per NVIDIA EULA), you could spend a lot less than $100K+ for a lot more throughput by purchasing consumer-grade cards and HEDT gaming hardware. Sure you'll have 4 GPUs per box and not 8, and sure, each GPU will have 11GB and not 32, but the whole machine (_with_ the GPUs) will cost just a tad more than a single V100. So if you don't really need 32GB of VRAM per GPU (and you most likely don't), it'd be insane to pay literally 5x as much as you have to.