> but wouldn’t you get much better bang for the buck with high end consumer GPUs from Nvidia It depends on the workload, but generally for large neural network

> but wouldn’t you get much better bang for the buck with high end consumer GPUs from Nvidia It depends on the workload, but generally for large neural network model tasks, no, while for most other accelerated tasks, yes. > Is there really no way to partition the workload to run with 16gb memory per card? With regards to the neural networks in the news, you can partition the workloads. The 8xH100 machines are also partitioned in the same way. Nonetheless the performance is very poor on the consumer cards, because they lack the very high speed interconnect of the H100 that makes partitioning performant. Additionally, the size of the RAM on the H100 is matched to the capabilities of the chip itself - it sits in at the peak of an upside down U of a performance curve - which is why it is not possible to "just" add more RAM to these chips.