The Apple M2 Ultra memory bandwidth is 800GB/s so it's not a long way off 1TB/s. A100 goes up to 2TB/s in the largest configuration, and H100 claims 3TB/s per

The Apple M2 Ultra memory bandwidth is 800GB/s so it's not a long way off 1TB/s. A100 goes up to 2TB/s in the largest configuration, and H100 claims 3TB/s per GPU node. (These figures keep changing with new variants.) But you can buy several Mac Studios for the price :-) The real use of H100 is for training, as it can pool many GPUs together with a dedicated high bandwidth network. You can't do that with Mac Studios.