4 ms·
That's true, and you do make a disclaimer about performance, but often when this is brought up, people either haven't actually tried MPS or are unaware that a G
by washadjeffmad 3y ago
That's true, and you do make a disclaimer about performance, but often when this is brought up, people either haven't actually tried MPS or are unaware that a GPU isn't required for inference.
The long and short of it is, right now, CUDA beats TFLOPS.
Now for training, $10K for an M2 with 192GB memory could break even somewhere around a dozen models with current costs, iff the compute is not saturated. The limit of memory bandwidth is only a function of the compute; it doesn't scale infinitely.