3 ms·
Not sure why you are being downvoted, we already know the performance numbers due to memory bandwidth constraints on the M4 Max chips, it would apply here as we
by bearjaws 2y ago
Not sure why you are being downvoted, we already know the performance numbers due to memory bandwidth constraints on the M4 Max chips, it would apply here as well.
525GB/s to 1000GB/s will double the TPS at best, which is still quite low for large LLMs.
- lanceflt 2y agoDeepseek R1 (full, Q1) is 14t/s on an M2 Ultra, so this should be around 20t/s