2 ms·
Does this translate to other models or was whisper cherry picked due to it's serial nature and integer math? looking at https://github.com/ml-explore/mlx-examp
by tgtweak 3y ago
Does this translate to other models or was whisper cherry picked due to it's serial nature and integer math? looking at https://github.com/ml-explore/mlx-examples/tree/main/stable_diffusion https://github.com/ml-explore/mlx-examples/tree/main/stable_... seems to hint that this is the case:
>At the time of writing this comparison convolutions are still some of the least optimized operations in MLX.
I think the main thing at play is the fact you can have 64+G of very fast ram directly coupled to the cpu/gpu and the benefits of that from a latency/co-accessibility point of view.
These numbers are certainly impressive when you look at the power packages of these systems.
Worth considering/noting that the cost of m3 max system with the minimum ram config is ~2x the price of a 4090...
- densh 3y agoApple’s silicon memory is fast only in comparison to consumer CPUs that stagnated for ages with having only 2 memory channels which was fine in 4 core era but wakes no sense at all with modern core counts. Memory scaling on GPUs is much better, even on the consumer front.