3 ms·
I’m afraid the article as well as your benchmarks can be misleading because there are a lot of different whisper implementations out there. For ex ctranslate o
by WhitneyLand 3y ago
I’m afraid the article as well as your benchmarks can be misleading because there are a lot of different whisper implementations out there.
For ex ctranslate optimized whisper “implements a custom runtime that applies many performance optimization techniques such as weights quantization, layers fusion, batch reordering…”
Intuitively, I would agree with your conclusion about Apple’s M-Series being impressive for what they do but not generally competitive with Nvidia in ML.
Objectively however, I don’t see concluding much with what’s on offer here. Once you start changing libraries, kernels, transformer code, etc you end up with an apples to oranges comparison.
- modeless 3y agoI think it's fair to compare the fastest available implementation on both platforms. I suspect that the MLX version can be optimized further. However, it will not close a 10x gap.