4 ms·
Too bad it lacks even the streaming mode SVE2 found in M4 cores. If only Apple would provide a full SVE2 implementation to put pressure on ARM to make it non-op
by crest 2y ago
Too bad it lacks even the streaming mode SVE2 found in M4 cores. If only Apple would provide a full SVE2 implementation to put pressure on ARM to make it non-optional so AArch64 isn't effectively restricted to NEON for SIMD.
- vlovich123 2y agoThis is for AI which is going to benefit more from use of metal / NPU than SIMD.
- bigyabai 2y agoSure, but larger models that fit in that 512gb memory are going to take a long time to tokenize/detokenize without hardware-accelerated BLAS.
- microtonal 2y agoWhy would you need BLAS for tokenization/detokenization? Pretty much everyone still uses BBPE which amounts to iteratively applying merges. (Maybe I'm missing something here.)
- ryao 2y agoTokenization/detokenization does not use BLAS.
- stouset 2y agoHell I’m just sitting here hoping the future M5 adopts SVE. Not even SVE2.