3 ms·
This is for AI which is going to benefit more from use of metal / NPU than SIMD.
by vlovich123 2y ago
This is for AI which is going to benefit more from use of metal / NPU than SIMD.
- bigyabai 2y agoSure, but larger models that fit in that 512gb memory are going to take a long time to tokenize/detokenize without hardware-accelerated BLAS.
- microtonal 2y agoWhy would you need BLAS for tokenization/detokenization? Pretty much everyone still uses BBPE which amounts to iteratively applying merges. (Maybe I'm missing something here.)
- ryao 2y agoTokenization/detokenization does not use BLAS.