44 ms·
PSA: This is broken for Apple silicon. It relies on the CUDA-only `flash_attn` package. Alas, I wanted to try Mistral's MoE on my MacBook. :( Understandable tho
by a_wild_dandan 3y ago
PSA: This is broken for Apple silicon. It relies on the CUDA-only `flash_attn` package. Alas, I wanted to try Mistral's MoE on my MacBook. :( Understandable though! It's a new model with an incredibly bizarre release. My phalanges are crossed.
- king_magic 3y agoI think "broken" is a bit of an exaggeration, when the actual problem is "just not compatible with your hardware".