3 ms·
MLX does not support dual chunk attention[1] that these models use for long contexts, yet. [1]: https://arxiv.org/abs/2402.17463 https://arxiv.org/abs/2402.174
by woadwarrior01 2y ago
MLX does not support dual chunk attention[1] that these models use for long contexts, yet.
[1]: https://arxiv.org/abs/2402.17463 https://arxiv.org/abs/2402.17463