4 ms·
Interesting, and that requirement is repeated on the cloud deployment pages, even the unfinished ones where that is the only requirement listed so far. https://
by sp332 3y ago
Interesting, and that requirement is repeated on the cloud deployment pages, even the unfinished ones where that is the only requirement listed so far. https://docs.mistral.ai/category/cloud-deployment https://docs.mistral.ai/category/cloud-deployment I wonder if that sliding context window really blows up the RAM usage or something.
- sebzim4500 3y agoUnless I've misunderstood something, the sliding context window should decrease memory usage at inference compared to normal flash attention.