3 ms·
The 4k sliding window context seems like a controversial choice after Mistral 7B mostly failed at showing any benefits from it. What was the rationale behind th
by moffkalast 2y ago
The 4k sliding window context seems like a controversial choice after Mistral 7B mostly failed at showing any benefits from it. What was the rationale behind that instead of just going for full 8k or 16k?
- alekandreev 2y agoThis is mostly about inference speed, while maintaining long context performance.