2 ms·
The latest DeepSeek model has sparse attention. Though sparse attention is still not linear. Close enough perhaps.
by cubefox 8mo ago
The latest DeepSeek model has sparse attention. Though sparse attention is still not linear. Close enough perhaps.