4 ms·
They're probably taking shortcuts such as taking advantage of sparsity. There are various tricks like that mentioned in some papers, although the big companies
by FartyMcFarter 7mo ago
They're probably taking shortcuts such as taking advantage of sparsity. There are various tricks like that mentioned in some papers, although the big companies are getting more and more secretive about how their models work so you won't necessarily find proof.
- cubefox 7mo agoThe latest DeepSeek model has sparse attention. Though sparse attention is still not linear. Close enough perhaps.