4 ms·
Flash attention was also a set of common techniques in other areas of optimized software, yet the big guys weren't doing the optimizations when it came out and
by cma 2y ago
Flash attention was also a set of common techniques in other areas of optimized software, yet the big guys weren't doing the optimizations when it came out and it significantly improved everything.
- whimsicalism 2y agoyes, i agree that low-level & infra work is where a lot of deepseek's improvement came from