4 ms·
Hi, one of the authors of this blog post (Horace He), along with Driss Guessous, Yanbo Liang, and Joy Dong. We’re quite happy with this abstraction - happy to
by chillee 2y ago
Hi, one of the authors of this blog post (Horace He), along with Driss Guessous, Yanbo Liang, and Joy Dong.
We’re quite happy with this abstraction - happy to answer any questions about it!
- doctorpangloss 2y ago[flagged]
- zaptrem 2y agoFor those of us using the 2D NATTEN kernel from their library along with torch.compile, is this faster? Especially given all their tricks (e.g., the non-deterministic KV-parallelism)
- chillee 2y agoIn my (very amateurish) testing, I think the performance seemed pretty comparable (for non-dilated natten). I need to do some proper benchmarking though!
- imjonse 2y agoIs this for Ampere and newer only as FA2?
- chillee 2y agoI believe it should run on V100 as well (although definitely not tested as well), and an user reported that they got it running on T4 too.