4 ms·

Thanks! I think computers are fun and I want reading about them to be fun too. I was also reminded of HazyResearch's MegaKernels. Didn't want to distract from

Thanks! I think computers are fun and I want reading about them to be fun too. I was also reminded of HazyResearch's MegaKernels. Didn't want to distract from the main thrust of the post, but definitely think that's a promising approach.

emaadm 1y ago
There's some interesting work in NeurIPS this year on fused kernels for MoE too: https://flash-moe.github.io/ https://flash-moe.github.io/