shreyansh26

5 ms·

1.
by shreyansh26 8mo ago
What’s covered: 1. Hierarchical scans: block-local scan → write block totals → scan totals → carry-in add 2. Single-pass scans: the "domino" idea, and why naive inter-block propagation can stall / deadlock without the right c
2.
by shreyansh26 8mo ago
A short deep-dive on Multi-Head Latent Attention (MLA) (from DeepSeek): intuition + math, then a walk from MHA → GQA → MQA → MLA, with PyTorch code and the fusion/absorption optimizations for KV-cache efficiency.
3.
Understanding Multi-Head Latent Attention (From DeepSeek)(shreyansh26.github.io)
2 points by shreyansh26 8mo ago|1 comments
4.
Deriving the gradient for the backward pass of Layer Normalization(shreyansh26.github.io)
3 points by shreyansh26 1y ago|0 comments
5.
GTC'25 Notes: CUDA Techniques to Maximize Memory Bandwidth – Part 1(shreyansh26.github.io)
1 points by shreyansh26 2y ago|0 comments
6.
FlashAttention in PyTorch(github.com)
2 points by shreyansh26 3y ago|1 comments
7.
by shreyansh26 3y ago
A simplified implementation of FlashAttention in PyTorch for educational purpose.
8.
by shreyansh26 3y ago
Haha, so all one has to do is to add some fancy terms like LLMs, vectors, hyper-fast and 300+ Github stars. What a a joke lol!
9.
Understanding FlashAttention(shreyansh26.github.io)
2 points by shreyansh26 4y ago|0 comments
10.
by shreyansh26 4y ago
OpenAI is actually the one which has made AI so open and accessible for all. Perhaps not open source but all the developers building cool AI products these days are mostly due to OpenAI.
11.
by shreyansh26 4y ago
That's helpful. Thanks!
12.
Ask HN: What are some good resources on Recommender Systems?
14 points by shreyansh26 4y ago|3 comments
13.
by shreyansh26 4y ago
Reproduced on Edge on Windows 11 as well. What a bug! Really need to know the root cause of this.
14.
by shreyansh26 5y ago
Someone post something about astronomy also please!