4 ms·
The Sparsity Nexus: Bypassing O(N²) Attention with Judy Arrays
- elena__s 7mo agoThis is an interesting direction, especially the idea of treating attention as an indexed lookup. Would be great to see benchmarks or real-world use cases.
- lunarain 7mo agoGlad you found the lookup approach interesting! We don't have public benchmarks to share just yet, as the focus right now is on the architectural grounding. A lot of 'new' tech is built on fundamentals that haven't actually changed, but we often ignore existing solutions to memory and sparsity because they aren't 'trendy.' This is an attempt to stop ignoring those 'good ol' problems' and apply proven data structures to the attention bottleneck.