7 ms·
The context window has been shattered: Subquadratic debuts a 12M token window
- Bombthecat 6mo agoI believe it, when I see it.
- roger_ 6mo agoHave they published?
- refibrillator 6mo agoPrevious discussion here (with links to actual primary source): https://news.ycombinator.com/item?id=48023079 https://news.ycombinator.com/item?id=48023079 No technical report published yet, unlikely code or weights will be either given VC funding.
- Chance-Device 6mo agoIt’s probably something like deepseek’s native sparse attention with content based granularity. They might not be publishing anything because it’s not such a strong value proposition and doing so would lead to commentary that would tank their investment opportunities.
- SilverElfin 6mo agoOr maybe because giving it away would tank their investment opportunities.
- regularfry 6mo agoThere's ways and means. Pushing something out in the sub-30B range would gain them mindshare and they could keep bigger models to themselves. I can't see any indication of what size their model is though.
- dfilppi 6mo ago[dead]
- flowerthoughts 6mo agoFor Claude Code, I feel 1M is enough. I've had a compaction once, but that was because I was forcing Claude to do something it clearly had a hard time understanding. For general chat bots where the user doesn't understand what a context window is, what do you do about context? Latest few messages and then a memory tool? Compaction?
- brianush1 6mo agoClaude does compaction in the regular web chat interface now, too
- whoodle 6mo agoThis is so interesting to me, I frequently experience compaction on long running features and still find Claude is better than starting with fully fresh prompts. Every dev seems to use these tools differently.
- anon22981 6mo agoI feel the 1M context is way too large —- the model gets ”drunk” way before it gets anywhere near 1 million. Imo the 1M context window is a huge downgrade.
- vagrantJin 6mo agoI use a tool called context-mode. Update agent to save session summary every 100k tokens
- t4sty 6mo agoFeels like this is to AI what JPG is to image
- Alifatisk 6mo agoWaiting for the paper and model card. I believe it when I see it.