3 ms·
The entire industry is so desperate to anthropomorphize. What the paper describes is an offline recurrent consolidation phase: the model runs multiple forward p
by IAmGraydon 5mo ago
The entire industry is so desperate to anthropomorphize. What the paper describes is an offline recurrent consolidation phase: the model runs multiple forward passes over recently accumulated context, updates persistent fast weights in SSM blocks, then clears the KV cache before continuing. It has absolutely nothing to do with sleeping, but I believe the authors had a goal in mind when creating this title, and it was for journalists to pick it up and run with it, further inflating the AI-is-just-like-us hype bubble.
- genxy 5mo agoIt is a descriptive analogy, get over yourself.
- IAmGraydon 5mo agoAn intelligent reply from an obviously intelligent guy! A more appropriate title would have been something like "Offline Recurrent Memory Consolidation for Long-Context Language Models". This is supposed to be a research paper, not a story book. The title should give context to other researchers, and not be clearly engineered for clicks. If you don't think so, that's your prerogative, but you're objectively wrong.
- genxy 5mo agoYou write the paper, you write the title. So much anger over a title, you are graydon, make this about yourself.