2 ms·
It’s nonzero, because they carry state while performing inference, and in the surrounding processes like chain-of-thought and mixture-of-experts.
by layer8 5mo ago
It’s nonzero, because they carry state while performing inference, and in the surrounding processes like chain-of-thought and mixture-of-experts.
- knollimar 5mo agoI think they have working memory but not short term memory. I suppose that's pedantic or anthropomorphizing but it feels like I felt tbh