4 ms·
I’m curious where you get the estimate that humans hold “like four” parameters in their mind at once?
by bumby 2mo ago
I’m curious where you get the estimate that humans hold “like four” parameters in their mind at once?
- rDr4g0n 2mo agoupdated the comment. i meant four "concepts". i dont reason about my own thinking in terms of parameters.
- btown 2mo agoCowan (2001) is an oft-cited paper proposing three to five "chunks" of capacity in human attention: https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/magical-number-4-in-shortterm-memory-a-reconsideration-of-mental-storage-capacity/44023F1147D4A1D44BDC0AD226838496 https://www.cambridge.org/core/journals/behavioral-and-brain... The full PDF is worth a read (Figure 1 may be of interest to many here): https://www.cambridge.org/core/services/aop-cambridge-core/content/view/44023F1147D4A1D44BDC0AD226838496/S0140525X01003922a.pdf/the-magical-number-4-in-short-term-memory-a-reconsideration-of-mental-storage-capacity.pdf https://www.cambridge.org/core/services/aop-cambridge-core/c... If "attention is all you need" then it's something we do indeed lack, in comparison to LLMs! But it's an interesting question: might machine cognition benefit from similar bottlenecks in an attention algorithm? Advancements like Kimi Linear seem to indicate that we're far from the finish line: https://arxiv.org/abs/2510.26692 https://arxiv.org/abs/2510.26692