3 ms·
I've been waiting to see some paper that is like "a shallow tree of key/values is all you need" to tackle model plasticity. AI memory seems predominately a ten
by MrLeap 2y ago
I've been waiting to see some paper that is like "a shallow tree of key/values is all you need" to tackle model plasticity.
AI memory seems predominately a tension between compression and lookup speed.
These large vectors are keys for lookups, a language of expression, and a means of compression. Learning new things is always easier when you can map it back to something you already know. There's this page about forest fire simulations in a scientific computing book I read back in college more than a decade ago. I remember it viscerally because I've solved 100 different problems with that as its seed. I can barely remember anything else in the book. I don't remember it because I read it over and over, I remember it because it was useful and kept being useful.
If some new technique or idea is 90% similar to something I already know, I'll learn it easily. If it's 60%, I need to churn it around, put in a lot of learning effort. If it's 0%, it's noise from this angle.
> 4. Establishes meaningful links based on similarities
> 5. Enables dynamic memory evolution and updates
Wondering how much compression occurs in #5.
- manmal 2y agoFast forward and B+ trees end up making sense (again).