5 ms·
An associative memory can model truth tables of logic circuits, and so the steps to go from associative memory to arbitrary computation pretty much requires add
by vidarh 3y ago
An associative memory can model truth tables of logic circuits, and so the steps to go from associative memory to arbitrary computation pretty much requires adding state and a loop. And we do provide state and a loop. As such, while that does not prove that any given model is capable of reasoning, describing what goes on at any step as associative memory says nothing meaningful about the computational power of the wider system.
It's very possible the way we train current models will turn out to place fundamental limits on the abilities the models will have, but that will not be because the models act as associative memories.
- ein0p 3y agoI think the current models will be a fundamental part of larger, more comprehensive models, and they will handle the “easy” parts, much like we, humans, use our associative memory to be able to do things in realtime, “without thinking”. Yann LeCun wrote a paper on this which more people should read: https://openreview.net/pdf?id=BZ5a1r-kVsf https://openreview.net/pdf?id=BZ5a1r-kVsf. Moreover, Meta’s research is starting to move in that direction https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-joint-embedding-predictive-architecture/ https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-jo...