5 ms·

On a more serious note, I do agree with you that memory and self-excitation seem like they are the last push thats needed to get to something more akin to "AGI"

• MacsHeadroom 4y ago
  >I do agree with you that memory and self-excitation seem like they are the last push thats needed to get to something more akin to "AGI" "We show that transformer-based large language models are computationally universal when augmented with an external memory. Any deterministic language model that conditions on strings of bounded length is equivalent to a finite automaton, hence computationally limited. However, augmenting such models with a read-write memory creates the possibility of processing arbitrarily large inputs and, potentially, simulating any algorithm." From "Memory Augmented Large Language Models are Computationally Universal" https://deepai.org/publication/memory-augmented-large-language-models-are-computationally-universal https://deepai.org/publication/memory-augmented-large-langua...