3 ms·
I believe that MemGPT makes AGI possible this year, if it hasn't already happened. Thanks to the structure inherent in writing, Word2Vec maps words into a cohe
by mikewarot 3y ago
I believe that MemGPT makes AGI possible this year, if it hasn't already happened.
Thanks to the structure inherent in writing, Word2Vec maps words into a coherent latent space.
Thanks to positional encoding, self attention mechanisms are enabled. Giving rise to next word prediction at superhuman levels in Transformer LLMs.
Thanks to synthetic data, you can train an LLM to follow directions.
Thanks to being able to train LLM s, you can generate synthetic examples of them successfully using a memory prosthesis.
They can be trained on those examples, and suddenly you have MemGPT which can be a packer[1].
It remains to be seen if a mapper can also be emulated by an LLM. I'm not sure how you could generate a ton of examples of finding an inconsistency between internal chunks of knowledge for it to train against.
[1] https://reciprocality.org/Reciprocality/r0/index.html#:~:text=Mapping%20versus%20Packing,large%20numbers%20of%20knowledge%20packets https://reciprocality.org/Reciprocality/r0/index.html#:~:tex....
- keepamovin 3y agoThis is excellent, thank you. I didn't know any of that, fascinating!