4 ms·
Sparse transformers are a thing. Dictionary lookup is a thing. The transformer can probably be trained to store long chains of information in a dedicated memory
by FeepingCreature 5y ago
Sparse transformers are a thing. Dictionary lookup is a thing. The transformer can probably be trained to store long chains of information in a dedicated memory system, retrieving it using keywords.
These are the things that came to mind in ten seconds.
This is not going to be the problem that meaningfully delays AGI.
- whimsicalism 5y agoAs I said in another comment, I think memory will be the easiest of these challenges to solve. That said, it hasn't really been solved yet. If we can scale up S4-style architectures to this size, maybe it will be solved.