3 ms·
There are specialized architectures (the Tolman-Eichenbaum Machine)* that are able to complete this kind of task. Interestingly, once trained, their activations
by nomadpenguin 1y ago
There are specialized architectures (the Tolman-Eichenbaum Machine)* that are able to complete this kind of task. Interestingly, once trained, their activations look strikingly similar to place and grid cells in real brains. The team were also able to show (in a separate paper) that the TEM is mathematically equivalent to a transformer.
* https://www.sciencedirect.com/science/article/pii/S009286742031388X https://www.sciencedirect.com/science/article/pii/S009286742...
- Mallowram 1y ago[dead]
- nerpderp82 1y agoRelating transformers to models and neural representations of the hippocampal formation https://www.semanticscholar.org/paper/Relating-transformers-to-models-and-neural-of-the-Whittington-Warren/06e40b7a703079c280f8f0886ac2bd984cd318ce https://www.semanticscholar.org/paper/Relating-transformers-... by the same authors as the TEM paper