4 ms·
Thank you very much, that's great. So there is a mutable working memory for the session that's persistent through the session. That's something that had been mi
by simonh 4y ago
Thank you very much, that's great. So there is a mutable working memory for the session that's persistent through the session. That's something that had been missing from previous explanations I'd seen.
- GistNoesis 4y agoThis mutable working memory is implicit. It comes from the transformer architecture. What the neural network weight are encoding is the dynamic of this implicit working memory such that it produces the desired output. If you use something like LSTM instead of transformers, it's easier to see the memory cells. But "transformers are RNNs" even if it's a little harder to see.