3 ms·
A cache is just a cache. I'm not sure what significance you're ascribing to it.
by wizzwizz4 1mo ago
A cache is just a cache. I'm not sure what significance you're ascribing to it.
- user43928 1mo agoWhat is put in the cache?
- wizzwizz4 1mo agoThings that the software running the model would otherwise recompute, if not for the cache. What special meaning are you assigning to it?
- user43928 1mo agoBeats me how it works, honestly can't wrap my head around it. From what I understand, at position Aha in each layer it's constructing a query based on the current activation and looking at the key of each other token position for that layer, in order to decide how much attention to pay to the value. In this way it attends to the previous values, such as perhaps the incorrect assumption and plausible explanation.