2 ms·
With LLAMA at least, not just past tokens, but also past states for each layer and each token. So LLama has information not just on the word "Berlin", but also
by kolinko 3y ago
With LLAMA at least, not just past tokens, but also past states for each layer and each token. So LLama has information not just on the word "Berlin", but also on why it outputted "Berlin".
- YetAnotherNick 3y agoThat is called caching, which is used to save computation. But only tokens are enough to calculate next token and state could be recomputed.