4 ms·
The article explains that this is not a problem but an advantage.
by cubefox 1y ago
The article explains that this is not a problem but an advantage.
- Sabinus 1y agoApart from execution speedup due to caching, how does the article explain this is an advantage?
- mattnewton 1y agoI wouldn't agree it's an advantage per se, just that the caching workaround works better than you would expect! It's still quite slow to generate each token at a time versus diffusing batches, it's just hard to cache all the work you do in each encoder step.