3 ms·
textgenrnn (https://github.com/minimaxir/textgenrnn https://github.com/minimaxir/textgenrnn) uses a simple Attention Weighted Average at the end of the model fo
by minimaxir 8y ago
textgenrnn (https://github.com/minimaxir/textgenrnn https://github.com/minimaxir/textgenrnn) uses a simple Attention Weighted Average at the end of the model for text generation, which in my testing allows the model to learn much better.