2 ms·
It should be noted that per the included Jupyter Notebook, the resulting network took 24 hours to train, and the resulting network (of a 3-layer 512 cell LSTM)
by minimaxir 9y ago
It should be noted that per the included Jupyter Notebook, the resulting network took 24 hours to train, and the resulting network (of a 3-layer 512 cell LSTM) would be hundreds of MB. (too much to share on GitHub anyways).
The approach in the repo also uses a word-based approach instead of a character-based approach. Although a word-approach avoids typoes/nonsense, it makes prediction much more difficult since the selection space is harder. (from notebook, error is high as well). There are lots of trade offs in this field.
Training these networks is not for the faint of heart and one of the reasons I developed a tool to train text generators using a pretrained RNN (https://github.com/minimaxir/textgenrnn https://github.com/minimaxir/textgenrnn) on a much more simple network architecture that trains quickly, although it probably would not hit the depth of all of Game of Thrones.