4 ms·
Visualizing Transformer Language Models: Illustrated GPT-2
- somebodythere 7y agoMy understanding of state-of-the-art language models is limited, but it seems like there has been a lot invested into having models that generate their output one token at a time, no edits. Have there been any experiments into models that can re-read and edit their work, before spitting out entire sentences, paragraphs, or documents?
- p1esk 7y agohttps://arxiv.org/abs/1811.04454 https://arxiv.org/abs/1811.04454
- ReDeiPirati 7y agoThis is now my second-best reading on GPT-2 after this one: https://blog.floydhub.com/gpt2/ https://blog.floydhub.com/gpt2/.