3 ms·
I think using AE for text generation is a good idea, and is pretty old one (I tried it myself back in 2015 without particularly good results), but I wouldn't ca
by p1esk 6y ago
I think using AE for text generation is a good idea, and is pretty old one (I tried it myself back in 2015 without particularly good results), but I wouldn't call it a breakthrough. To me a breakthrough/new paradigm would be something like this: https://arxiv.org/abs/1906.05317 https://arxiv.org/abs/1906.05317
- The_rationalist 6y agoThe breakthrough is not that they are the first to use AE for text. It is that they are the first to eliminate the bottleneck that prevented AEs to beat transformers. You linked paper seems interesting on paper but does it bring any new SOTA?
- p1esk 6y agoAEs to beat transformers Can you point me to some example of generated text this model produced? Something similar in quality to that unicorn story from GPT-2? but does it bring any new SOTA? Looking at their tables, seems so. The code is open source, and there's demo at https://mosaickg.apps.allenai.org/ https://mosaickg.apps.allenai.org/