3 ms·
Did you read the paper? https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf https://d4mucfpksywv.
by p1esk 7y ago
Did you read the paper? https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf https://d4mucfpksywv.cloudfront.net/better-language-models/l...
You don’t need any seed, and can generate summaries (section 3.6).
GPT-2 is the model to learn about if you’re interested in NLP.
- applecrazy 7y agoAre there any papers benchmarking a transformer NN architecture in comparison to something like a pointer-generator network? I'm doing a bit of work in this area (i.e. reimplementing papers), and I'm curious if GPT2-like models can derive greater semantic meaning.
- p1esk 7y agoBoth GPT-2 and pointer-generator network are open source, and pretrained models are available, so it should be straightforward to compare them.