3 ms·
I don't think anyone argues that transformers haven't revolutionised text generation. The real question is how good this text generation generalises to other l
by disgruntledphd2 5y ago
I don't think anyone argues that transformers haven't revolutionised text generation.
The real question is how good this text generation generalises to other language tasks. That's the more interesting one to me, at least.
- moconnor 5y agoTransformers are state-of-the-art for other language tasks - sentiment analysis, question and answering, document summarisation, translation etc. The shocking thing about GPT-3 was that it was close-ish to state of the art on many language tasks without training for them , using only a prompt illustrating the task was enough. InstructGPT takes this further.
- disgruntledphd2 5y agoSee this I'm not convinced about. Having tested a bunch of hugging face models for sentiment analysis, the results are very very poor. We see so many uses of text generation, but if document summarization is so good then where are all the startups doing this for money? I agree that GPT et Al perform well on the benchmarks, but that's not the same thing at all.