3 ms·
I played around with this on the demo page and found that while the generated "TLDR" are pretty good, they tend to generate sentences composed of fragments of e
by etaioinshrdlu 6y ago
I played around with this on the demo page and found that while the generated "TLDR" are pretty good, they tend to generate sentences composed of fragments of existing sentences. Basically, it seems vaguely extractive in nature. Never did I see it summarize a concept in new words, or try to dumb down a complicated concept further than the original paper did. Given the results of GPT3 I would think that it should be possible to do much better by now, at least with enough data and compute time.
- arolihas 6y agoI think you’re underestimating how hard what you’re describing is. GPT-3 can mimic the language of reasoning but that doesn’t mean it’s capable of higher order reasoning.
- FL33TW00D 6y agoAfter reading through https://www.gwern.net/GPT-3 https://www.gwern.net/GPT-3 , I suspect that GPT-3 is capable of higher order reasoning, given the right motivation (prompt).
- arolihas 6y agoIt’s impressive but doesn’t the need for a “good” prompt kind of show it doesn’t have the strong reasoning required to do the task you’re describing? Also there’s some interesting critique here https://www.lesswrong.com/posts/ZHrpjDc3CepSeeBuE/gpt-3-a-disappointing-paper https://www.lesswrong.com/posts/ZHrpjDc3CepSeeBuE/gpt-3-a-di...
- msamwald 6y agoBeing very efficient at mostly extractive summarization and abstaining from abstractive summarization does seem a better bet though, because fewer things can go wrong and it is easier to check the summaries against the full text.