The quality difference does matter. For a while I ran an AI app with summary features, with T5/BART in 2020 and then generative from 2021 (using GPT-3 API). It

The quality difference does matter. For a while I ran an AI app with summary features, with T5/BART in 2020 and then generative from 2021 (using GPT-3 API). It's an unfortunate truth that when I tried using Babbage (GPT-2 size), it was ok for happy path examples, but user retention crashed. Even Curie was much worse than Davinci in terms of user satisfaction. One interesting finding from my data was that shorter outputs had less difference in quality between models - longer responses, it mattered more which model you used (this was before Chat, and even before instruct/RLHF GPT models, so before the constrained output forms of verbosity conversational defaults) For AI applications, users have a 1 strike policy in general for trust. So you kind of need to overdial things to retain users. Might be different for dataset work, but there absolutely is a quality difference over enough uses. BART or T5 were "better" than barebones GPT-2 for summarisation initially, but that does not mean anybody really uses them for summarisation (in a sense, the task they were most optimised to do during creation). I do think most of us feel summaries from latest models are getting to as good as is reasonably expectable. So sure if GPT-5 comes and is particularly great at remaining tricky tasks, why not stick with 4 for summaries. That's not to mention context length is longer in terms of easy "do it today" ways to access a model for ChatGPT or GPT-4, vs GPT-2 models which are much shorter context lengths in terms of input.