3 ms·
Of all the models to perform such a comparison, Flan-T5-XL, a 3B parameter model from 2022 is a genuinely baffling choice. The paper is nearly worthless for thi
by famouswaffles 3y ago
Of all the models to perform such a comparison, Flan-T5-XL, a 3B parameter model from 2022 is a genuinely baffling choice. The paper is nearly worthless for this reason alone.
- pilipolio 3y agoIt seems the authors implicitly chose the Flan-T5-XL model based on a previous paper they refer where it outperformed prompted engineered ChatGPT 3.5 and 4 on a slightly different but similar task. This should definitely need to be further explained and possibly confirmed on their tasks. "Our best-performing fine-tuned models outperformed zero- and few-shot performance of ChatGPT-family models [...]" from Large language models to identify social determinants of health in electronic health records (https://arxiv.org/abs/2308.06354 https://arxiv.org/abs/2308.06354)
- zwaps 3y agoFlan-T5 is actually not a bad model, even today (for fine-tuning)
- famouswaffles 3y agoI don't think it's a bad model. It's just strange to use it as the ambassador for LLM performance as it's nowhere near SOTA.