3 ms·
Evaluating Factual Consistency of Summaries with Large Language Models
- belter 3y ago"...Based on the previous comparisons, we summarize that text-davinci-003 and GPT-4 are two best models for the factual consistency task, while being less sensitive to the availability of exemplars. On the other hand, code-davinci-002 requires providing a few demo examples to potentially work well. Importantly, Flan-T5 achieves surprising results in general – under a zero-shot setting in SummEval and XSumSota, Flan-T5 not only beats all the baselines, but also outperforms both the GPT-3.5 variants that are orders of magnitude larger..."