8 authors attached to this. > SimpleQA is a simple but challenging benchmark for evaluating the factuality of frontier models. A main limitation in SimpleQ

8 authors attached to this. > SimpleQA is a simple but challenging benchmark for evaluating the factuality of frontier models. A main limitation in SimpleQA is its scope—while SimpleQA is accurate it only measures factuality under the constrained setting of short, fact-seeking queries with a single, verifiable answer. Whether the ability to provide factual short answers correlates with the ability to write lengthy responses filled with numerous facts remains an open research question. OpenAI going to have some rounds of layoffs in the future.