3 ms·
> So the "Verified" part of "SWE Bench Verified" means.. not "Verified" at all. Seems on-brand for an LLM-related thing to claim that it has verified something
by jsheard 1y ago
> So the "Verified" part of "SWE Bench Verified" means.. not "Verified" at all.
Seems on-brand for an LLM-related thing to claim that it has verified something without actually checking.
- geekymartian 1y agothat was my exact thought. how fitting
- hhh 1y agoVerified has a completely different meaning for this, it's that the questions have verified valid solutions.