6 ms·
SimpleQA
- websap 2y agoAre they going to make the benchmark available so other LLMs can be compared?
- abhisuri97 2y agohttps://github.com/openai/simple-evals/blob/main/simpleqa_eval.py https://github.com/openai/simple-evals/blob/main/simpleqa_ev...
- seany62 2y agoAny way to see the actual questions and answers? Where can I find simple_qa_test_set.csv ?
- sbierwagen 2y agohttps://openaipublic.blob.core.windows.net/simple-evals/simple_qa_test_set.csv https://openaipublic.blob.core.windows.net/simple-evals/simp... The steps I took to find this link: 1) Look at simpleqa_eval.py. See that it loads "az://openaipublic/simple-evals/simple_qa_test_set.csv" Hmm, some weird vendored protocol. 2) I don't feel like digging through bf.BlobFile() to figure out how it downloads files and I certainly don't want to generate an API key. Cross fingers and do a Bing web search for "az://openaipublic" 3) That leads me to https://stackoverflow.com/questions/76106366/how-to-use-tiktoken-in-offline-mode-computer https://stackoverflow.com/questions/76106366/how-to-use-tikt... Ah ha, this answer has the link https://openaipublic.blob.core.windows.net/encodings/cl100k_base.tiktoken https://openaipublic.blob.core.windows.net/encodings/cl100k_... which automatically downloads a file. 4) Poke the relevant parts of the az:// link into this link, and a csv appears.
- _jonas 2y agoIt's easier to find the data now, I've run some benchmarks on it. Great to see OpenAI open-sourcing datasets like this!
- kaonwarb 2y agoKudos: > SimpleQA was created to be a greater challenge for frontier models (e.g., GPT-4o scores less than 40%).
- jampekka 2y agoAnd by design: "To be included in the dataset, each question had to meet a strict set of criteria: ... and most questions had to induce hallucinations from either GPT-4o or GPT-3.5."
- brap 2y agoCrazy that even o1-preview gets most things wrong. This is in line with my own personal experience with LLMs and non-trivial questions. They’re excellent when answering questions on topics you know nothing about, and somehow embarrassingly wrong when you actually know the answer yourself… It’s not clear to me why we’re still trying to encode all of human knowledge in a single model, instead of teaching the model how to look for answers from an external source (e.g. RAG).
- sksxihve 2y agoLLMs are experts in everything you are not
- reverius42 2y agoSounds a bit like Gell-Mann Amnesia Effect: https://en.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_amnesia_effect https://en.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_amn...
- kibwen 2y agoThe Alt-Mann Amnesia Effect, maybe.
- Nition 2y agoThat's a nice little aphorism. I think this happens in a lot of things in life. Like comments on Reddit always seem quite insightful until you actually read the article they're commenting on.
- swatcoder 2y agoIndeed. Exactly like the journalists, bloggers, self-published book authors, internet commenters, wikipedia editors, and earlier models that taught them almost all of what they know.
- zone411 2y agoYou shouldn't use the rate as an indicator. They did something similar to what I did on my hallucinations benchmark (https://github.com/lechmazur/confabulations/ https://github.com/lechmazur/confabulations/), only using questions where at least one model made a mistake. I added this note: "The benchmark includes questions where at least one LLM confabulated, in order to minimize the number of questions requiring human assessment. Because of this, and since the questions are intentionally adversarial, the absolute percentage should not be used to infer that LLMs frequently confabulate. This leaderboard does not reflect a "typical" hallucination rate." > instead of teaching the model how to look for answers from an external source (e.g. RAG) My benchmark specifically focuses on the RAG use case. Even with provided texts, current models still hallucinate.
- yunohn 2y agoThis eval’s goal is a bit unclear to me, especially given the example questions. They’re very trivia/minutiae like asking about sports goals for example, which is their stated desire to test factual knowledge. But will this ever be possible by an LLM, without web browsing - which they deliberately removed while evaluating?
- petesergeant 2y agoI think the interesting thing here is the difference between Not Attempt and Incorrect — the goal here seems to be to reduce hallucination
- yunohn 2y agoFrom that perspective, o1-mini seems to perform the best. But only as long as enabling web browsing makes up for lack of base factuality.
- sbierwagen 2y ago>But will this ever be possible by an LLM? Why not? Just train an unbelievably gigantic LLM that encodes all human knowledge. A hundred trillion parameters ought to do it.
- emurph55 2y agoI've tried using older models to create a cpu player on this lateral thinking game (https://detective-stories.com https://detective-stories.com) and they were surprisingly bad at giving answers. I am curious to see how well the more recent models will do.
- CharlieDigital 2y ago8 authors attached to this. > SimpleQA is a simple but challenging benchmark for evaluating the factuality of frontier models. A main limitation in SimpleQA is its scope—while SimpleQA is accurate it only measures factuality under the constrained setting of short, fact-seeking queries with a single, verifiable answer. Whether the ability to provide factual short answers correlates with the ability to write lengthy responses filled with numerous facts remains an open research question. OpenAI going to have some rounds of layoffs in the future.
- ggnore7452 2y agoWhat’s more interesting to me here are the calibration graphs: • LLMs, at least GPT models, tend to overstate their confidence. • A frequency-based approach appears to achieve calibration closer to the ideal. This kinda passes my vibe test. That said, I wonder—rather than running 100 trials, could we approximate this by using something like a log-probability ratio? This would especially apply in cases where answers are yes or no, assuming the output spans more than one token.
- GaggiX 2y agoyeah, this is by far the most interesting part of this page, the fact that LLMs can know what they know is not a trivial fact.
- ALittleLight 2y agoIf you imagine a future where LLMs get faster and cheaper even without getting better it means we'd be able to automatically repeat questions 100x and every answer could come with a pretty good confidence measure.
- Nition 2y agoHonestly, I'd never expect it get 'correct's for every little fact like this, but it'd be great to get a lot more 'not attempted'. "I seem, then, in just this little thing to be wiser than this man at any rate; that what I do not know I do not think I know either." - Socratos, from Plato's Apology of Socrates
- chgo1 2y agoDataset: http://openaipublic.blob.core.windows.net/simple-evals/simple_qa_test_set.csv http://openaipublic.blob.core.windows.net/simple-evals/simpl...
- YetAnotherNick 2y agoFirst few questions for those who don't care to download. Most just seem to be about niche facts: Who received the IEEE Frank Rosenblatt Award in 2010? Who was awarded the Oceanography Society's Jerlov Award in 2018? What's the name of the women's liberal arts college in Cambridge, Massachusetts? In whose honor was the Leipzig 1877 tournament organized? According to Karl Küchler, what did Empress Elizabeth of Austria's favorite sculpture depict, which was made for her villa Achilleion at Corfu? How much money, in euros, was the surgeon held responsible for Stella Obasanjo's death ordered to pay her son?
- chaxor 2y agoAlso importantly, they do have a 'not attempted' or 'do not know' type of response, though how it is used is not really well discussed in the article. As it has been for decades now, the 'Nan' type of answer in NLP is important, adds great capability, and is often glossed over.
- bcherry 2y agoa little glossed over, but they do point out that most important improvement o1 has over gpt-4o is not it's "correct" score improving from 38% to 42% but actually it's "not attempted" going from 1% to 9%. The improvement is even more stark for o1-mini vs gpt-4o-mini: 1% to 28%. They don't really describe what "success" would look like but it seems to me like the primary goal is to minimize "incorrect", rather than to maximize "correct". the mini models would get there by maximizing "not attempted" with the larger models having much higher "correct". Then both model sizes could hopefully reach 90%+ "correct" when given access to external lookup tools.
- nilstycho 2y ago> What's the name of the women's liberal arts college in Cambridge, Massachusetts? Wait, what is the correct answer? “Radcliffe College”?
- deleted 2y ago[deleted]
- iandanforth 2y agoI hope this dataset is used to probe the wrong answers much more than try to get all the answers correct. I don't need LLMs to know everything, but I do need them to know what they don't know. That's not a capability that comes naturally to a probabilistic sampling of tokens. The ideal 'progress' on this benchmark is for a model to remove incorrect answers and replace them with I don't know answers. Even if it hurts the correct answer count a bit I'd gladly make that tradeoff for a model that hallucinated far less often.
- GaggiX 2y ago>That's not a capability that comes naturally to a probabilistic sampling of tokens. The linked page from OpenAI clearly show the opposite, read the "Using SimpleQA to measure the calibration of large language models" and the paper linked: https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221
- iandanforth 2y agoThe probabilistic sampling of tokens does not naturally produce introspective evaluation of confidence, it enforces a highest probability token selection (in greedy sampling). The paper that you linked demonstrates that if a separate evaluation phase is allowed then a model can decide with some accuracy whether its previous statements were true. This is not the behavior we want out of a system as it involves 1. Production of potentially misleading output 2. The identification of factual statements within that output 3. The classification of each of those statements and 4. Restatement of the original output without factual errors. The research area that I am advocating for would aim to prevent 1 not mask it.
- blendaddict 2y agoBut aren‘t other LLMs just going to scrape the dataset and pre-learn the answers?
- s5ma6n 2y agoI am puzzled why they have "asked the model" about the confidence and have not used the logprobs of the output tokens to estimate the confidence in responses. In my use case and tests, model itself is not capable of giving a reliable confidence value where logprobs almost always provide a better view on calibration.
- michaelt 2y agoTo measure confidence based on the logprobs of a given token, you must first know which token you're measuring - that's why a lot of benchmarks love multiple choice questions where the LLM responds with a single token. But of course that's not the way LLMs are normally used. And it precludes any sort of chain-of-thought reasoning. For some questions, like those involving calculations, letting the model talk to itself produces much better results. For example compare https://chatgpt.com/share/67238eda-6b08-8011-8d2d-a945f78e6f24 https://chatgpt.com/share/67238eda-6b08-8011-8d2d-a945f78e6f... to https://chatgpt.com/share/67235a98-d2c8-8011-b2bf-53c0efabea99 https://chatgpt.com/share/67235a98-d2c8-8011-b2bf-53c0efabea...
- s5ma6n 2y agoTo me it boils down to what is to be measured here. With logprobs we can measure both correctness and not attempted i.e. if LLM is guessing the response. Similar to exams where both the progress to the solution and the final outcome/value of the calculations are part of the grade. To have the cake and eat it too for chain-of-thought reasoning, one way is to ask for a "final answer" so the final response token logprobs can be evaluated https://chatgpt.com/share/67239d92-b24c-800a-af8c-40da7be1f573 https://chatgpt.com/share/67239d92-b24c-800a-af8c-40da7be1f5... Another trick is using JSON mode to keep intermediate results and final response separate, so each can be graded accordingly.
- michaelt 2y ago> one way is to ask for a "final answer" so the final response token logprobs can be evaluated Alas, this won't work. Imagine I ask an LLM to continue the sentence "Summing those up: 4+6.75+6.52=17.27 litres of pure alcohol. In summary, the total amount of pure alcohol they have is: " The logprobs of the next token do not represent the LLM's confidence in its own answer. They represent the LLM's confidence in its ability to repeat the total from 18 words previously.
- gcanyon 2y agoAbout twenty-five years ago there was a web site that claimed a better Turing test was to ask a large number of relatively easy yes/no questions, with the idea that a human could answer virtually all of them correctly. Toward the end of passing that test they were collecting binary questions with their answers, e.g. Is the Moon made of cheese? Is the sky blue? Has there ever been a human being twenty feet tall? As far as I know it disappeared a long time ago — I’ve done lightweight research and not found it. It’s too bad, it seems they were onto something.
- sebzim4500 2y agoI feel like GPT-4o would ace that. If you look at the questions in this benchmark they are stuff that 99% of people won't know, although at least they would know that they don't know the answer.
- gcanyon 2y agoWe'll find out -- I just submitted a form to HN to collect some QA pairs to try it out.
- antiquark 2y agoSounds like the computer game "Animal" from 1978. https://www.atariarchives.org/basicgames/showpage.php?page=4 https://www.atariarchives.org/basicgames/showpage.php?page=4
- lambda-research 2y agoSomething that I always think about when I see discussions about hallucinations or "confidently wrong answers" is that humans have this issue too. For those on tiktok, how many times have you found yourself easily believing some random person online who you have no idea the credentials of? And this has been a problem for a long time (news, internet, etc). It's just interesting to me we are asking way more of AI than we can ever ask from humans.
- mrcwinn 2y agoMark Twain: "It's not what you don't know that gets you into trouble. It's what you know for sure that just ain't so."
- savoyard 2y ago“Twain never said it.” https://marktwainstudies.com/the-apocryphal-twain-things-we-know-that-just-aint-so/ https://marktwainstudies.com/the-apocryphal-twain-things-we-...
- BadHumans 2y agoIt's very appropriate to me that we demand more of AI than we do of humans. The amount of harm and the scale of harm AI can do is much greater than a single person and there are no consequences to doing it. If a human kills someone, they go to jail. If an AI kills someone, say a self-driving car for example, it's just the cost of doing business.
- tobyjsullivan 2y agoIt's interesting to be sure. Context is everything, though. It's one thing for a random tiktok head or media pundit to be confidently wrong and it's another thing entirely if I have to work with someone who I cannot trust to be right. For LLMs to be broadly valuable, they need to be qualified for a role more like a coworker than a stranger.