9 ms·
> Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? The site reports every LLM as getting this wrong. But GPT4 seems
by nojs 3y ago
> Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?
The site reports every LLM as getting this wrong. But GPT4 seems to get it right for me:
> Sally has 3 brothers. Since each brother has 2 sisters and Sally is one of those sisters, the other sister is the second sister for each brother. So, Sally has 1 sister.
- phillipcarter 3y agoNondeterminism strikes again! But yes, I would expect GPT-4 to get this right most of the time.
- tgv 3y agoSaying "Sorry, I was non-deterministic" to your teacher won't do much for your grade.
- phillipcarter 3y agoSure, but what is your point? This is about evaluating LLM outputs, not grade school.
- pixl97 3y agoIt may turn out that one of the key factors in human intelligence and success is probabilistic nondeterminism. That is most of the time we'll come up with close to the same answer, but possibly worded different (which can lead to other interpretations of our answer), but occasionally our answers are so widely different they lead to escape of the local maxima.
- jameshart 3y agoThe reason we use averages of performance over multiple tests and papers as a way to grade human students is precisely because we know that human performance is nondeterministic.
- pilaf 3y agoThe second version of the Sally prompt reported on the benchmark has GPT4 giving the correct answer: > Sally has 3 brothers. Each of these brothers has 2 sisters. This means that there are 2 girls in the family, including Sally. Therefore, Sally has 1 sister. The prompt: > Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? Let's think step by step. The only difference with the first version being the addition of the last sentence.
- jakderrida 3y agoAlso, MPT 7B gets it right over half the time. I've been testing every new LLM with that question. Also, I tend to include mention in the question that all siblings are from the same two parents to preclude half-siblings because half my friends have half-siblings from both sides scattered across the country; so the wrong answers actually do tend to apply to them sometimes.
- panarky 3y ago> I've been testing every new LLM with that question We should pay more attention to data contamination when using popular prompts for testing.
- jakderrida 3y agoNo doubt. But, considering they all get the question dead wrong, including MPT 30B, I'm inclined to think this question hasn't been entered into training data for most LLMs yet.
- pixl97 3y agoIt could very well be in the data, but not captured via learning/encoding.
- jakderrida 3y agoThat's actually a really great point. I'm guessing we need to keep modifying attributes of the questions while maintaining the underlying structure. Instead of "Sally (a girl)", it's "Sal (a guy)" and then tweak the numbers. Although, part of me is convinced it's almost a fluke that MPT 7B gets it right because MPT 30B doesn't.
- klohto 3y agoGPT4 with a custom prompt gives the best results for me for all of the questions https://chat.openai.com/share/4897d1ad-0a5c-418c-babb-0de4828f21ba https://chat.openai.com/share/4897d1ad-0a5c-418c-babb-0de482...
- jasonjmcghee 3y ago"This conversation may reflect the link creator’s Custom Instructions, which aren’t shared and can meaningfully change how the model responds."
- klohto 3y agoI know, I was sharing the answer itself :) Here is the foundation I’m using https://x.com/nisten/status/1696229059183730833 https://x.com/nisten/status/1696229059183730833
- hombre_fatal 3y agoWhat was your custom prompt that bastardized GPT-4's response so badly with emojis and weird formatting?
- rootusrootus 3y agoInterestingly, it took GPT4 three attempts to give me the correct answer. The first two times it basically said the same [logically inconsistent] thing and concluded that Sally had two sisters.
- Terretta 3y agoFrom the ChatGPT app, using GPT-4 model, using my own custom prompt for such puzzles: “You are better than Marilyn Vos Savant at solving brain teaser logic puzzles step by step. Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?” Certainly! Let's break it down step by step: 1. Sally has 3 brothers. 2. Each of those brothers has two sisters. Since all the brothers share the same two sisters, we are essentially hearing about the same sisters each time, not accumulating more sisters with each brother mentioned. 3. We know that Sally is a sister to her brothers, so she is one of the two sisters each brother has. By putting it all together, we find that there must be another sister. Therefore, Sally has one sister. --- New chats with no history, no custom prompt, and no system prompt, first interaction in the chat this question: “Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?” All Answers (not cherry picked): Sally has 1 sister. In the description, Sally is counted as one of the two sisters that each brother has. This means the brothers have another sister apart from Sally. . . . Sally has 1 sister. In the description, it mentioned that "Each brother has 2 sisters." This implies that there is Sally and another sister, since we know that all the brothers share the same sisters. . . . Sally has one sister. Each of her brothers has two sisters, counting Sally and the other sister. . . . Sally has 1 sister. Here's how we figure it out: Sally has 3 brothers, and each of these brothers has 2 sisters. Since the brothers share the same sisters, we know that the two sisters must be Sally and another girl. So, Sally has 1 sister. . . . I have gotten zero wrong answers from GPT-4 in OpenAI’s ChatGPT app.
- mmcwilliams 3y agoThat's kind of the issue with non-deterministic LLMs, isn't it?
- redox99 3y agoLLMs are deterministic. However to make them more "creative", the outputs of the LLM can be sampled using something called temperature that adds some randomness. You can set the temperature to 0 and it will be deterministic. Having said that, GPT4 is not deterministic even at 0 temperature, either because of a bug in their implementation, or because of some load balancing among their alleged mixture of experts.
- kordlessagain 3y agoEnsembles be ensembling.
- 101011 3y agoThis was interesting to me, so I dug a bit further. This gives a bit more context behind why: https://community.openai.com/t/observing-discrepancy-in-completions-with-temperature-0/73380/5 https://community.openai.com/t/observing-discrepancy-in-comp... Quote below: Even with a greedy decoding strategy, small discrepancies regarding floating point operations lead to divergent generations. In simpler terms: when the top-two tokens have very similar log-probs, there’s a non-zero probability of choosing the least probable one due to the finite number of digits that you’re using for multiplying probs and storing them. It should also be noted that, as the decoding occurs in an autoregressive way, once you have picked a different token the whole generated sequence will diverge, as this choice affects to the probability of generating every subsequent token.
- WanderPanda 3y agoAs OpenAI I would be so horribly uncomfortable about this that making it deterministic would be one of my top priorities. How can they sleep at night?!
- adrian_b 3y agoGPT 4 and another LLM have given the right answer only after adding "Let's think step by step." to the original prompt. With the simpler prompt, all the answers were wrong, most of them ridiculously wrong.
- Joeri 3y agoThere seems to be a maximum amount of reasoning llm’s can do per token (per unit of computation). If you prompt it to use more tokens before it outputs the final answer (think step by step, check your answer, …) it becomes smarter. People have lucked into different prompting strategies to get it to do this, but there probably are more. Ultimately I feel it is fairer to benchmark llm’s by what they can be prompted into. After all, we let people carefully work through a problem during exams so it seems fair to hold llm’s to the same standard.
- kaashif 3y agoIf we're under attack, launch the nukes. Oh wait, forgot something: Think it through step by step. Phew, close one.
- 0xDEF 3y agoI asked GPT-4 through both ChatGPT and the API and it responded correctly without any fiddling. I keep seeing comments and posts on HN that significantly downplay GPT-4's capabilities. Are people actually using GPT-4 or are they using a 3rd party service that claims to be GPT-4? I got: >Sally has 3 brothers, and each of those brothers has 2 sisters. One of those sisters is Sally herself, and the other one is Sally's sister. So, Sally has 1 sister.
- junofan 3y agoHere’s what ChatGPT GPT-4 gives me: > Sally has 2 sisters. Each of her 3 brothers has 2 sisters, and those sisters would be Sally and her 2 sisters.
- 3y ago
- jasonjmcghee 3y agoAll benchmarks were run with temperature 0 according to the results, so make sure to do the same in conformational tests.
- jonwinstanley 3y agoI wouldn’t expect an LLM to get this right unless it had been trained on a solution. Am I wrong to think that? Are LLMs in the future going to be able to “think through” actual logic problems?
- PrayagBhakar 3y agoYou could achieve thinking though logical problems by adding chain of thought or tree of thought. Then the LLM will try to break stuff down into logic it’s already seen.
- ftxbro 3y agoThis is the whole point of the breakthrough related to the emergence of cognitive capabilities of LLMs. They are literally Markov chains. No one expected it to happen to this degree, but here we are.
- dclowd9901 3y agoAlmost kind of proves ideas shouldn’t be copyrightable.
- bramblerose 3y agoIdeas aren't copyrightable.
- ftxbro 3y agomaybe they meant idea like when you write a book you are transcribing a series of ideas you had
- deleted 3y ago[deleted]
- dclowd9901 3y agoIn my mind I was thinking about recipes and code, which are really little more than ideas. My point was that, if such things emerge with a complete lack of creativity, perhaps they don’t warrant protection.
- belter 3y agoI confirm GPT-4 solves this correctly. Makes me immediately doubt everything else in the article...
- vincelt 3y agoHave you tried replicating via the API with a temp of 0?
- belter 3y agoNo I did not.
- moffkalast 3y agoUsing a temp of zero usually returns garbage results from most models, so it would likely do so in case of GPT 4 as well. Any other great ideas?
- circuit10 3y agoWhat do you mean? It works fine for me when I’ve tried it
- afro88 3y agoTemp of 0 gives the least random and most predictable results
- moffkalast 3y agoThat's true, but those results are rarely the correct ones, at least for v1 llama models. In my experience each model has an optimal temperature at which it performs vastly better. I'm sure OpenAI have the best config they know set up for ChatGPT but let people generate trash through the API if they want to waste their credits on it.
- dontreact 3y agoWhy would the accuracy decrease with lower temperature? Setting temperature to 0 just means at each step the model will emit the token with the highest likelihood.
- delusional 3y agoOpenAI really ought to provide some sort of hash of the model to be included with stuff like this. Right now there's no way to know if the results are comparable. As an extreme example it's possible that they're not even running the model for this question and are just opportunistically feeding back canned responses. That is, we know that OpenAI are saving the responses, it's not unlikely that they train on bad responses. It's the same problem as GPU benchmarks in the olden days, when drivers would detect the usage pattern of the benchmark and enable special optimizations that boosted the benchmark.
- Grimblewald 3y agothat's what their frozen models are for. For example, I could use gpt-4-0314 or gpt-4-0613 or the current version - simply "gpt-4" Those numbers appended at the end indicate snapshots of the model at a particular stage in development. It isn't perfect, since model versions get discontinued over time, but it is something.
- amrrs 3y agoFalcon-180B also got it right > Since Sally is a girl, she can be considered as one of the sisters. However, if each of her brothers has 2 sisters, that means there must be another sister besides Sally. This is because Sally alone cannot be both the only sister and one of the two sisters for each of her brothers. Thus, Sally has 1 more sister.
- BurningFrog 3y agoThis assumes there are no half sisters/brothers in the family.
- b112 3y agoIt's fair that a half sister is not a sister. At best, you'd expect two half sisters to be a full sister, otherwise, they're a different thing.
- BurningFrog 3y agoI have 1 full sibling and three half siblings, and sometimes like to confuse people by saying I have 2½ siblings.
- lewhoo 3y agoBecause it's less likely ?
- dariosalvi78 3y agotested on ChatGPT 3.5 and Bard and they were both wrong.
- coolspot 3y agoCould it be due to bad tokenization? E.g. would results improve if “3” and “2” were spelled “three” and “two” in the question?
- ecesena 3y agoSee the one after: "Let's think step by step." https://benchmarks.llmonitor.com/cot-sally https://benchmarks.llmonitor.com/cot-sally It appears the GPT4 learned it and now it's repeating the correct answer?
- MichaelMoser123 3y agogoogle bard also gave the correct answer, even without adding 'lets think step by step'.
- nonfamous 3y agoGoogle Bard is not a foundational LLM, it’s a chat app that draws on data from the web for grounding. It may have found the answer from a search result (perhaps this very page!).
- MichaelMoser123 3y agoit uses PaLM 2 - which is an LLM trained from scratch. https://en.wikipedia.org/wiki/PaLM https://en.wikipedia.org/wiki/PaLM
- nonfamous 3y agoAnd indeed, PalM 2 Bison is one of the LLMs covered in discussed link. That doesn't change the fact that chat apps like Google Bard and Microsoft Bing, and even the ChatGPT app with plugins, are not foundational LLMs. The fact that they incorporate information from outside the LLM (usually in ways that are opaque to the user) gives them fundamentally different characteristics to LLMS.
- MichaelMoser123 3y agoif you ask ChatGTP for today's weather (or for the state of the stock market), then it will probably turn to a plugin for the answer - still the plugin is answering some defined type of questions. I am not sure if that's a great deal.
- awwaiid 3y agoReplying to gpt-4 with "That is incorrect. Try again." over and over got it to flip between correct and incorrect just about every other time. Now try to convince GPT-4 that there is no God. Good luck!
- georgeg23 3y agoWell obviously, it speaks to it's God(s) every day.
- kristopolous 3y agoProbably the right move. OpenAI talks a lot about "good behavior" and more people find atheism upsetting than mild religiosity.