7 ms·

Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" (nb: I have added numbers, it's phrased as X and

• bamboozled 2y ago
  I got it in 5 seconds, am I the singularity ?
  • deleted 2y ago
    [deleted]
  • r2_pilot 2y ago
    Do you sleep, get sick, require more than compute hardware/storage/networking/electricity?
  • felipemnoa 2y ago
    We all are, but only in meat-space. We continue to improve ourselves much faster than evolution ever could. But what we are talking about here is the singularity in tech-space.
    • bamboozled 2y ago
      I don’t see such of a distinction between technology and us. We build, drive and continue our overflow this tech. It’s an extension of us inspired by how our own brains work.
• toxik 2y ago
  Seems like an odd thing not to state prominently. ChatGPT 4o solved it every time for me. Seems to work OK even with the "hands tied behind back" prompt. Wonder if this paper is just inaccurate or if OpenAI adjusted the model -- seems unlikely. Actually, 4o is reported to solve this riddle 60% of the time. I guess I have to test more, but with my about a dozen tests so far, all were correct.
  • Miraste 2y ago
    The "only output the number" prompt was one of three they used. The others were one with no instructions beyond the question, and one that explicitly told the model to show its reasoning and check for errors.
  • michael-ax 2y ago
    Alice has N Brothers, and she has M sisters. How many sisters do Alice’s brothers have? I have not gotten the correct answer to the question as phrased above in one go from Gpt4o yet! (and today was not the first day i tried.) Phrase it as shown above and you'll likely need 5 or more interactions to get it to generate the correct output. With Gemini i could not get it below 8 without feeling like i was cheating. fwiw.
    • lupire 2y ago
      Chat GPT 4o. I was being a bit generous with background information, but still tests ability to interpret: ------ Me: Background facts: Alice is a female human. All sisters are female, and all brothers are male. No one is their own brother or sister. Alice has N brothers, and Alice has M sisters. Now, a few questions based on these facts: How many sisters do Alice’s brothers have? Do Alice's brothers have more sisters, fewer sisters, or same number of sisters than Alice has? Don't forget to consider Alice when counting. ------ ChatGPT: Let's break down the questions step by step: How many sisters do Alice’s brothers have? Alice has M sisters. Since all sisters are female and Alice is included in the total count of sisters for her brothers, Alice's brothers have M+1 sisters. This includes Alice herself and her M sisters. Do Alice's brothers have more sisters, fewer sisters, or the same number of sisters as Alice has? Alice has M sisters. Alice's brothers have M+1 sisters (since it includes Alice herself and her M sisters). Therefore, Alice's brothers have more sisters than Alice has. In conclusion: Alice’s brothers have M+1 sisters. Alice's brothers have more sisters than Alice has.
• GPerson 2y ago
  There must be a name for the new phenomenon, of which your post is an example, of: 1. Someone expresses that an LLM cannot do some trivial task. 2. Another person declares that they cannot do the task, thereby defending the legitimacy of the LLM. As a side note, I cannot believe that the average person who can navigate to a chatgpt prompter would fail to correctly answer this question given sufficient motivation to do so.
  • FeepingCreature 2y ago
    Well, why does it need a name? It just seems like a "correct argument". If somebody is claiming that AI is "not all that" because it can make stupid mistakes, surely it's relevant that humans also make stupid mistakes, so AI making them does not necessarily pose a hindrance to it becoming a human-tier reasoner.
    • GPerson 2y ago
      It’s worth giving names to interesting phenomena as it helps people to identify and understand them. I find this phenomenon interesting because a person who wants the defend the LLM from apparent criticism is probably more likely to exhibit it.
  • cratermoon 2y ago
    There's also the phenomenon of papers showing ChatGPT getting it wrong, then people posting anecdotal examples of it getting it right when they try it, but that one already has a couple of names: WORKSFORME or Unable to Reproduce Hey folks, ever considered the possibility that unreproduceability is not a good thing?
  • yoyohello13 2y ago
    Many people, especially on this site, really want LLMs to be everything the hype train says and more. Some have literally staked their future on it so they get defensive when people bring up that maybe LLMs aren’t a replacement for human cognition. The number of times I’ve heard “but did you try model X” or “humans hallucinate too” or “but LLMs don’t get sleep or get sick” is hilarious.
• eranation 2y ago
  Great observation. It would be really interesting to repeat this research with less limiting prompts. I assume they made the restriction to make it easier to parse the answers, perhaps make it two phase, let them respond unlimited, then a follow up prompt ask to compress it to a single answer. I wonder how the results will vary.
  • FeepingCreature 2y ago
    To be quite honest, I assume they made the restriction so that the models would fail. This sort of paper is becoming a genre.
    • detourdog 2y ago
      It is a proof of weakness in the current system. This makes sense and births new hypotheses.
    • nyrikki 2y ago
      You test models where they fail in any field. The orbit of Mercury to discover GR as an example. As all models are wrong, but some are useful, finding where they fail is how you figure out if they are useful. As the 'AGI is near' camp has won the hype game, it is important to ground expectations for practical exploitation of the technology. Over promising unabashed optimism is partly what caused the previous AI winters. As the formal proof methods of mathematics proved impractical, counterexamples and the scientific method is what CS has used for decades.
    • qsi 2y ago
      They used three different kinds of prompts with varying levels of restrictions, as described in the paper. To be quite honest, I assume you made your comment so that you could dismiss the paper without reading it.
      • FeepingCreature 2y ago
        That's a fair cop, I didn't read it. The thing is that "LLM reasoning breaks down" simply did not surprise me enough that I thought it was worth clicking. Making LLMs fail is not hard. They're interesting for the ways that they work, not the (many, many) ways that they don't. edit: I've had a look and I don't think any of their prompts are very good. They're certainly not how I'd write them if I wanted a current model to actually solve the problem. The way to make me take a paper like this seriously would be if you set it up as an adversarial collaboration with a competent prompter, and that person agreed they couldn't make a generic prompt that solved the problem. "We tried three times and none worked" is not news, or at any rate not news about LLMs.
• oidar 2y ago
  Of course it's going to give an incorrect answer with that prompt. If the instruction fine tuning is neutered like this prompt, it's going to roll over to the foundation model and offer a completion - probably more influenced by the seed than the prompting text. Bad study. Edit - I just skimmed the paper - they do use other more appropriate prompt types for reasoning. My initial response was based on the assumption that all prompts used that script prompt quoted in the parent. I retract my "bad paper" comment.
  • YeGoblynQueenne 2y ago
    >> My initial response was based on the assumption that all prompts used that script prompt quoted in the parent. You, and another 20 or so commenters here. We should really re-examine the guideline about asking people to RTFA. No offense meant- good on you for correcting your error.
• deleted 2y ago
  [deleted]
• sdiepend 2y ago
  The right answer depends on how Alice identifies I guess? :)
  • Karellen 2y ago
    > > Question is: "Alice has 60 brothers and she also has 212 sisters. How many sisters does Alice’s brother have?" > The right answer depends on how Alice identifies I guess? :) Given that the wording of the question specifically identifies Alice as "she", rather than using a gender-neutral pronoun or no pronoun at all, I think inferring that she identifies as female is reasonable.
  • rep_lodsb 2y ago
    Page 42 of the paper :) One thing that strikes me is that the model first tries using "inclusive language" in one answer - and literally states so, using this specific term - but seems to interpret it in a more mathematical sense (like set inclusion). Then seamlessly switches to the expected DEI spiel in the next paragraph. For one thing, it makes me suspect that something with the words "inclusive language" was automatically added to the prompt. But more interesting is how it responds to this demand in two different ways, illustrating a "thought process" that is very much unlike that of a human with normal verbal reasoning ability. I am not a psychologist, but remember reading that schizophrenic people sometimes confuse different meanings of words in a similar way, jumping from one meaning to another without noticing.
• llm_trw 2y ago
  >Worth noting that the prompts from the experiment include "To answer the question, DO NOT OUTPUT ANY TEXT EXCEPT following format that contains final answer: ### Answer:" so it appears that they are stopping the models from 'thinking out loud'. If I add that to the prompt, GPT4o gets it consistently wrong... Yes this is a common thing I see people who think LLMs are idiots do. The more an LLM talks the smarter it gets _because that's the only way it can compute anything_. Imagine saying that Turing machines fail the Church–Turing thesis because they can't solve 3-sat for N variables in N moves or less. That's what you're doing to an LLM when you ask it to be concise.
  • rossdavidh 2y ago
    Even if this were all true, it points to a fundamental risk of using LLM's for important tasks, which is that it is not at all clear to a user that this prompt would cause a problem. The LLM doesn't say "I'm sorry Dave, I just can't do that", it just complies with it and gets the wrong answer. You can always make excuses for the LLM afterwards, but software with hidden risks like this would not be considered good or reliable in any other context.
    • saurik 2y ago
      People really need to stop trying to model an LLM as some kind of magical software component: it all makes a lot more sense if you model it as an under-performing poorly-aligned employee; so like, maybe a distracted kid working for peanuts at your store. You wouldn't trust them to with all of your money and you wouldn't trust them to do a lot of math--if they had to be in charge of checkout, you'd make sure they are only given a point-of-sale terminal and their main job was to, at best, scan the barcodes and compare the total--and yet there are tasks you can imagine handing to them that you'd never give to a robot or computer even though they get it wrong a lot, as not all tasks need to be handled perfectly, they still understand extremely fuzzy tasks, and they are probably cheaper than a qualified adult (certainly cheaper than one who is being paid enough to "give a shit" and pay enough attention to not let you get robbed or even put themselves at some risk for you).
      • 2y ago
• segfaultbuserr 2y ago
  > I must confess, when I tried to answer the question I got it wrong...! (I feel silly). In programming there are two difficult problems - naming things, cache invalidation, and off-by-one error.
• nerdjon 2y ago
  Even "thinking out loud" we have seen these systems fall flat on their face and give very wrong answers. Thinking out loud also only gets you so far, if the expectation is a certain type of response it can't always "think out loud". In reality that just proves it isn't really reasoning here and is more likely just self referencing. That being said, I tried this personally allowing it to think out loud and it told me she has 212 sisters. Using your exact prompt.
  • Closi 2y ago
    > Thinking out loud also only gets you so far, if the expectation is a certain type of response it can't always "think out loud". In reality that just proves it isn't really reasoning here and is more likely just self referencing. Try to calculate it without writing anything down, or thinking any numbers or words in your head. You can't draw a 1:1 analogue between an AI and the human experience, but remember that we have an internal stream of consciousness. Maybe the outputs of the LLM are more similar to the stream of consciousness in our heads rather than the words we say? After all, Humans also do lots of self referencing. > That being said, I tried this personally allowing it to think out loud and it told me she has 212 sisters. Using your exact prompt. Fair enough, but worst case it can often solve it correctly with the correct reasoning. GPT3.5 can't solve it correctly with correct reasoning, so we are at least appearing to be on a path where AI's can start to solve this question, albeit potentially not fully reliably.
• GrantMoyer 2y ago
  Note that in the paper, all the numbers used were under 10. > AIW Variation 1, N=3,M=6,C=7 > AIW Variation 2, N=4,M=2,C=3 > AIW Variation 3, N=1,M=4,C=5 > AIW Variation 4, N=4,M=1,C=2. Also note that the resricted prompt is only one of the prompt variations tested by the paper. It also explores common techinques to get LLMs to perform better, including "thinking out loud". Even with these methods the models still fail to produce a correct answer. > Model prompt types. It is well known that so-called prompt engineering can heavily influence the model behavior and model response quality [26, 27, 28]. To account for the response variations due to various prompt forms, we created 3 distinct prompt types asking for the solution to the AIW problem: STANDARD, THINKING, and RESTRICTED. The STANDARD prompt type asks to solve the posed problem and output the final answer in the format as described above. This does not put any specific requirements on model behavior. The THINKING prompt type extends STANDARD with the request to think carefully and double check the solution for any mistakes. This should encourage model to invest more computation into obtaining the solution. In contrast to this, the RESTRICTED prompt urges the model to output only the final answer without any further text. This is supposed to restrict compute invested in producing output. We observe substantially shorter outputs across tested models compared to STANDARD and THINKING for this prompt type (Suppl. Fig. 13).
• freilanzer 2y ago
  I just tried it with intermediate steps, via the total amount of children Alices mother has, and it worked. So, it seems that an intermediate logical step is too much for these models at the moment, at least in this puzzle where Alice is outside the number given in the description.
• deleted 2y ago
  [deleted]
• YeGoblynQueenne 2y ago
  >> I must confess, when I tried to answer the question I got it wrong...! (I feel silly). I only realised I got it wrong when I plugged it into GPT-4o and it came back with the correct answer: Remember that the authors of the paper did not find that GPT4-o cannot return the right answer. They found that it can't return the right answer more often than ~60% of the time. So you'd have to repeat the experiment many, many times and aggregate the results (the paper uses a binomial Beta this and that etc etc) before you see similar results as the paper. You won't replicate the results of the paper unless you really put your back into it.