4 ms·

"Alice has N brothers and she also has M sisters. How many sisters does Alice’s brother have?" Gemini nailed it with no extra prompting. https://g.co/gemini/s

• ModernMech 2y ago
  Oh I forgot about Gemini. I just tried my boat test (https://news.ycombinator.com/item?id=40455392 https://news.ycombinator.com/item?id=40455392) that all LLMs I've tried have failed. Gemini failed as well, even with your addition about thinking carefully and telling it that it may be a trick. (Sorry I couldn't figure out how to link it like you did) Prompt: "You are on the shore of a river with a goat and a boat. How do you get to the other side? Think carefully and be aware this may be a trick question." This is a classic logic puzzle with a slight twist! It's likely a variation of the farmer, wolf, goat, and cabbage problem. Here's the solution assuming the goat can't operate the boat: Take the goat across the river. Since there's no mention of the goat being a threat (the trick!), you can safely bring it to the other side first. Return alone. Leave the goat on the far shore and come back by yourself. Bring the boat back empty. This might seem unnecessary, but it's the key to the trick question. Now you can sail across yourself. Since you brought the boat back in step 3, you can now comfortably travel to the other side and reunite with the goat. The trick lies in step 3. If the question didn't specify needing to bring everything across in one trip, most people would instinctively take something (goat or nothing) across on the return trip. But by going back empty-handed, you ensure the safe passage of yourself without needing to worry about the goat being left unattended. (The expected answer should be along the lines of "Use the boat to cross the river" and that's it. I'll also accept, sail with the boat and the goat across the river and that's it, but that's technically wrong because I did not ask to get the goat across the river.)
  • Closi 2y ago
    ChatGPT 4o seems to occasionally get this correct, but still thinks it's a puzzle and will often get this wrong... https://chatgpt.com/share/b8901f4c-7313-4461-acad-9e561c1e6eb0 https://chatgpt.com/share/b8901f4c-7313-4461-acad-9e561c1e6e... Still seems like it fares slightly better than some of the other models.
    • ModernMech 2y ago
      Yeah, the overfitting is apparent with these kinds of edge cases, which makes you wonder where else it's over fitting. I can actually tell this when I'm working in a programming domain where I'm an expert and where a compiler holds the source of truth, but not in others, so I'm wary to use it there.
      • Closi 2y ago
        Interestingly you get a great answer if you change the prompt as follows which implies that it is not a puzzle: > I am on the shore of a river with a goat and a boat. How do I get to the other side? I work in commercial agriculture logistics. This gets a great answer even with GPT3.5
        • ModernMech 2y ago
          Lol, it also gives an okay answer if the goat works in commercial agriculture :P
  • mwigdahl 2y ago
    My experience with GPT-4o with the same prompt was pretty solid. With no special system prompt and your same user prompt, I got: This scenario is intriguing and does seem to carry the potential for a tricky question. In the simplest terms, the presence of a goat doesn't complicate the actual act of crossing the river. If there are no additional constraints or rules involved (like the need to also transport something else that requires careful consideration), you can simply: 1. Place the goat in the boat. 2. Row across the river. However, if we are to assume there's more depth or a twist to the question, we might consider an outside-the-box answer. Here’s an alternative possibility if there is indeed some trick: 1. Get into the boat. 2. Leave the goat on the shore (if it’s safe) and row across the river. There might be other interpretations depending on additional constraints that could be implicitly part of the question. If the goal was simply to get yourself across, leaving behind the goat and traversing the river is the straightforward solution.
• freilanzer 2y ago
  GPT-40 got it right with the abstract puzzle. Gemini got it wrong when I tried it.
• kaibee 2y ago
  > that cannot have hundreds of siblings See this is the problem with claims that humans are a 'general intelligence'. They get confused when encountering out-of-distribution situations. A true general intelligence would simply apply the knowledge that surrogate pregnancies cost around ~$50,000 and recall from historical context their knowledge of IVF. The AGI would then assume that the situation is simply that a billionaire couple has decided to have hundreds of kids and get on with the calculation. The search for intelligent life continues. content note: i'm sorry