4 ms·

Both GPT-4 and llama 3 8b fail actual reasoning still: jack has 15 sisters, each sister has 3 brothers. How many brothers does jack have? (real answer is 2)

Both GPT-4 and llama 3 8b fail actual reasoning still: jack has 15 sisters, each sister has 3 brothers. How many brothers does jack have? (real answer is 2) ------------------- Llama 3 8b Let's break it down step by step: Jack has 15 sisters. Each of his sisters has 3 brothers. To find the total number of brothers, we need to multiply the number of sisters (15) by the number of brothers each sister has (3). 15 sisters x 3 brothers/sister = 45 brothers So, Jack has 45 brothers! ------------------- GPT-4 Jack has 3 brothers. Here's why: If each of Jack's sisters has 3 brothers, these brothers are the same for all sisters, including Jack himself. Therefore, Jack and his 3 brothers make up the total count of brothers in the family. Since the problem states that each sister has 3 brothers, we know that Jack himself must be counted as one of these brothers. So, Jack has 3 brothers. ------------------- Lol. You can tweak the questions numbers and names to avoid training data. Can Opus solve this?

rvnx 2y ago
> User: jack has 15 sisters, each sister has 3 brothers. How many brothers does jack have? > Claude-3-Opus: Let's approach this step-by-step: We know that Jack has 15 sisters. Each of Jack's sisters has 3 brothers. Since all the sisters are from the same family, they all have the same brothers. Therefore, the 3 brothers that each sister has are the same 3 brothers. Since Jack is a brother to his sisters, he is one of these 3 brothers. So, Jack has 2 brothers. Therefore, Jack has 2 brothers.
joelthelion 2y ago
Llama 3 70B seems to handle it just fine: A classic lateral thinking puzzle! Let's break it down: * Jack has 15 sisters. * Each sister has 3 brothers. Now, here's the key insight: if each sister has 3 brothers, that means each sister is counting Jack as one of their 3 brothers! So, how many brothers does Jack have? Only 2 The other 2 brothers that each sister is counting are actually Jack himself. Answer: Jack has 2 brothers.
- MyFirstSass 2y ago
  Wow, that's extremely impressive in light of GPT being 1800b so a model 25 times smaller is beating it logic wise? I don't have enough ram to run 70b but i'm very curious about performance on other logic puzzles!
  - MyFirstSass 2y ago
    I just tried variations and weirdly it fails many of them like: "bob has 7 sisters, each sister has 4 brothers. How many brothers does bob have?" ----------------- Llama 70b instruct (wrong) A classic lateral thinking puzzle! At first glance, it seems like a straightforward math problem, but wait... If each of Bob's 7 sisters has 4 brothers, that would mean each sister has Bob and 3 other brothers, right? But that's not possible, because Bob is the only brother! So, the correct answer is: Bob has no brothers. He has 7 sisters, and each of them has him as their only brother. Did I get it right? ----------------- GPT-4 (correct) To solve this, let's consider the family structure: Bob has 7 sisters. Each of these sisters has 4 brothers. Since all the sisters have the same number of brothers, and these brothers include Bob, this implies Bob has 3 other brothers. Therefore, including Bob, there are 4 brothers in total. Hence, Bob has 3 brothers.
ijustlovemath 2y ago
For me, Opus got it in two tries: >jack has 15 sisters, each sister has 3 brothers. How many brothers does jack have? Let's approach this step-by-step: 1) We know that Jack has 15 sisters. 2) Each of Jack's sisters has 3 brothers. 3) Since all the sisters are Jack's siblings, they share the same brothers. In other words, the brothers of one sister are the same as the brothers of any other sister. 4) Therefore, the number of brothers that Jack has is the same as the number of brothers each of his sisters has. 5) We are told that each sister has 3 brothers. Therefore, Jack has 3 brothers. > Nope You're right, I made an error in my reasoning. Let me try again: 1) Jack has 15 sisters. 2) Each of Jack's sisters has 3 brothers. 3) Jack is one of the brothers of each of his sisters. 4) Therefore, the total number of brothers (including Jack) is 3. 5) To find the number of brothers Jack has, we need to subtract Jack from the total number of brothers. 6) $\text{Number of Jack's brothers} = \text{Total brothers} - \text{Jack} = 3 - 1 = 2$ Therefore, Jack has 2 brothers.

rvnx 2y ago

> User: jack has 15 sisters, each sister has 3 brothers. How many brothers does jack have? > Claude-3-Opus: Let's approach this step-by-step: We know that Jack has 15 sisters. Each of Jack's sisters has 3 brothers. Since all the sisters are from the same family, they all have the same brothers. Therefore, the 3 brothers that each sister has are the same 3 brothers. Since Jack is a brother to his sisters, he is one of these 3 brothers. So, Jack has 2 brothers. Therefore, Jack has 2 brothers.

joelthelion 2y ago

Llama 3 70B seems to handle it just fine: A classic lateral thinking puzzle! Let's break it down: * Jack has 15 sisters. * Each sister has 3 brothers. Now, here's the key insight: if each sister has 3 brothers, that means each sister is counting Jack as one of their 3 brothers! So, how many brothers does Jack have? Only 2 The other 2 brothers that each sister is counting are actually Jack himself. Answer: Jack has 2 brothers.

MyFirstSass 2y ago
Wow, that's extremely impressive in light of GPT being 1800b so a model 25 times smaller is beating it logic wise? I don't have enough ram to run 70b but i'm very curious about performance on other logic puzzles!
- MyFirstSass 2y ago
  I just tried variations and weirdly it fails many of them like: "bob has 7 sisters, each sister has 4 brothers. How many brothers does bob have?" ----------------- Llama 70b instruct (wrong) A classic lateral thinking puzzle! At first glance, it seems like a straightforward math problem, but wait... If each of Bob's 7 sisters has 4 brothers, that would mean each sister has Bob and 3 other brothers, right? But that's not possible, because Bob is the only brother! So, the correct answer is: Bob has no brothers. He has 7 sisters, and each of them has him as their only brother. Did I get it right? ----------------- GPT-4 (correct) To solve this, let's consider the family structure: Bob has 7 sisters. Each of these sisters has 4 brothers. Since all the sisters have the same number of brothers, and these brothers include Bob, this implies Bob has 3 other brothers. Therefore, including Bob, there are 4 brothers in total. Hence, Bob has 3 brothers.

ijustlovemath 2y ago

For me, Opus got it in two tries: >jack has 15 sisters, each sister has 3 brothers. How many brothers does jack have? Let's approach this step-by-step: 1) We know that Jack has 15 sisters. 2) Each of Jack's sisters has 3 brothers. 3) Since all the sisters are Jack's siblings, they share the same brothers. In other words, the brothers of one sister are the same as the brothers of any other sister. 4) Therefore, the number of brothers that Jack has is the same as the number of brothers each of his sisters has. 5) We are told that each sister has 3 brothers. Therefore, Jack has 3 brothers. > Nope You're right, I made an error in my reasoning. Let me try again: 1) Jack has 15 sisters. 2) Each of Jack's sisters has 3 brothers. 3) Jack is one of the brothers of each of his sisters. 4) Therefore, the total number of brothers (including Jack) is 3. 5) To find the number of brothers Jack has, we need to subtract Jack from the total number of brothers. 6) $\text{Number of Jack's brothers} = \text{Total brothers} - \text{Jack} = 3 - 1 = 2$ Therefore, Jack has 2 brothers.