3 ms·

Sadly no matter what kind of prompting that I try it fails the sisters question: > Sally (a girl) has three brothers. Each of her brothers has two sisters. How

Sadly no matter what kind of prompting that I try it fails the sisters question: > Sally (a girl) has three brothers. Each of her brothers has two sisters. How many sisters does Sally have? Sally (a girl) has three brothers. Each of her brothers has two sisters. How many sisters does Sally have? Sally has two sisters. Here's the reasoning: Sally has three brothers. Each of her brothers has two sisters. Since her brothers are the ones who have two sisters each, and we know there are three brothers, we can conclude that each brother has two sisters. Therefore, there are two sisters in total (one for each brother). So, Sally has two sisters. https://www.lepton.ai/playground/chat?model=jetmoe-8b-chat https://www.lepton.ai/playground/chat?model=jetmoe-8b-chat

moffkalast 3y ago
Not exactly something you'd expect a model of this size to solve, Mixtral fails it too (if you switch the names/genders that is, since it's contaminated with a few versions). It does at least indicate that their training data might indeed be as clean as they say.
- vertis 3y ago
  More complex models fail this question, so it would be exceptional if it passed it: ollama run llama2:13b >>> Sally (a girl) has three brothers. Each of her brothers has two sisters. How many sisters does Sally have? This is a classic lateral thinking puzzle that requires you to think outside the box and challenge your assumptions. The answer is not what you might expect at first glance. To solve this puzzle, we need to break it down step by step: 1. Sally has three brothers. 2. Each of her brothers has two sisters. So, let's count the number of sisters that each of Sally's brothers has: Brother 1: 2 sisters Brother 2: 2 sisters Brother 3: 2 sisters Now, let's add up all the sisters that Sally has: Sister 1 + Sister 2 + Sister 3 + Sister 4 + Sister 5 = 2 + 2 + 2 + 2 + 2 = 8 Therefore, Sally has 8 sisters.

moffkalast 3y ago

Not exactly something you'd expect a model of this size to solve, Mixtral fails it too (if you switch the names/genders that is, since it's contaminated with a few versions). It does at least indicate that their training data might indeed be as clean as they say.

vertis 3y ago
More complex models fail this question, so it would be exceptional if it passed it: ollama run llama2:13b >>> Sally (a girl) has three brothers. Each of her brothers has two sisters. How many sisters does Sally have? This is a classic lateral thinking puzzle that requires you to think outside the box and challenge your assumptions. The answer is not what you might expect at first glance. To solve this puzzle, we need to break it down step by step: 1. Sally has three brothers. 2. Each of her brothers has two sisters. So, let's count the number of sisters that each of Sally's brothers has: Brother 1: 2 sisters Brother 2: 2 sisters Brother 3: 2 sisters Now, let's add up all the sisters that Sally has: Sister 1 + Sister 2 + Sister 3 + Sister 4 + Sister 5 = 2 + 2 + 2 + 2 + 2 = 8 Therefore, Sally has 8 sisters.