5 ms·

So I know people are probably tired of posts that just plug something into ChatGPT, but I think it might be interesting to see how GPT3.5/ChatGPT might handle i

So I know people are probably tired of posts that just plug something into ChatGPT, but I think it might be interesting to see how GPT3.5/ChatGPT might handle it. It got the answer to the hen problem wrong every time. I kept modifying the prompt to let it know it was a tricky problem, to think step by step, to self-critique and adjust, & tried at temperatures of 0, 0.7, and 1 multiple times and after a dozen tries it kept giving either 12, 36 (most common), or 144. (Starting a new session every time.) > This is a tricky arithmetic word problem. Think step by step. After having an initial answer, critique it, double check it, and change it to the correct answer if necessary: 3 hens lay 3 eggs in 3 days. How many eggs do 12 hens give in 12 days? Again, show your work and don’t just give the answer right away but reason through the problem. That is, UNTIL I gave it the specific advice that the article gave, adding this to the beginning of the above prompt (with another fresh session): >I know that humans are biased for intuitive understanding rather than logical one. Intuitions are shortcuts that are very valuable, but sometimes they are just plainly wrong. …and then it got the right answer immediately, first try. So this trick isn’t just a neat blog post, but empirically useful, and I’ll probably include it in prompts from now on for these kinds of problems… (And note this is a blog post from today, so the blog post was not in the training set.)

mlajtos 4y ago
I love your comment. I tried GPT-4 via ChatGPT and it nailed it without any prompt engineering. --- Prompt: 3 hens lay 3 eggs in 3 days. How many eggs do 12 hens give in 12 days? GPT-4: Let's first determine how many eggs one hen can lay in 12 days. We know that 3 hens can lay 3 eggs in 3 days. This means that each hen lays 1 egg in 3 days (3 hens / 3 eggs = 1 hen / 1 egg). To find out how many eggs one hen can lay in 12 days, we need to multiply the number of eggs laid in 3 days by 4 (since 12 days is 4 times longer than 3 days). So, one hen will lay 1 egg * 4 = 4 eggs in 12 days. Now, we have 12 hens. To find out how many eggs they can lay in 12 days, we just multiply the number of eggs one hen can lay in 12 days by the number of hens: 12 hens * 4 eggs per hen = 48 eggs in 12 days. --- To be completely fair, this math problem can be googled, so it is probably in the training set.
- gorpy7 4y ago
  somehow this answer reminded me of the good advice for starting to tighten a nut. first turn left till you feel the nut drop- then turn right. I feel there is a universal lesson here. Maybe going, as the crow flies, directly toward your goal isn’t always best.
- xyzzy123 4y ago
  Interestingly if you give GPT-4 the problem but append the instruction "answer only, no explanation" it gets it wrong (I got "108" as the answer in that case). It is fascinating how much it needs to work step by step.
codeflo 4y ago
Very much yes to your first sentence, but I don’t think “just plug something into ChatGPT” is a fair summary of your (very interesting) post, so you’re fine with me. I’ve seen several statements recently, including one in a talk by a Microsoft researcher working with OpenAI, that claim that all the RLHF safety training introduces human biases and makes the models worse at abstract reasoning. I really wonder why a relatively small amount of human feedback has that effect, while training on this gigantic human-written corpus doesn’t seem to introduce the same biases. That hints at something interesting going on.
- mlajtos 4y ago
  We are gonna need to prefix all prompts with "From now on, act without human biases."
- panarky 4y ago
  To human ears, the problem of 3 hens, 3 eggs, 3 days is a simplified and more obvious problem than the original: "If a hen and a half can lay an egg and a half in a day and a half ..." But it's the same problem in the same form, and GPT4 nails it without extra goofy prompting like "don't replicate human cognitive biases". A more interesting question would cross logic with problem solving. "If a hen and a half can lay an egg and a half in a day and a half, how many hens are needed to fill Yankee Stadium with eggs in a year?" But GPT4 doesn't have very good intuition on how to estimate the volume of Yankee Stadium: Yankee Stadium has a seating capacity of approximately 50,000 spectators. Let's assume each spectator occupies a space of 2 cubic meters (which is a generous estimate, including seating and legroom). Total volume (approx.) = 50,000 seats × 2 m³/seat = 100,000 m³
kqr 4y ago
> So this trick isn’t just a neat blog post, but empirically useful, and I’ll probably include it in prompts from now on for these kinds of problems… Alernative take: this is a very classic problem used to illustrate this kind of System 1 inaccuracy, so by deliberately including words that increase the conditional probability of us looking at a text that includes discussion on System 1 inaccuracies, you have essentially narrowed down the search space for our dear stochastic parrot to those texts that do contain the right answer. This sounds dismissive, but I don't mean it that way. I'm just saying that "think about human biases" may not be a universally useful cue, it may only work specifically when trying to get the model to look like a discussion on human biases, and the typical sample problems used to illustrate that.
still_grokking 4y ago
I didn't know this "puzzle" until just now. Stupid question: What is the "tricky" part about it? How can someone who isn't completely retarded get this even wrong? This puzzle is trivial, and especially doesn't require any kind of "thinking around the corner". So what is this about? That language models can't do math (or logic) and can't even reliably tell which of two given written out numbers is the bigger one is imho a different story. Language models are great for text. But when you need a tool for math and/or logic you should use an adequate tool. We have for example algebra systems for that. Or prove assistants. Or just good old Prolog. It makes no sense to use the wrong tool and than wonder that the results are terribly wrong. But for the above "puzzle" you don't even need a calculator. So I really don't get the point.
- alanfalcon 4y ago
  It is tricky because humans are biased for intuitive understanding rather than logical understanding. Intuitions are shortcuts that are very valuable, but sometimes they are just plainly wrong. Many people will try to apply intuition rather than logic because that works with many other similarly worded questions.
  - still_grokking 4y ago
    Still don't get it. The imho intuitive answer is the right one. I really tried to get what could be tricky about this question. But there is just nothing. I have no clue how anybody could arrive at anything else[†] than 48. It's trivial. There is no "puzzle" at all. † (As long as you don't consider RL biology, as than then there is no definite answer, only some probability.)
    - Robotbeat 4y ago
      For a 10 year old.
      - still_grokking 4y ago
        I would expect every not retarded 10 year old to give the correct answer. We're talking here about third or forth graders. Those kids should be able to do basic arithmetic with small numbers.
Kranar 4y ago
With GPT 4 it nailed it on the first try: >3 hens lay 3 eggs in 3 days. How many eggs do 12 hens give in 12 days? >Let's break down the given information and solve this problem step by step. We know that 3 hens lay 3 eggs in 3 days. First, we'll find out how many eggs 1 hen lays in 3 days. 3 hens lay 3 eggs, so 1 hen lays 1 egg in 3 days. Now, let's find out how many eggs 1 hen lays in 12 days. In 3 days, 1 hen lays 1 egg, so in 12 days, 1 hen will lay (12 / 3) * 1 = 4 eggs. Finally, we'll find out how many eggs 12 hens lay in 12 days. If 1 hen lays 4 eggs in 12 days, then 12 hens will lay 12 * 4 = 48 eggs in 12 days.

mlajtos 4y ago

I love your comment. I tried GPT-4 via ChatGPT and it nailed it without any prompt engineering. --- Prompt: 3 hens lay 3 eggs in 3 days. How many eggs do 12 hens give in 12 days? GPT-4: Let's first determine how many eggs one hen can lay in 12 days. We know that 3 hens can lay 3 eggs in 3 days. This means that each hen lays 1 egg in 3 days (3 hens / 3 eggs = 1 hen / 1 egg). To find out how many eggs one hen can lay in 12 days, we need to multiply the number of eggs laid in 3 days by 4 (since 12 days is 4 times longer than 3 days). So, one hen will lay 1 egg * 4 = 4 eggs in 12 days. Now, we have 12 hens. To find out how many eggs they can lay in 12 days, we just multiply the number of eggs one hen can lay in 12 days by the number of hens: 12 hens * 4 eggs per hen = 48 eggs in 12 days. --- To be completely fair, this math problem can be googled, so it is probably in the training set.

gorpy7 4y ago
somehow this answer reminded me of the good advice for starting to tighten a nut. first turn left till you feel the nut drop- then turn right. I feel there is a universal lesson here. Maybe going, as the crow flies, directly toward your goal isn’t always best.
xyzzy123 4y ago
Interestingly if you give GPT-4 the problem but append the instruction "answer only, no explanation" it gets it wrong (I got "108" as the answer in that case). It is fascinating how much it needs to work step by step.

codeflo 4y ago

Very much yes to your first sentence, but I don’t think “just plug something into ChatGPT” is a fair summary of your (very interesting) post, so you’re fine with me. I’ve seen several statements recently, including one in a talk by a Microsoft researcher working with OpenAI, that claim that all the RLHF safety training introduces human biases and makes the models worse at abstract reasoning. I really wonder why a relatively small amount of human feedback has that effect, while training on this gigantic human-written corpus doesn’t seem to introduce the same biases. That hints at something interesting going on.

mlajtos 4y ago
We are gonna need to prefix all prompts with "From now on, act without human biases."
panarky 4y ago
To human ears, the problem of 3 hens, 3 eggs, 3 days is a simplified and more obvious problem than the original: "If a hen and a half can lay an egg and a half in a day and a half ..." But it's the same problem in the same form, and GPT4 nails it without extra goofy prompting like "don't replicate human cognitive biases". A more interesting question would cross logic with problem solving. "If a hen and a half can lay an egg and a half in a day and a half, how many hens are needed to fill Yankee Stadium with eggs in a year?" But GPT4 doesn't have very good intuition on how to estimate the volume of Yankee Stadium: Yankee Stadium has a seating capacity of approximately 50,000 spectators. Let's assume each spectator occupies a space of 2 cubic meters (which is a generous estimate, including seating and legroom). Total volume (approx.) = 50,000 seats × 2 m³/seat = 100,000 m³

kqr 4y ago

> So this trick isn’t just a neat blog post, but empirically useful, and I’ll probably include it in prompts from now on for these kinds of problems… Alernative take: this is a very classic problem used to illustrate this kind of System 1 inaccuracy, so by deliberately including words that increase the conditional probability of us looking at a text that includes discussion on System 1 inaccuracies, you have essentially narrowed down the search space for our dear stochastic parrot to those texts that do contain the right answer. This sounds dismissive, but I don't mean it that way. I'm just saying that "think about human biases" may not be a universally useful cue, it may only work specifically when trying to get the model to look like a discussion on human biases, and the typical sample problems used to illustrate that.

still_grokking 4y ago

I didn't know this "puzzle" until just now. Stupid question: What is the "tricky" part about it? How can someone who isn't completely retarded get this even wrong? This puzzle is trivial, and especially doesn't require any kind of "thinking around the corner". So what is this about? That language models can't do math (or logic) and can't even reliably tell which of two given written out numbers is the bigger one is imho a different story. Language models are great for text. But when you need a tool for math and/or logic you should use an adequate tool. We have for example algebra systems for that. Or prove assistants. Or just good old Prolog. It makes no sense to use the wrong tool and than wonder that the results are terribly wrong. But for the above "puzzle" you don't even need a calculator. So I really don't get the point.

alanfalcon 4y ago
It is tricky because humans are biased for intuitive understanding rather than logical understanding. Intuitions are shortcuts that are very valuable, but sometimes they are just plainly wrong. Many people will try to apply intuition rather than logic because that works with many other similarly worded questions.
- still_grokking 4y ago
  Still don't get it. The imho intuitive answer is the right one. I really tried to get what could be tricky about this question. But there is just nothing. I have no clue how anybody could arrive at anything else[†] than 48. It's trivial. There is no "puzzle" at all. † (As long as you don't consider RL biology, as than then there is no definite answer, only some probability.)
  - Robotbeat 4y ago
    For a 10 year old.
    - still_grokking 4y ago
      I would expect every not retarded 10 year old to give the correct answer. We're talking here about third or forth graders. Those kids should be able to do basic arithmetic with small numbers.

Kranar 4y ago

With GPT 4 it nailed it on the first try: >3 hens lay 3 eggs in 3 days. How many eggs do 12 hens give in 12 days? >Let's break down the given information and solve this problem step by step. We know that 3 hens lay 3 eggs in 3 days. First, we'll find out how many eggs 1 hen lays in 3 days. 3 hens lay 3 eggs, so 1 hen lays 1 egg in 3 days. Now, let's find out how many eggs 1 hen lays in 12 days. In 3 days, 1 hen lays 1 egg, so in 12 days, 1 hen will lay (12 / 3) * 1 = 4 eggs. Finally, we'll find out how many eggs 12 hens lay in 12 days. If 1 hen lays 4 eggs in 12 days, then 12 hens will lay 12 * 4 = 48 eggs in 12 days.