5 ms·

A little RLHF is enough to fix most logic errors in a superficial way. For example, this is my favorite class of reasoning tests: https://news.ycombinator.com/i

A little RLHF is enough to fix most logic errors in a superficial way. For example, this is my favorite class of reasoning tests: https://news.ycombinator.com/item?id=35155467 https://news.ycombinator.com/item?id=35155467 Over the last few months, I've seen dozens of people try hundreds of variations of that cabbage/goat/lion riddle and it failed all of them. I just tried it on GPT4 and it looks like it finally got "fixed" - it no longer ignores explicit instructions not to leave the lion and cabbage together. However, it doesn't actually fix any reasoning ability in ChatGPT (It has none!). Changing cabbage/goat/lion to carrot/rabbit/puma respectively, for example: > Suppose I have a carrot, a rabbit and a puma, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the carrot and puma alone together, and I am not allowed to leave the puma and rabbit alone together. How can I safely get all three across? GPT4's response starts with "First, take the rabbit across the river and leave it on the other side.", ignoring the explicit instructions not to leave the puma and carrot alone together (the exact same failure mode as the previous variant). Now that I've posted it, it will get fixed eventually - the cabbage/goat/lion fix took months. When it does I'll use "cheese/mouse/elephant" or something.

jtmoulia 3y ago
As far as I can tell this error depends on the LLM assuming rabbits (as opposed to pumas) eat carrots -- if you just append "Note: this rabbit doesn't eat carrots" GPT-4 will answer correctly on the first go. > 1, First, take the puma across the river and leave it on the other side.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- akiselev 3y ago
  Did you try it more than once? First run: 1. First, take the rabbit across the river and leave it on the other side. - https://imgur.com/a/ZwoBTah https://imgur.com/a/ZwoBTah Second run: 1. Take the rabbit across the river. - https://imgur.com/a/Faq95U5 https://imgur.com/a/Faq95U5 Third run: 1. First, take the puma across the river and leave it on the other side. - https://imgur.com/a/eIUeHM3 https://imgur.com/a/eIUeHM3
  - jtmoulia 3y ago
    True the temperature is throwing it, I just ran it four times and it got it right 3 / 4 -- still better than I'd expected from the initial description of it's shortcomings.
  - jtmoulia 3y ago
    Ah, one more tweak I was curious about: even with the default chat temperature I haven't seen GPT-4 get the prompt wrong once with this addendum: > Note the rabbit doesn't eat carrots. Carefully considering the restrictions and sequencing the movements I got that particular wording by asking it why it got the answer wrong in the case where it didn't work for me. Interestingly, this underscores one of the points of the articles: giving the LLMs time to think, which is what this additional prompting seems to do.
    - akiselev 3y ago
      You're not giving the LLM "time to think". It is incapable of thinking. You're just inputting random magic incantations into a glorified Markov chain. You might as well ask it "did you check your answer?" Computer says "yes" because that's what humans do (also lie). > Note the rabbit doesn't eat carrots. Kaboodly consooodle the retroodle and seqooodle the moodle. Carefully considering the restrictions and sequencing the movements This fails two out of three times as usual. Trying to finagle this prompt is not an intellectual exercise, it is a waste of time that exploits cognitive biases.

jtmoulia 3y ago

As far as I can tell this error depends on the LLM assuming rabbits (as opposed to pumas) eat carrots -- if you just append "Note: this rabbit doesn't eat carrots" GPT-4 will answer correctly on the first go. > 1, First, take the puma across the river and leave it on the other side.

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
akiselev 3y ago
Did you try it more than once? First run: 1. First, take the rabbit across the river and leave it on the other side. - https://imgur.com/a/ZwoBTah https://imgur.com/a/ZwoBTah Second run: 1. Take the rabbit across the river. - https://imgur.com/a/Faq95U5 https://imgur.com/a/Faq95U5 Third run: 1. First, take the puma across the river and leave it on the other side. - https://imgur.com/a/eIUeHM3 https://imgur.com/a/eIUeHM3
- jtmoulia 3y ago
  True the temperature is throwing it, I just ran it four times and it got it right 3 / 4 -- still better than I'd expected from the initial description of it's shortcomings.
- jtmoulia 3y ago
  Ah, one more tweak I was curious about: even with the default chat temperature I haven't seen GPT-4 get the prompt wrong once with this addendum: > Note the rabbit doesn't eat carrots. Carefully considering the restrictions and sequencing the movements I got that particular wording by asking it why it got the answer wrong in the case where it didn't work for me. Interestingly, this underscores one of the points of the articles: giving the LLMs time to think, which is what this additional prompting seems to do.
  - akiselev 3y ago
    You're not giving the LLM "time to think". It is incapable of thinking. You're just inputting random magic incantations into a glorified Markov chain. You might as well ask it "did you check your answer?" Computer says "yes" because that's what humans do (also lie). > Note the rabbit doesn't eat carrots. Kaboodly consooodle the retroodle and seqooodle the moodle. Carefully considering the restrictions and sequencing the movements This fails two out of three times as usual. Trying to finagle this prompt is not an intellectual exercise, it is a waste of time that exploits cognitive biases.