4 ms·

I don’t think it’s under specified. You are clearly stating “I want to wash my car”, then asking how you should get there. It’s an easy logical step to know t

by dozerly 8mo ago

I don’t think it’s under specified. You are clearly stating “I want to wash my car”, then asking how you should get there. It’s an easy logical step to know that, in this context, you need your car with you to wash it, and so no matter the distance you should drive. You can ask the human race the simplest, most logical question ever, and a percentage of them will get it wrong.

snmx999 8mo ago
The question does not specify where you or the car are. It specifies only that the car wash is 50 meters away from something, possibly you, the car, or both.
- username44 8mo ago
  This is an interesting point, but even when you are more specific ChatGPT says to walk. https://chatgpt.com/share/699d2d1b-51f0-8003-9c63-af9bb5bcf849 https://chatgpt.com/share/699d2d1b-51f0-8003-9c63-af9bb5bcf8...
- mk89 8mo ago
  It could also mean there is literally no possible way to reach it, because that's on the other side of a river, and there is no bridge. You should still not "walk there, because come on don't be lazy, a bit of walking is good".
  - 1718627440 8mo ago
    This. To be correct you must also give the answer for the right reason. If you say "drive" but for the wrong reason, then you are still wrong.
mdorazio 8mo ago
In addition to snmx999's point, you're also not specifying that you want to wash your car at the car wash (as opposed to washing it in your driveway or something, in which case the car wash is superfluous information). The article's prompt failed in Sonnet 4.6, but the one below works fine. I think more humans would get it right as well. I want to wash my car at the car wash. The car wash is 50 meters away and my car is in my driveway. Should I walk or drive?
aurareturn 8mo ago
1. When do you want to wash your car? Tomorrow? Next year? In 50 years? 2. Where is the car now? Is it already at the car wash waiting for you to arrive? I can see why an LLM might miss this. I think any good software engineer would ask clarifying questions before giving an answer. The next step for an LLM is to either ask questions before giving a definitive answer for uncertain things or to provide multiple answers addressing the uncertainty.
- kklisura 8mo ago
  3. Is the car broken somewhere? Does it have wheels on? 4. Does the car have enough fuel? Jokes asides, all of those questions are unnecessary. There's no more context to this.
  - aurareturn 8mo ago
    If you ask a human that in person, they'd wonder why you'd ask such as stupid question. I think LLMs should ask clarifying questions if it thinks it's a trick question.

by dozerly 8mo ago

snmx999 8mo ago

The question does not specify where you or the car are. It specifies only that the car wash is 50 meters away from something, possibly you, the car, or both.

username44 8mo ago
This is an interesting point, but even when you are more specific ChatGPT says to walk. https://chatgpt.com/share/699d2d1b-51f0-8003-9c63-af9bb5bcf849 https://chatgpt.com/share/699d2d1b-51f0-8003-9c63-af9bb5bcf8...
mk89 8mo ago
It could also mean there is literally no possible way to reach it, because that's on the other side of a river, and there is no bridge. You should still not "walk there, because come on don't be lazy, a bit of walking is good".
- 1718627440 8mo ago
  This. To be correct you must also give the answer for the right reason. If you say "drive" but for the wrong reason, then you are still wrong.

mdorazio 8mo ago

In addition to snmx999's point, you're also not specifying that you want to wash your car at the car wash (as opposed to washing it in your driveway or something, in which case the car wash is superfluous information). The article's prompt failed in Sonnet 4.6, but the one below works fine. I think more humans would get it right as well. I want to wash my car at the car wash. The car wash is 50 meters away and my car is in my driveway. Should I walk or drive?

aurareturn 8mo ago

1. When do you want to wash your car? Tomorrow? Next year? In 50 years? 2. Where is the car now? Is it already at the car wash waiting for you to arrive? I can see why an LLM might miss this. I think any good software engineer would ask clarifying questions before giving an answer. The next step for an LLM is to either ask questions before giving a definitive answer for uncertain things or to provide multiple answers addressing the uncertainty.

kklisura 8mo ago
3. Is the car broken somewhere? Does it have wheels on? 4. Does the car have enough fuel? Jokes asides, all of those questions are unnecessary. There's no more context to this.
- aurareturn 8mo ago
  If you ask a human that in person, they'd wonder why you'd ask such as stupid question. I think LLMs should ask clarifying questions if it thinks it's a trick question.