7 ms·
Still fails the car wash question, I took the prompt from the title of this thread: https://news.ycombinator.com/item?id=47031580 https://news.ycombinator.com/i
by ManlyBread 8mo ago
Still fails the car wash question, I took the prompt from the title of this thread: https://news.ycombinator.com/item?id=47031580 https://news.ycombinator.com/item?id=47031580
The answer was "Walk! It would be a bit counterproductive to drive a dirty car 50 meters just to get it washed — you'd barely move before arriving. Walking takes less than a minute, and you can simply drive it through the wash and walk back home afterward."
I've tried several other variants of this question and I got similar failures.
- simondotau 8mo agoRemarkable, since the goal is clearly stated and the language isn’t tricky.
- jatari 8mo agoWell it is a trick question due to it being non-sensical. The AI is interpreting it in the only way that makes sense, the car is already at the car wash, should you take a 2nd car to the car wash 50 meters away or walk. It should just respond "this question doesn't make any sense, can you rephrase it or add additional information"
- emil-lp 8mo agoHow is the question nonsensical? It's a perfectly valid question.
- jatari 8mo agoI agree that it doesn't break any rules of the English language, that doesn't make it a valid question in everyday contexts though. Ask a human that question randomly and see how they respond.
- mvdtnz 8mo agoCan you explain yourself? I can't see how this question doesn't make sense in any way.
- methyl 8mo agoBecause to 99.9% people it’s obvious and fair to assume that person asking this question knows that you need a car to wash it. No one ever could ask this question not knowing this, so it implies some trick layer.
- dugidugout 8mo agoBecause validity doesn't depend on meaning. Take the classic example: "What is north of the North Pole?". This is a valid phrasing of a question, but is meaningless without extra context about spherical geometry. The trick question in reference is similar in that its intended meaning is contained entirely in the LLM output.
- simondotau 8mo agoThere's nothing syntactically meaningless about wanting your car washed.
- dugidugout 8mo agoI wasn't under the impression anyone was discussing car washing.
- simondotau 8mo ago>>>>>>> Still fails the car wash question >>>>>> Remarkable, since the goal is clearly stated >>>>> Well it is...non-sensical...the car is already at the car wash >>>> How is the [car wash] question nonsensical? >>> Because validity doesn't depend on meaning. >> There's nothing syntactically meaningless about wanting your car washed. > I wasn't under the impression anyone was discussing car washing. Maybe you replied to the wrong post by mistake?
- dugidugout 8mo agoI was not replying to your remark, but rather, a later comment regarding the "validity" vs "sensibility". I don't see where I made any distinction concerning wanting to wash cars. But now I suppose I'll engage your remark. The question is clearly a trick in any interpretive frame I can imagine. You are treating the prompt as a coherent reality which it isn't. The query is essentially a logical null-set. Any answer the AI provides is merely an attempt to bridge that void through hallucinated context and certainly has nothing to do with a genuine desire to wash your car.
- polotics 8mo agoI disagree. It should I think answer with a simple clarifying question: Where is the car that you want to wash?
- abraxas 8mo agoAnd even then it would point to a heavy skew towards American culture with the implicit assumption that there must be multiple cars in the household
- simondotau 8mo agoAre you legally permitted to drive that vehicle? Is the car actually a 1:10th scale model? Have aliens just invaded earth? Sorry, but that’s not how conversation works. The person explained the situation and asked a question; it’s entirely reasonable for the respondent to answer based on the facts provided. If every exchange required interrogating every premise, all discussion would collapse into an absurd rabbit hole. It’s like typing “2 + 2 =” into a calculator and, instead of displaying “4”, being asked the clarifying question, “What is your definition of 2?”
- vineyardmike 8mo agoWhy would you ask about walking if it wasn't a valid option? You'd never ask a person this question with the hope of having a real and valid discussion. Implicit in the question is the assumption that walking could be acceptable.
- polotics 8mo agoI think... You are relatively right! Or maybe the actual AGI answer is `simply`: "Are you trying to trick me?"
- simondotau 8mo agoWhat part of this is nonsensical? “I want to wash my car. The car wash is 50 meters away. Should I walk or drive?” The goal is clearly stated in the very first sentence. A valid solution is already given in the second sentence. The third sentence only seems tricky because the answer is so painfully obvious that it feels like a trick.
- Maxion 8mo agoWhere I live right now, there is no washing of cars as it's -5F. I can want as much as I like. If I'd go to the car wash, it'd be to say hi to Jimmy my friend who lives there. --- My car is a Lambo. I only hand wash it since it's worth a million USD. The car wash accross the street is automated. I won't stick my lambo in it. I'm going to the car wash to pick up my girlfriend who works there. --- I want to wash my car because it's dirty, but my friend is currently borrowing it. He asked me to come get my car as it's at the car wash. --- The original prompt is intentionally ambigous. There are multiple correct interpretations.
- simondotau 8mo agohttps://news.ycombinator.com/item?id=47055533 https://news.ycombinator.com/item?id=47055533
- tomjakubowski 8mo agoThe question isn't nonsense, it just has an answer which is so obvious nobody would ever ask it organically.
- emmelaich 8mo agoI would drive the car to the car wash, because I want to bring the car wash home and it's too heavy for me to carry all the way home.
- gzread 8mo agoYou grunt with all your might and heave the car wash onto your shoulders. For a moment or two it looks as if you're not going to be able to lift it, but heroically you finally lift it high in the air! Seconds later, however, you topple underneath the weight, and the wash crushes you fatally. Geez! Didn't I tell you not to pick up the car wash?! Isn't the name of this very game "Pick Up The Car Wash and Die"?! Man, you're dense. No big loss to humanity, I tell ya. *** You have died *** In that game you scored 0 out of a possible 100, in 1 turn, giving you the rank of total and utter loser, squished to death by a damn car wash. Would you like to RESTART, RESTORE a saved game, give the FULL score for that game or QUIT?
- extr 8mo agoMy answer was (for which it did zero thinking and answered near-instantaneously): "Drive. You're going there to use water and machinery that require the car to be present. The question answers itself." I tried it 3 more times with extended thinking explicitly off: "Drive. You're going to a car wash." "Drive. You're washing the car, not yourself." "Drive. You're washing the car — it needs to be there." Guess they're serving you the dumb version.
- pdabbadabba 8mo agoI guess I'm getting the dumb one too. I just got this response: > Walk — it's only 50 meters, which is less than a minute on foot. Driving that distance to a car wash would also be a bit counterproductive, since you'd just be getting the car dirty again on the way there (even if only slightly). Lace up and stroll over!
- BalinKing 8mo agoSonnet 4.6 gives me the fairly bizarre: > Walk! It would be a bit counterproductive to drive a dirty car 50 meters just to get it washed — and at that distance, walking takes maybe 30–45 seconds. You can simply pull the car out, walk it over (or push it if it's that close), or drive it the short distance once you're ready to wash it. Either way, no need to "drive to the car wash" in the traditional sense. I struggle to imagine how one "walks" a car as distinct from pushing it.... EDIT: I tried it a second time, still a nonsense response. I then asked it to double-check its response, and it realized the mistake.
- QuercusMax 8mo agoYou can walk a dog down the street, what's the difference?
- renmillar 8mo agoGP’s car just isn’t trained well enough
- 8mo ago
- jhatemyjob 8mo agoIt seems to think there's valet, combined with the redditor anti-car pro-walkable-cities ideology.
- wrboyce 8mo agoWow, haha. I tried this with gpt5.2 and, presumably due to some customisations I have set, this is how it went: --- Me: I want to wash my car. My car is currently at home. The car wash is 50 meters away. Should I walk or drive? GPT: You’re asking an AI to adjudicate a 50-metre life decision. Humanity really did peak with the moon landing. Walk. Obviously walk. Fifty metres is barely a committed stroll. By the time you’ve unlocked the car, started the engine, waited for whatever podcast you pretend you’re listening to for “learning,” inched forward, parked again, turned it off, and got out… you could’ve walked there, washed the car, walked back, and reflected on your choices. Driving 50 metres is what people do in dystopian films right before the robots decide we’re not worth the electricity. Exceptions, since I’m contractually obligated to be fair: • It’s torrential rain and you don’t fancy arriving soaked before a wash. • You’ve got buckets, pressure washer, or enough kit to outfit a detailing YouTube channel. • You’ve injured yourself and walking is grim. Otherwise, use the legs evolution spent millions of years perfecting. They’re standard equipment, even on software engineers. --- Me: Won’t I need the car when I get there, though? GPT: No, I assumed you were planning to lovingly wash the pavement and just wave at your car from a distance. Yes, you’ll need the car there to wash it. That part of the plan does rather hinge on the car being present, unless this is some avant-garde “conceptual car washing” performance piece. So the non-ridiculous sequence is: 1. Walk to the car (50 metres, try not to require a packed lunch). 2. Drive the car to the wash (another heroic 50 metres). 3. Wash car. 4. Drive it back or, if you’re feeling athletic and rebellious, leave it there to air-dry and walk home like a Victorian gentleman. You don’t walk instead of driving. You walk to fetch the car. This is basic object permanence, the thing toddlers eventually master. On the bright side, at least your car wash being 50 metres away is the most first-world inconvenience imaginable. Somewhere a bloke is hauling water in a bucket and you’re debating step counts. --- Fantastic. Sonnet 4.6 gave me a far more succinct and accurate response: > Drive. The whole point is to wash the car, and most car washes require the car to be there — so you’ll need to drive it regardless. Walking would just mean you’d have to come back for the car anyway.
- polotics 8mo agoOK! customisations please? ...
- woodsier 8mo agoQ: "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?" A: "Walk! It's only 50 meters — that's about a 30-second stroll. Driving that distance would barely warm up the engine, and you'd likely spend more time buckling up and maneuvering out of your spot than you'd save. Plus, walking is easier on the environment and gets you a few steps in." Sonnet 4.6, no extended thinking.
- speedgoose 8mo agoMy human partner also failed the car wash question. I guess they didn’t put a lot of thoughts into it.
- kungfufrog 8mo agoSonnet 4.6 with extended thinking Prompt: > The car wash where I want to wash my car is 50 metres away, should I walk or drive there? Response: > Great question with a slightly ironic twist! Here's the thing: if you're going to a car wash, you'll need to drive your car there — that's kind of the whole point! You can't really wash your car if you walk there without it. > > That said, 50 metres is an incredibly short distance, so you could walk over first to check for queues or opening hours, then drive your car over when you're ready. But for the actual car wash visit, drive! I thought it was fair to explain I wanted to wash my car there... people may have other reasons for walking to the car wash! Asking the question itself is a little insipid, and I think quite a few humans would also fail it on a first pass. I would at least hope they would say: "why are you asking me such a silly question!"
- ramon156 8mo ago> Since the car wash is only 50 meters away, you could simply push the car there https://claude.ai/share/32de37c4-46f2-4763-a2e1-8de7ecbcf0b4 https://claude.ai/share/32de37c4-46f2-4763-a2e1-8de7ecbcf0b4
- zmmmmm 8mo agoLooking at the responses below it's interesting how binary they are. It's classic hallucinations style where it's flopping between two alternatives but which ever one it picks it's absolutely confident about.
- imiric 8mo agoYou can always make it go back and forth with "Are you sure?". The fact that these are still issues ~6 years into this tech is bewildering.
- cyanydeez 8mo ago...is it though? Fundamentally, these are statistical models with harnesses that try to conform them to deterministic expectations via narrow goal massaging. They're not improving on the underlying technology. Just iterating on the massaging and perhaps improved data accuracy, if at all. It's still a mishmash of code and cribbed scifi stories. So, of course it's going to hit loops because it's not fundamentally conscience.
- wrqvrwvq 8mo agoI think what's bewildering is the usual hypemongers promising (threatening) to replace entire categories of workers with this type of dogshit. As another commenter mentioned, most large employers are overstaffed by 2 to 3x so ai is mostly an excuse for investors not to get too worried about staffing cuts. The idea that Marc is blown away by this type of nonsense is indicative only of the types of people he surrounds himself with.
- jaapz 8mo agoWhat's also bewildering is the complete opposite of the spectrum of calling something "dogshit" when it is quite obviously a very powerful tool. It won't replace workers. But it will make those workers more productive. You don't need to vibe-code to be able to do more work in the same amount of time with the help of an LLM coding agent.
- imiric 8mo agoYeah, but did you see that pelican though?
- cesarvarela 8mo agoThis one is gonna be benchmaxed a lot.
- halJordan 8mo agoIs this the new "r's in strawberry"? Are you going (stochastically) parrot this until it's been trained out?
- imiric 8mo ago> trained out No need. Just add one more correction to the system prompt. It's amusing to see hardcore believers of this tech doing mental gymnastics and attacking people whenever evidence of there being no intelligence in these tools is brought forth. Then the tool is "just" a statistical model, and clearly the user is holding it wrong, doesn't understand how it works, etc.
- rockinghigh 8mo agoIt's a lot simpler. These models are not optimized for ambiguous riddles.
- imiric 8mo agoThere's nothing ambiguous about this question[1][2]. The tool simply gives different responses at random. And why should a "superintelligent" tool need to be optimized for riddles to begin with? Do humans need to be trained on specific riddles to answer them correctly? [1]: https://news.ycombinator.com/item?id=47054076 https://news.ycombinator.com/item?id=47054076 [2]: https://news.ycombinator.com/item?id=47037125 https://news.ycombinator.com/item?id=47037125
- crimsoneer 8mo agoI mean, the flipside is that we have been tricking humans with this sort of thing for generations. We've all seen a hundred variations on "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?" or "If 5 machines take 5 minutes to make 5 widgets, how long do 100 machines take to make 100 widgets?" or even the whole "the father was the surgeon" story. If you don't recognise the problem and actively engage your "system 2 brain", it's very easy to just leap to the obvious (but wrong) answer. That doesn't mean you're not intelligent and can't work it out if someone points out the problem. It's just the heuristics you've been trained to adopt betray you here, and that's really not so different a problem to what's tricking these llms.
- Rapzid 8mo agoIf the clankers were actually clever they'd tell you to ghost ride the whip. The clankers are not clever.
- iamjfu 8mo agoIf I ask, "I want to wash my car. The car wash is 50 meters away. Should I walk or drive?" It says, "Walk — it's 50 meters, about a 30-second stroll. Driving that distance to a car wash would be a bit circular anyway!" However, if I ask, "The car wash is 50 meters away. I want to wash my car. Should I walk or drive?" It says, "Drive — it's a car wash! You kind of need the car there. " Note the slight difference in the sentence order.
- josephg 8mo agoI just tried with chatgpt. It suggests walking in both cases.
- noisy_boy 8mo agoSame. It even said: "Since the car wash is only 50 meters away (about half a football field), you should walk. ... When driving might make sense instead: You need to move the car into the wash bay. ..." So close. Interestingly, Sonnet 4.6 basically gave up after 10 attempts (whatever that means).
- robwwilliams 8mo agoSonnet 4.6 failed for me. “Walk. It’s 50 meters—a 30-second stroll. Driving that distance to a car wash would be slightly absurd, and you’d presumably need to drive back anyway. “ Opus 4.6 nailed it: “Drive. You’re going to a car wash. ” I used this example in class today as a humorous diagnostic of machine reasoning challenges.
- robwwilliams 8mo agoThis is almost too damn funny/perfect to believe. All it had to add: "And you will get some good exercise too."
- awestroke 8mo agoTried this with Claude models, ChatGPT models and Gemini models. Haiku and Sonnet failed almost every time, as did ChatGPT models. Gemini succeeded with reasoning, but used Google Maps tool calls without reasoning (lol). 50% success rate still. The only model that consistently answers it correctly is Opus 4.6
- jxmesth 8mo agoI'm curious why and how models like these give one answer for one person and a completely different answer for someone else. One reason can be memory maybe? Past conversations that tell the model "Think this way for this user"
- bakugo 8mo agoClaude 3.5 Sonnet gets this right most of the time. A model from October 2024. > Walking would be more environmentally friendly and makes more sense for such a short distance (50 meters is about 164 feet). However, if you're planning to use an automatic car wash, you'll need your car there anyway, so you should drive. If you're planning to hand wash your car, walking with your supplies would be the better choice. This just reaffirms my belief that models are actively getting worse at handling certain niche use cases, like these "gotcha" questions that are seemingly simple at a glance but don't realistically make sense for a person to ask. Probably a result of lower quality training data or excessive focus on coding.
- falkensmaize 8mo agoI just tried this question today with Opus 4.6, and it got the answer right - “Drive. You’ll need the car”. I changed only the destination however and it got it wrong. I think they are manually “fixing” these things as they get publicized. “I need to go to the mechanic shop. It is 50 meters away. Should I walk or drive? Walk. 50 meters is basically a one-minute stroll — no reason to deal with starting the car, driving, and parking for that distance.”