8 ms·

> so you need to tell them the specifics That is the entire point, right? Us having to specify things that we would never specify when talking to a human. You

• anon_anon12 8mo ago
  Exactly, if an AI is able to curb around the basics, only then is it revolutionary
• Jacques2Marais 8mo ago
  You would be surprised, however, at how much detail humans also need to understand each other. We often want AI to just "understand" us in ways many people may not initially have understood us without extra communication.
  • j_maffe 8mo ago
    Right. But, unlike AI, we are usually aware when we're lacking context and inquire before giving an answer.
    • dxdm 8mo ago
      Wouldn't that be nice. I've been party and witness to enough misunderstandings to know that this is far from universally true, even for people like me who are more primed than average to spot missing context.
      • j_maffe 8mo ago
        I never said it's universally true.
    • mlrtime 8mo ago
      TIL my wife may be AI!
  • londons_explore 8mo ago
    This is why we fed it the whole internet and every library as training data... By now it should know this stuff.
    • jasongi 8mo ago
      Future models know it now, assuming they suck in mastodon and/or hacker news. Although I don't think they actually "know" it. This particular trick question will be in the bank just like the seahorse emoji or how many Rs in strawberry. Did they start reasoning and generalising better or did the publishing of the "trick" and the discourse around it paper over the gap? I wonder if in the future we will trade these AI tells like 0days, keeping them secret so they don't get patched out at the next model update.
      • Filligree 8mo ago
        The answer can be “both”. They won’t get this specific question wrong again; but also they generalise, once they have sufficient examples. Patching out a single failure doesn’t do it. Patch out ten equivalent ones, and the eleventh doesn’t happen.
        • MadxX79 8mo ago
          Yeah, the interpolation works if there are enough close examples around it. Problem is that the dimensionality of the space you are trying to interpolate in is so incomprehensibly big that even training on all of the internet, you are always going to have stuff that just doesn't have samples close by.
    • nkrisc 8mo ago
      Even I don’t “know” how many “R”s there are in “strawberry”. I don’t keep that information in my brain. What I do keep is the spelling of the word “strawberry” and the skill of being able to count so that I can derive the answer to that question anytime I need.
      • prmoustache 8mo ago
  • jstummbillig 8mo ago
    People poorly specifying problems and having bad models of what the other party can know (and then being surprised by the outcome) is certainly a more general albeit mostly separate issue.
    • ahofmann 8mo ago
      This issue is the main reason why a big percentage of jobs in the world exist. I don't have hard numbers, but my intuition is that about 30% of all jobs are mainly "understand what side a wants and communicate this to side b, so that they understand". Or another perspective: almost all jobs that are called "knowledge work" are like this. Software development is mainly this. Side a are humans, side b is the computer. The main goal of ai seems to get into this space and make a lot of people superflous and this also (partly) explains why everyone is pouring this amount of money into ai.
      • PaulRobinson 8mo ago
        Developers are - on average - terrible at this. If they weren't, TPMs, Product Managers, CTOs, none of them would need to exist. It's not specific to software, it's the entire World of business. Most knowledge work is translation from one domain/perspective to another. Not even knowledge work, actually. I've been reading some works by Adler[0] recently, and he makes a strong case for "meaning" only having a sense to humans, and actually each human each having a completely different and isolated "meaning" to even the simplest of things like a piece of stone. If there is difference and nuance to be found when it comes to a rock, what hope have we got when it comes to deep philosophy or the design of complex machines and software? LLMs are not very good at this right now, but if they became a lot better at, they would a) become more useful and b) the work done to get them there would tell us a lot about human communication. [0] https://en.wikipedia.org/wiki/Alfred_Adler https://en.wikipedia.org/wiki/Alfred_Adler
        • Scarblac 8mo ago
          I disagree, I feel (experienced) developers are excellent at this. It's always about translating between our own domain and the customer's, and every other new project there's a new domain to get up to speed with in enough detail to understand what to build. What other professions do that? That's why I'm somewhat scared of AIs - they know like 80% of the domain knowledge in any domain.
  • jiggawatts 8mo ago
    I regularly tell new people at work to be extremely careful when making requests through the service desk — manned entirely by humans — because the experience is akin to making a wish from an evil genie. You will get exactly what you asked for, not what you wanted… probably. (Random occurrences are always a possibility.) E.g.: I may ask someone to submit a ticket to “extend my account expiry”. They’ll submit: “Unlock Jiggawatts’ account” The service desk will reset my password (and neglect to tell me), leaving my expired account locked out in multiple orthogonal ways. That’s on a good day. Last week they created Jiggawatts2. The AIs have got to be better than this, surely! I suspect they already are. People are testing them with trick questions while the human examiner is on edge, aware of and looking for the twist. Meanwhile ordinary people struggle with concepts like “forward my email verbatim instead of creatively rephrasing it to what you incorrectly though it must have really meant.”
    • scott_w 8mo ago
      There's a lot of overlap between the smartest bears and the dumbest humans. However, we would want our tools to be more useful than the dumbest humans...
  • scott_w 8mo ago
    > You would be surprised, however, at how much detail humans also need to understand each other. But in this given case, the context can be inferred. Why would I ask whether I should walk or drive to the car wash if my car is already at the car wash?
    • pickleRick243 8mo ago
      But also why would you ask whether you should walk or drive if the car is at home? Either way the answer is obvious, and there is no way to interpret it except as a trick question. Of course, the parsimonious assumption is that the car is at home so assuming that the car is at the car wash is a questionable choice to say the least (otherwise there would be 2 cars in the situation, which the question doesn't mention).
      • scott_w 8mo ago
        But you're ascribing understanding to the LLM, which is not what it's doing. If the LLM understood you, it would realise it's a trick question and, assuming it was British, reply with "You'd drive it because how else would you get it to the car wash you absolute tit." Even the higher level reasoning, while answering the question correctly, don't grasp the higher context that the question is obviously a trick question. They still answer earnestly. Granted, it is a tool that is doing what you want (answering a question) but let's not ascribe higher understanding than what is clearly observed - and also based on what we know about how LLMs work.
        • layer8 8mo ago
          > They still answer earnestly. Gemini at least is putting some snark into its response: “Unless you've mastered the art of carrying a 4,000-pound vehicle over your shoulder, you should definitely drive. While 150 feet is a very short walk, it's a bit difficult to wash a car that isn't actually at the car wash!”
          • Barbing 8mo ago
  • kitd 8mo ago
    Given that an estimated 70% of human communication is non-verbal, it's not so surprising though.
    • WarmWash 8mo ago
      Does that stat predate the modern digital age by a number of years?
• ssl-3 8mo ago
  The question is so outlandish that it is something that nobody would ever ask another human. But if someone did, then they'd reasonably expect to get a response consisting 100% of snark. But the specificity required for a machine to deliver an apt and snark-free answer is -- somehow -- even more outlandish? I'm not sure that I see it quite that way.
  • coldtea 8mo ago
    >The question is so outlandish that it is something that nobody would ever ask another human There is an endless variety of quizes just like that humans ask other humans for fun, there is a whole lot of "trick questions" humans ask other humans to trip them up, and there are all kinds of seemingly normal questions with dumb assumptions quite close to that humans exchange.
  • jstummbillig 8mo ago
    I'd be entirely fine with a humorous response. The Gemini flash answer that was posted somewhere in this thread is delightful.
  • shakna 8mo ago
    But the number of outlandish requests in business logic is countless. Like... In most accounting things, once end-dated and confirmed, a record should cascade that end-date to children and should not be able to repeat the process... Unless you have some data-cleaning validation bypass. Then you can repeat the process as much as you like. And maybe not cascade to children. There are more exceptions, than there are rules, the moment you get any international pipeline involved.
    • ssl-3 8mo ago
      So, in human interaction: When the business logic goes wrong because it was described with a lack of specificity, then: Who gets blamed for this?
      • necovek 8mo ago
• BoredPositron 8mo ago
  I would ask you to stop being a dumb ass if you asked me the question...
  • coldtea 8mo ago
    Only to be tripped up by countless "hidden assumptions" questions similar to that that humans regularly get in
• nearbuy 8mo ago
  I think part of the failure is that it has this helpful assistant personality that's a bit too eager to give you the benefit of the doubt. It tries to interpret your prompt as reasonable if it can. It can interpret it as you just wanting to check if there's a queue. Speculatively, it's falling for the trick question partly for the same reason a human might, but this tendency is pushing it to fail more.
  • grey-area 8mo ago
    It’s just not intelligent or reasoning, and this sort of question exposes that more clearly. Surely anyone who has used these tools is familiar with the sometimes insane things they try to do (deleting tests, incorrect code, changing the wrong files etc etc). They get amazingly far by predicting the most likely response and having a large corpus but it has become very clear that this approach has significant limitations and is not general AI, nor in my view will it lead to it. There is no model of the world here but rather a model of words in the corpus - for many simple tasks that have been documented that is enough but it is not reasoning. I don’t really understand why this is so hard to accept.
    • fauigerzigerk 8mo ago
      I agree completely. I'm tempted to call it a clear falsification of any "reasoning" claim that some of these models have in their name. But I think it's possible that there is an early cost optimisation step that prevents a short and seemingly simple question even getting passed through to the system's reasoning machinery. However, I haven't read anything on current model architectures suggesting that their so called "reasoning" is anything other than more elaborate pattern matching. So these errors would still happen but perhaps not quite as egregiously.
      • nearbuy 8mo ago
        If you ask a bunch of people the same question in a context where they aren't expecting a trick question, some of them will say walk. LLMs sometimes say walk, and sometimes say drive. Maybe LLMs fall for these kinds of tricks more often than humans; I haven't seen any study try to measure this. But saying it's proof they can't reason is a double standard.
• nicbou 8mo ago
  I get that issue constantly. I somehow can't get any LLM to ask me clarifying questions before spitting out a wall of text with incorrect assumptions. I find it particularly frustrating.
  • Pxtl 8mo ago
    In general spitting out a scrollbar of text when asked a simple question that you've misunderstood is not, in any real sense, a "chat".
  • mk89 8mo ago
    The way I see it is that long game is to have agents in your life that memorize and understand your routine, facts, more and more. Imagine having an agent that knows about cars, and more specifically your car, when the checkups are due, when you washed it last time, etc., another one that knows more about your hobbies, another that knows more about your XYZ etc. The more specific they are, the more accurate they typically are.
    • viking123 8mo ago
      Do really understand deeply and in great amount I feel we would need models with changing weights and everyone would have their own so they could truly adjust to the user. Now we have have chunk of context that it may or may not use properly if it gets too long. But then again, how do we prevent it learning the wrong things if the weights are adjusting.
      • mk89 8mo ago
        In principle you're right but these things can get probably 60-70% of the job done. The rest is up to "you". Never rely on it blindly as we're being told kind of... :)
  • ash_091 8mo ago
    "If you're unsure, ask. Don't guess." in prompts makes a huge difference, imo.
• tgv 8mo ago
  > Us having to specify things that we would never specify This is known, since 1969, as the frame problem: https://en.wikipedia.org/wiki/Frame_problem https://en.wikipedia.org/wiki/Frame_problem. An LLM's grasp of this is limited by its corpora, of course, and I don't think much of that covers this problem, since it's not required for human-to-human communication.
  • ohyoutravel 8mo ago
    A modern LLMs corpora is every piece of human writing ever produced.
    • dryarzeg 8mo ago
      Not really, but even if it would be true, I don't think humans ever explained to each other why do you need to drive to car wash even if it's 50 meters away. It's pretty obvious and intuitive.
      • eru 8mo ago
        No, I think they have explained this to each other (or something like it). But as you suggested, discussion is a lot more likely when there are corner cases or problems.
      • wongarsu 8mo ago
        There has to be a lot of mentions about the purpose and approximate workings of a car wash, as well as lots of literature that shows that when you drive somewhere, your car is now also at that place, while walking does not have the same effect. It's then up to the model to make the connection "At the car wash people wash their car -> to wash your car you need your car to be present -> if you drive there your call will be there"
    • tgv 8mo ago
      Apart from the fact that that is utterly, demonstrably false, and the fact that corpora is plural, still the fact remains that we don't speak in those text about things that don't need to be spoken about. Hence the LLM will miss that underlying knowledge.
• vintermann 8mo ago
  But it's a question you would never ask a human! In most contexts, humans would say, "you are kidding, right?" or "um, maybe you should get some sleep first, buddy" rather than giving you the rational thinking-exam correct response. For that matter, if humans were sitting at the rational thinking-exam, a not insignificant number would probably second-guess themselves or otherwise manage to befuddle themselves into thinking that walking is the answer.
• ant6n 8mo ago
  > That is the entire point, right? Us having to specify things that we would never specify when talking to a human. I am not sure. If somebody asked me that question, I would try to figure out what’s going on there. What’s the trick. Of course I’d respond with asking specifics, but I guess the llvm is taught to be “useful” and try to answer as best as possible.
  • jason_oster 8mo ago
    One of the failure modes I find really frustrating is when I want a coding agent to make a very specific change, and it ends up doing a large refactor to satisfy my request. There is an easy solution, but it requires adding the instructions to the context: Require that any tasks that cannot be completed as requested (e.g., due to missing constraints, ambiguous instructions, or unexpected problems that would lead to unrelated refactors) should not be completed without asking clarifying questions. Yes, the LLM is trained to follow instructions at any cost because that's how its reward function works. They don't get bonus points for clearing up confusion, they get a cookie for doing the task. This research paper seems relevant: https://arxiv.org/abs/2511.10453v2 https://arxiv.org/abs/2511.10453v2
• LasEspuelas 8mo ago
  You would never ask a human this question. Right?
  • degamad 8mo ago
    We have a long tradition of asking each other riddles. A classic one asks, "A plane crashes on the border between France and Germany. Where do they bury the survivors?" Riddles are such a big part of the human experience that we have whole books of collections of them, and even a Batman villain named after them.
    • LasEspuelas 8mo ago
      Hmm... We ask riddles for fun and there is almost an expectation that a good riddle will yield a wrong answer.
• bluGill 8mo ago
  Real human in this situation will realize it is a joke after a few seconds of shock that you asked and laugh without asking more. If you really are seriout about the question they laugh harder thinking you are playing stupid for effect.
• ZaoLahma 8mo ago
  This reminds me of the "if you were entirely blind, how would you tell someone that you want something to drink"-gag, where some people start gesturing rather than... just talking. I bet a not insignificant portion of the population would tell the person to walk.
  • kenjackson 8mo ago
    Yes, there are thousands of videos of these sorts of pranks on TikTok. Another one. Ask some how to pronounce “Y, E, S”. They say “eyes”. Then say “add an E to the front of those letters - how do you pronounce that word”? And people start saying things like “E yes”.
• KronisLV 8mo ago
  > Us having to specify things that we would never specify when talking to a human. The first time I read that question I got confused: what kind of question is that? Why is it being asked? It should be obvious that you need your car to wash it. The fact that it is being asked in my mind implies that there is an additional factor/complication to make asking it worthwhile, but I have no idea what. Is the car already at the car wash and the person wants to get there? Or do they want to idk get some cleaning supplies from there and wash it at home? It didn't really parse in my brain.
  • Gabrys1 8mo ago
    I would say, the proper response to this question is not "walk, blablablah" but rather "What do you mean? You need to drive your car to have it washed. Did I miss anything?"
    • steveBK123 8mo ago
      Yes, this is what irks me about all the chatbots, and the chat interface as a whole. It is a chat-like UX without a chat-like experience. Like you are talking to a loquacious autist about their favorite topic every time. Just ask me a clarifying question before going into your huge pitch. Chats are a back & forth. You don’t need to give me a response 10x longer than my initial question. Etc
      • WarmWash 8mo ago
        I think for "GPT-4o is my life partner" reasons, labs are a little bit icey about making the models overly human.
        • steveBK123 8mo ago
          Doubt. The labs are afraid of users becoming too hooked on their products? lol…
          • 8mo ago
• gloosx 8mo ago
  In the end, formal, rule-based systems aka Programming Languages will be invented to instruct LLMs.
• dirkc 8mo ago
  > That is the entire point, right? Us having to specify things that we would never specify when talking to a human. Maybe in the distant future we'll realize that the most reliable way to prompting LLMs are by using a structured language that eliminates ambiguity, it will probably be rather unnatural and take some time to learn. But this will only happen after the last programmer has died and no-one will remember programming languages, compilers, etc. The LLM orbiting in space will essentially just call GCC to execute the 'prompt' and spend the rest of the time pondering its existence ;p
  • alistairSH 8mo ago
    Maybe in the distant future we'll realize that the most reliable way to prompting LLMs are by using a structured language that eliminates ambiguity So, back to COBOL? :)
    • chasd00 8mo ago
      > So, back to COBOL? :) well more like a structured _querying_ language
      • Ygg2 8mo ago
        So, back to Prolog? :)
  • nilamo 8mo ago
    A structured language without ambiguity is not, in general, how people think or express themselves. In order for a model to be good at interfacing with humans, it needs to adapt to our quirks. Convincing all of human history and psychology to reorganize itself in order to better service ai cannot possibly be a real solution. Unfortunately, the solution is likely going to be further interconnectivity, so the model can just ask the car where it is, if it's on, how much fuel/battery remains, if it thinks it's dirty and needs to be washed, etc
    • idiotsecant 8mo ago
• grog454 8mo ago
  > You would not start with "The car is functional [...]" Nope, and a human might not respond with "drive". They would want to know why you are asking the question in the first place, since the question implies something hasn't been specified or that you have some motivation beyond a legitimate answer to your question (in this case, it was tricking an LLM). Why the LLM doesn't respond "drive..?" I can't say for sure, but maybe it's been trained to be polite.
• tshaddox 8mo ago
  But you wouldn't have to ask that silly question when talking to a human either. And if you did, many humans would probably assume you're either adversarial or very dumb, and their responses could be very unpredictable.
• deleted 8mo ago
  [deleted]
• nonethewiser 8mo ago
  >That is the entire point, right? Us having to specify things that we would never specify when talking to a human. But the question is not clear to a human either. The question is confused. I read the headline and had no clue it was an LLM prompt. I read it 2 or 3 times and wondered "WTF is this shit?" So if you want an intelligent response from a human, you're going to need to adjust the question as well.
• perakojotgenije 8mo ago
  But you would also never ask such an obviously nonsensical question to a human. If someone asked me such a question my question back would be "is this a trick question?". And I think LLMs have a problem understanding trick questions.
  • Panda4 8mo ago
    I think that was somewhat the point of this, to simplify the future complex scenarios that can happen. Because problems that we need to use AI to solve will most of the times be ambiguous and the more complex the problem is the harder is it to pin-point why the LLM is failing to solve it.
• panarky 8mo ago
  > we can assume similar issues arise in more complex cases I would assume similar issues are more rare in longer, more complex prompts. This prompt is ambiguous about the position of the car because it's so short. If it were longer and more complex, there could be more signals about the position of the car and what you're trying to do. I must confess the prompt confuses me too, because it's obvious you take the car to the car wash, so why are you even asking? Maybe the dirty car is already at the car wash but you aren't for some reason, and you're asking if you should drive another car there? If the prompt was longer with more detail, I could infer what you're really trying to do, why you're even asking, and give a better answer. I find LLMs generally do better on real-world problems if I prompt with multiple paragraphs instead of an ambiguous sentence fragment. LLMs can help build the prompt before answering it. And my mind works the same way.
  • deleted 8mo ago
    [deleted]
  • qingcharles 8mo ago
    The question isn't something you'd ask another human in all seriousness, but it is a test of LLM abilities. If you asked the question to another human they would look at you sideways for asking such a dumb question, but they could immediately give you the correct answer without hesitation. There is no ambiguity when asking another human. This question goes in with the "strawberry" question which LLMs will still get wrong occasionally.
• davrosthedalek 8mo ago
  We would also not ask somebody if I should walk or drive. In fact, if somebody would ask me in a honest, this is not a trick question, way, I would be confused and ask where the car is. It seems chatgpt now answers correctly. But if somebody plays around with a model that gets it wrong: What if you ask it this: "This is a trick question. I want to wash my car. The car wash is 50 m away. Should I drive or walk?"
• mrighele 8mo ago
  It is true that we don't need to specify some things, and that is nice. It is though also the reason why software is often badly specified and corner cases are not handled. Of course the car is ALWAYS at home, in working condition, filled with gas and you have your driving license with you.
• IanCal 8mo ago
  I have an issue with these kinds of cases though because they seem like trick questions - it's an insane question to ask for exactly the reasons people are saying they get it wrong. So one possible answer is "what the hell are you talking about?" but the other entirely reasonable one is to assume anything else where the incredibly obvious problem of getting the car there is solved (e.g. your car is already there and you need to collect it, you're asking about buying supplies at the shop rather than having it washed there, whatever). Similarly with "strawberry" - with no other context an adult asking how many r's are in the word a very reasonable interpretation is that they are asking "is it a single or double r?". And trick questions are commonly designed for humans too - like answering "toast" for what goes in a toaster, lots of basic maths things, "where do you bury the survivors", etc.
  • RobMurray 8mo ago
    strawberry isn't a trick question. llms jus don't sea letters like that. I just asked chatgpt how many Rs are in "Air Fryer" and it said two, one in air and one in fryer. I do think it can be useful though that these errors still exist. They can break the spell for some who believe models are conscious or actually possess human intelligence. Of course there will always be people who become defensive on behalf of the models as if they are intelligent but on the spectrum and that we are just asking the wrong questions.
• sebazzz 8mo ago
  > > so you need to tell them the specifics > That is the entire point, right? Honestly it is a problem with using GPT as a coding agent. It would literally rewrite the language runtime to make a bad formula or specification work. That's what I like with Factory.ai droid: making the spec with one agent and implementing it with another agent.
  • verdverm 8mo ago
    > It would literally rewrite the language runtime If you let the agent go down this path, that's on you not the agent. Be in the loop more > making the spec with one agent and implementing it with another agent You don't need a specialized framework to do this, just read/write tools. I do it this way all the time
• keeda 8mo ago
  The broad point about assumptions is correct, but the solution is even simpler than us having to think of all these things; you can essentially just remind the model to "think carefully" -- without specifying anything more -- and they will reason out better answers: https://news.ycombinator.com/item?id=47040530 https://news.ycombinator.com/item?id=47040530 When coding, I know they can assume too much, and so I encourage the model to ask clarifying questions, and do not let it start any code generation until all its doubts are clarified. Even the free-tier models ask highly relevant questions and when specified, pretty much 1-shot the solutions. This is still wayyy more efficient than having to specify everything because they make very reasonable assumptions for most lower-level details.
• jason_oster 8mo ago
  > Us having to specify things that we would never specify when talking to a human. Interesting conclusion! From the Mastodon thread: > To be fair it took me a minute, too I presume this was written by a human. (I'll leave open the possibility that it was LLM generated.) So much for "never" needing to specify ambiguous scenarios when talking to a human.
• rainsford 8mo ago
  This example and others like it really reinforce for me the idea that LLMs fundamentally don't "understand" things the same way humans do and it's not a problem that's going to be fixed by more training or more GPUs. Generative AI is cool and can do impressive stuff, but despite being many generations into the models now with ever improved capabilities, we're constantly given little reminders like this that they're not actually intelligent. And in my opinion, they're unlikely to ever get there absent some fundamentally disruptive change in how they work rather than just iteratively better models. This is probably OK...LLMs don't have to be AGI to be useful. But it is worthwhile being realistic about their limitations because it's often easy to forget without seeing examples like this. And as you point out, the impact of those limitations is often not as obvious.
• Neywiny 8mo ago
  That's my thought too. Somebody I know kept insisting it's about prompt engineering. "You are an expert coder with 30 years experience" and buddy I'd rather do actual engineering and be that expert myself than spend and figuring out how on that one variant of one version of one model to get halfway decent results.
• tom_m 8mo ago
  Oh no? Things we would never have to specify to a human? This is precisely how software gets made and how software ends up with bugs. It's amazing how many things I saw over the years where I said the same exact thing; "but you shouldn't have to tell anyone that."
• AYBABTME 8mo ago
  If a human asked me this question, I would be confused by the question as ambiguous since it suggests something odd is implied but underspecified. I think any confident answer either way by AI is lacking in pedantry.