3 ms·

But also why would you ask whether you should walk or drive if the car is at home? Either way the answer is obvious, and there is no way to interpret it except

But also why would you ask whether you should walk or drive if the car is at home? Either way the answer is obvious, and there is no way to interpret it except as a trick question. Of course, the parsimonious assumption is that the car is at home so assuming that the car is at the car wash is a questionable choice to say the least (otherwise there would be 2 cars in the situation, which the question doesn't mention).

scott_w 8mo ago
But you're ascribing understanding to the LLM, which is not what it's doing. If the LLM understood you, it would realise it's a trick question and, assuming it was British, reply with "You'd drive it because how else would you get it to the car wash you absolute tit." Even the higher level reasoning, while answering the question correctly, don't grasp the higher context that the question is obviously a trick question. They still answer earnestly. Granted, it is a tool that is doing what you want (answering a question) but let's not ascribe higher understanding than what is clearly observed - and also based on what we know about how LLMs work.
- layer8 8mo ago
  > They still answer earnestly. Gemini at least is putting some snark into its response: “Unless you've mastered the art of carrying a 4,000-pound vehicle over your shoulder, you should definitely drive. While 150 feet is a very short walk, it's a bit difficult to wash a car that isn't actually at the car wash!”
  - Barbing 8mo ago
    Marketing plan comes to mind for labs: find AI tells, fix them, & astroturf on socials that only _your_ frontier model reallly understands the world
DharmaPolice 8mo ago
I think a good rule of thumb is to default to assuming a question is asked in good faith (i.e. it's not a trick question). That goes for human beings and chat/AI models. In fact, it's particularly true for AI models because the question could have been generated by some kind of automated process. e.g. I write my schedule out and then ask the model to plan my day. The "go 50 metres to car wash" bit might just be a step in my day.
- vintermann 8mo ago
  Rule of thumb for who, humans or chatbots? For a human, who has their own wants and values, I think it makes perfect sense to wonder what on earth made the interlocutor ask that.
  - DharmaPolice 8mo ago
    Rule of thumb for everyone (i.e. both). If I ask you a question, start by assuming I want the answer to the question as stated unless there is a good reason for you to think it's not meant literally. If you have a lot more context (e.g. you know I frequently ask you trick or rhetorical questions or this is a chit-chat scenario) then maybe you can do something differently. I think being curious about the motivations behind a question is fine but it only really matters if it's going to affect your answer. Certainly when dealing with technical problem solving I often find myself asking extremely simple questions and it often wastes time when people don't answer directly, instead answering some completely different other question or demanding explanations why I'm asking for certain information when I'm just trying to help them.
    - vintermann 8mo ago
      Sure, in a context in which you're solving a technical problem for me, it's fair that I shouldn't worry too much about why you're asking - unless, for instance, I'm trying to learn to solve the question myself next time. Which sounds like a very common, very understandable reason to think about motivations. So even in that situation, it isn't simple. This probably sucks for people who aren't good at theory of mind reasoning. But surprisingly maybe, that isn't the case for chatbots. They can be creepily good at it, provided they have the context - they just aren't instruction tuned to ask short clarifying questions in response to a question, which humans do, and which would solve most of these gotchas.
      - DharmaPolice 8mo ago
        I don't mind people asking why I asked something, I'd just prefer they answer the question as well. In the original scenario, the chatbot could answer the question as written AND enquire if that's what they really meant. It's the StackOverflow syndrome where people answer a different question to the one posed. If someone asks "How can I do this on Windows?" - telling me that Windows sucks and here's how to do it on Linux is only slightly useful. Answer the question and feel free to mention how much easier it is in Linux by all means. I personally love explaining to people who might want to solve the issue next time so I'm happy to bore them to tears if they want. But don't let us delay solving the problem this time.
layer8 8mo ago
Therefore the correct response would be to inquire back to clarify the question being asked.

scott_w 8mo ago

But you're ascribing understanding to the LLM, which is not what it's doing. If the LLM understood you, it would realise it's a trick question and, assuming it was British, reply with "You'd drive it because how else would you get it to the car wash you absolute tit." Even the higher level reasoning, while answering the question correctly, don't grasp the higher context that the question is obviously a trick question. They still answer earnestly. Granted, it is a tool that is doing what you want (answering a question) but let's not ascribe higher understanding than what is clearly observed - and also based on what we know about how LLMs work.

layer8 8mo ago
> They still answer earnestly. Gemini at least is putting some snark into its response: “Unless you've mastered the art of carrying a 4,000-pound vehicle over your shoulder, you should definitely drive. While 150 feet is a very short walk, it's a bit difficult to wash a car that isn't actually at the car wash!”
- Barbing 8mo ago
  Marketing plan comes to mind for labs: find AI tells, fix them, & astroturf on socials that only _your_ frontier model reallly understands the world

DharmaPolice 8mo ago

I think a good rule of thumb is to default to assuming a question is asked in good faith (i.e. it's not a trick question). That goes for human beings and chat/AI models. In fact, it's particularly true for AI models because the question could have been generated by some kind of automated process. e.g. I write my schedule out and then ask the model to plan my day. The "go 50 metres to car wash" bit might just be a step in my day.

vintermann 8mo ago
Rule of thumb for who, humans or chatbots? For a human, who has their own wants and values, I think it makes perfect sense to wonder what on earth made the interlocutor ask that.
- DharmaPolice 8mo ago
  Rule of thumb for everyone (i.e. both). If I ask you a question, start by assuming I want the answer to the question as stated unless there is a good reason for you to think it's not meant literally. If you have a lot more context (e.g. you know I frequently ask you trick or rhetorical questions or this is a chit-chat scenario) then maybe you can do something differently. I think being curious about the motivations behind a question is fine but it only really matters if it's going to affect your answer. Certainly when dealing with technical problem solving I often find myself asking extremely simple questions and it often wastes time when people don't answer directly, instead answering some completely different other question or demanding explanations why I'm asking for certain information when I'm just trying to help them.
  - vintermann 8mo ago
    Sure, in a context in which you're solving a technical problem for me, it's fair that I shouldn't worry too much about why you're asking - unless, for instance, I'm trying to learn to solve the question myself next time. Which sounds like a very common, very understandable reason to think about motivations. So even in that situation, it isn't simple. This probably sucks for people who aren't good at theory of mind reasoning. But surprisingly maybe, that isn't the case for chatbots. They can be creepily good at it, provided they have the context - they just aren't instruction tuned to ask short clarifying questions in response to a question, which humans do, and which would solve most of these gotchas.
    - DharmaPolice 8mo ago
      I don't mind people asking why I asked something, I'd just prefer they answer the question as well. In the original scenario, the chatbot could answer the question as written AND enquire if that's what they really meant. It's the StackOverflow syndrome where people answer a different question to the one posed. If someone asks "How can I do this on Windows?" - telling me that Windows sucks and here's how to do it on Linux is only slightly useful. Answer the question and feel free to mention how much easier it is in Linux by all means. I personally love explaining to people who might want to solve the issue next time so I'm happy to bore them to tears if they want. But don't let us delay solving the problem this time.

layer8 8mo ago

Therefore the correct response would be to inquire back to clarify the question being asked.