4 ms·
> Asimov's laws of robotics are flawed too, of course. Almost all of Asimovs writing about the three laws is written as a warning of sorts that language cannot
by dijit 6mo ago
> Asimov's laws of robotics are flawed too, of course.
Almost all of Asimovs writing about the three laws is written as a warning of sorts that language cannot properly capture intent.
He would be the very first person to say that they are flawed, that is the intent of them.
He uses robots and AI as the creatures that understand language but not intent, and, funnily enough that's exactly what LLMs do... how weird.
- atleastoptimal 6mo agoLLM's now can capture intent. I think the issue now is that the full landscape of human values never resolves cleanly when mapped from the things we state in writing as being human values. Asimov tried to capture this too, as in, if a robot was tasked with "always protect human life", would it necessarily avoid killing at all costs? What if killing someone would save the lives of 2 others? The infinite array of micro-trolly problems that dot the ethical landscape of actions tractable (and intractable) to literate humans makes a full-consistent accounting of human values impossible, thus could never be expected from a robot with full satisfaction.
- dijit 6mo ago“LLMs can capture intent now” reads to me the same as: AI has emotions now, my AI girlfriend told me so. I don’t discredit you as a person or a professional, but we meatbags are looking for sentience in things which don’t have it, thats why we anthropomorphise things constantly, even as children. We are easily fooled and misled.
- atleastoptimal 6mo agoLLM's capturing intent is a capabilities-level discussion, it is verifiable, and is clear just via a conversation with Claude or Chatgpt. Whether they have emotions, an internal life or whatever is an unfalsifiable claim and has nothing to do with capabilities. I'm not sure why you think the claim that they can capture intent implies they have emotions, it's simply a matter of semantic comprehension which is tied to pattern recognition, rhetorical inference, etc that are all naturally comprehensible to a language model.
- quirkot 6mo ago[dead]
- tvink 6mo agoIf it is verifiable, please show us. What if clear to you reeks delusion to me.
- svnt 6mo agoLook at any recent CoT output where the model is trying to infer from an underspecified prompt what the user wants or means. It is generally the first thing they do — try to figure out what did you mean with this prompt. When they can’t infer your intent, good models ask follow-on questions to clarify. I am wondering if this is a semantics issue as this is an established are of research, eg https://arxiv.org/pdf/2501.10871 https://arxiv.org/pdf/2501.10871
- batshit_beaver 6mo agoRight, and then look at any number of research papers showing that CoT output has limited impact on the end result. We've trained these models to pretend to reason.
- atleastoptimal 6mo agoIf it's only pretending to reason, then how is it that the CoT output improves performance on every single benchmark/test?
- Eisenstein 6mo ago> Right, and then look at any number of research papers showing that CoT output has limited impact on the end result. Which research papers? Do I have to find them? > We've trained these models to pretend to reason. I have no idea why that matters. Can you tell me what the difference is if it looks exactly the same and has the same result?
- semiquaver 6mo agoWhat do you think it means to “capture intent” and where do current models fall short on this description? From my perspective the models are pretty good at “understanding” my intent, when it comes to describing a plan or an action I want done but it seems like you might be using a different definition. Tell me, what’s your intent? :)
- dijit 6mo ago[dead]
- svnt 6mo agoThis lack of understanding is a you problem, not a them problem. Your definitions for these terms are too imprecise.
- deleted 6mo ago[deleted]
- Guvante 6mo ago> LLM's now can capture intent. Humans cannot capture intent so how can AI? It is well established that understanding what someone meant by what they said is not a generally solvable problem, akin to the three body problem. Note of course this doesn't mean you can't get good enough almost all of the time, but it in the context here that isn't good enough. After all the entire Asimov story is about that inability to capture intent in the absolute sense.
- bicepjai 6mo ago> LLM's now can capture intent No they can’t. Here is an example: Ask an llm to write a multi phase plan for a very large multi file diff that it created, with least ambiguity, most continuity across plans; let’s see if it can understand your intent.
- canjobear 6mo agoI think you're vastly underestimating how little of human intent is really encoded in language in a strict sense, and how much nontrivial inference of intents LLMs do every day with simple queries. This used to be an apparently insurmountable barrier in pre-LLM NLP, and now it is just not a problem. Suppose I'm in a cold room, you're standing next to a heater, and I say "it's cold". Obviously my intent is that I want you to turn on the heater. But the literal semantics is just "the ambient temperature in the room is low" and it has nothing to do with heaters. Yet ChatGPT can easily figure out likely intent in situations like this, just as humans do, often so quickly and effortlessly that we don't notice the complexity of the calculation we did. Or suppose I say to a bot "tell me how to brew a better cup of coffee". What is encoded in the literal meaning of the language here? Who's to say that "better" means "better tasting" as opposed to "greater quantity per unit input"? Or that by "cup of coffee" I mean the liquid drink, as opposed to a cup full of beans? Or perhaps a cup that is made out of coffee beans? In fact the literal meaning doesn't even make sense, as a "cup" is not something that is brewed, rather it is the coffee that should go into the cup, possibly via an intermediate pot. If the bot only understands literal language then this kind of query is a complete nonstarter. And yet LLMs can handle these kinds of things easily. If anything they struggle more with understanding language itself than with inferring intent.
- nevertoolate 6mo agoit’s cold -> turn on the heater I’d never just turn on the heater silently if someone said this to me. I think it means something else.
- hackable_sand 6mo agoIf someone just said "it's cold" then yeah that's kinda toxic. If they said "turn on the heater" then you have no ambiguity
- quibono 6mo agoI know what you're getting at but those examples are reaching
- deleted 6mo ago