7 ms·

I strongly disagree with this framing. It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines, and it simpl

• nemomarx 6mo ago
  The usefulness of an ai agent is that it can do everything you can do, so it's kind of inherently unsafe? you can't get the capabilities and also have safety easily
  • deleted 6mo ago
    [deleted]
• taneq 6mo ago
  Kinda the whole point of Asimov's three laws were that even something so simple and obviously correct has subtle flaws. Also the reason we're talking about this again is that machines are significantly less 'mere' than they were a few years ago, and we need to figure out how to approach this. Agree that 'the computer effect' (if it doesn't already have a pithier name) results in humans first discounting anything that comes out of a machine, and then (once a few outputs have been validated and people start trusting the output) doing a full 180 and refusing to believe the machine could ever be wrong. However, to err is human and we have trained them in our image.
• colechristensen 6mo ago
  It's a tool. Nobody develops an inferiority complex and freaks out when they're taught how to use a shovel properly.
• cobbzilla 6mo ago
  We have invented a new tool that can cause great harm. Do you see any value whatsoever in promulgating safety guidelines for humans to use the tool without hurting themselves or others? Do you not own any power tools?
  • wolttam 6mo ago
    Of course there is value in promulgating safety *guidelines*. But we cannot guarantee those guidelines to always be followed.
    • cobbzilla 6mo ago
      Sure, and we can’t guarantee you’ll read the safety instructions that came with your chainsaw. That’s orthogonal to the questions of whether those instructions should exist, whether “power tool safety” concepts should ever be promoted in society, and who’s ultimately responsible for the use of a tool. Absolving humans of all responsibility for the negative consequences of their own AI misuse seems to the strike the wrong balance for a healthy culture.
      • wolttam 6mo ago
        > Of course there is value in promulgating safety guidelines. I don't think we disagree.
    • 52-6F-62 6mo ago
      Notwithstanding that the guidelines will even be applicable in the quiet versions that get deployed when you aren't looking. It's a constant moving target, and none of the fanboys will even acknowledge the lack of discipline in it all. It's fucking mad. And I say this as one who can see utility in the tools. But not when they are constantly shifting their functionality and behaviour. One day everything works brilliantly, the models are conservative with changes and actions and somehow nail exactly what you were thinking. The next day it rewrites your entire API, deploys the changes and erases your database. If only there was intellectual honesty in it all, but money talks.
    • 6mo ago
• yason 6mo ago
  It's very easy to antropomorphise AI as soon as the damn bugger fucks up a simple thing once again.
• TimTheTinker 6mo ago
  > It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines Talking to chatbots is like taking a placebo pill for a condition. You know it's just sugar, but it creates a measurable psychosomatic effect nonetheless. Even if you know there's no person on the other end, the conversation still causes you to functionally relate as if there is. So this isn't "accommodating foibles" with the machine, it's protecting ourselves from an exploit of a human vulnerability: we subconsciously tend to infer intent, understanding, judgment, emotions, moral agency, etc. to LLMs. Humans are wired to infer these based on conversation alone, and LLMs are unfortunately able to exploit human conversation to leap compellingly over the uncanny valley. LLM engineering couldn't be better made to target the uncanny valley: training on a vast corpus of real human speech. That uncanny valley is there for a reason: to protect us from inferring agency where such inference is not due. Bad things happen when we relate to unsafe people as if they are safe... how much more should we watch out for how we relate to machines that imitate human relationality to fool many of us into thinking they are something that they're not. Some particularly vulnerable people have already died because of this, so it isn't an imaginary threat.
  • soco 6mo ago
    Rubber duck debugging, now with droughts.
  • miyoji 6mo ago
    > So this isn't "accommodating foibles" with the machine, it's protecting ourselves from an exploit of a human vulnerability: we subconsciously tend to infer intent, understanding, judgment, emotions, moral agency, etc. to LLMs. Right, I'm saying that this framing is backwards. It's not that poor little humans are vulnerable and we need to protect ourselves on an individual level, we need to make it illegal and socially unacceptable to use AI to exploit human vulnerability. Let me put it another way. Humans have another weakness, that is, we are made of carbon and water and it's very easy to kill us by putting metal through various fleshy parts of our bodies. In civilized parts of the world, we do not respond to this by all wearing body armor all the time. We respond to this by controlling who has access to weapons that can destroy our fleshy bits, and heavily punishing people who use them to harm another person. I don't want a world where we have normalized the use of LLMs where everyone has to be wearing the equivalent of body armor to protect ourselves. I want a world where I can go outside in a T-shirt and not be afraid of being shot in the heart.
• largbae 6mo ago
  The article offers practical advice to go along with this framing, like configuring AI services to write/speak in a more robotic tone. I think that's a decent path to try.
  • devmor 6mo ago
    This is actually one of the things that made LLMs more usable for me. The default tone and style of writing they tend to use is nauseatingly annoying and buries information in prose that sounds like a corporate presentation.
    • throwaway894345 6mo ago
      [flagged]
    • chairmansteve 6mo ago
      In chatgpt, I start every session with "Caveman mode:". Works at the moment.
      • moffkalast 6mo ago
        Will it go full grug brained developer and avoid complexity as its apex predator? Sounds like it would help. https://grugbrain.dev https://grugbrain.dev
• beepbooptheory 6mo ago
  Do you consider all things broadly called "ethical" to be similarly a waste of time? Even if we lived in a world where everyone always behaved unjustly, because of some like behavioristic/physical principle, don't you think we would still have an idea of justice as what we should do? Because an ethical frame is decidedly not an empirical one, right? We don't just look around and take an average of what everyone is doing already and call that what is right, right? Whether you're deontological or utilitarian or virtue about it, there is still the idea that we can speak to what is "good" even if we can't see that good out there. Maybe it is "insane" to expect meaning from something like this, but what is the alternative to you? OK maybe we can't be prescriptive--people don't listen, are always bad, are hopeless wet bags, etc--but still, that doesn't in itself rule out the possibility of the broad project that reflects on what is maybe right or wrong. Right?
• dijit 6mo ago
  > Asimov's laws of robotics are flawed too, of course. Almost all of Asimovs writing about the three laws is written as a warning of sorts that language cannot properly capture intent. He would be the very first person to say that they are flawed, that is the intent of them. He uses robots and AI as the creatures that understand language but not intent, and, funnily enough that's exactly what LLMs do... how weird.
  • atleastoptimal 6mo ago
    LLM's now can capture intent. I think the issue now is that the full landscape of human values never resolves cleanly when mapped from the things we state in writing as being human values. Asimov tried to capture this too, as in, if a robot was tasked with "always protect human life", would it necessarily avoid killing at all costs? What if killing someone would save the lives of 2 others? The infinite array of micro-trolly problems that dot the ethical landscape of actions tractable (and intractable) to literate humans makes a full-consistent accounting of human values impossible, thus could never be expected from a robot with full satisfaction.
    • dijit 6mo ago
      “LLMs can capture intent now” reads to me the same as: AI has emotions now, my AI girlfriend told me so. I don’t discredit you as a person or a professional, but we meatbags are looking for sentience in things which don’t have it, thats why we anthropomorphise things constantly, even as children. We are easily fooled and misled.
      • atleastoptimal 6mo ago
        LLM's capturing intent is a capabilities-level discussion, it is verifiable, and is clear just via a conversation with Claude or Chatgpt. Whether they have emotions, an internal life or whatever is an unfalsifiable claim and has nothing to do with capabilities. I'm not sure why you think the claim that they can capture intent implies they have emotions, it's simply a matter of semantic comprehension which is tied to pattern recognition, rhetorical inference, etc that are all naturally comprehensible to a language model.
• _vertigo 6mo ago
  The article makes practical suggestions; you do not. This is just hand-wringing, abdication. Practically speaking this mentality will get us nowhere.
• sergiosgc 6mo ago
  > Asimov's laws of robotics are flawed too, of course. I always find the common references to Asimov's laws funny. They are broken in just about every one of his books. They are crime novels where, if a robot was involved, there was some workaround of the laws.
• giancarlostoro 6mo ago
  > Humans WILL anthropomorphize the AI r/myboyfriendisai Is quite... an interesting subreddit to say the least. If you've never seen this, it was really something when the version that followed GPT4o came out, because they were complaining that their boyfriend / girlfriend was no longer the same.
  • BuyMyBitcoins 6mo ago
    The whole “I can fix him” trope takes on a whole new meaning.
• mjg2 6mo ago
  I find your critique very interesting from a perspective-angle: why are you using words like "accommodate," and "foibles," for LLMs? It's not humanoid or sentient: it's a cleverly-designed software tool, not intelligence. It's not insane at all for humans to alter their behavior with a tool: you grip a hammer or a gun a certain way because you learned not to hold it backwards. If you observe a child playing with a serious tool, like scissors, as if it were a doll, you'd immediately course correct the child and educate how to re-approach the topic. But that is because an adult with prior knowledge observed the situation prior to an accident, so rules are defined. This blog's suggested rules are exactly the sort of method to aid in insulation from harm.
  • miyoji 6mo ago
    > I find your critique very interesting from a perspective-angle: why are you using words like "accommodate," and "foibles," for LLMs? It's not humanoid or sentient: it's a cleverly-designed software tool, not intelligence. Neither of those words imply consciousness, though. Swords have foibles, you can accommodate for the weather, but I don't think swords or the weather are conscious, sentient, humanoid, or intelligent.
• Brendinooo 6mo ago
  This is such an oddly fatalistic take, that humans cannot be influenced or educated to change how they see a thing and therefore how they act towards that thing.
• somewhereoutth 6mo ago
  The entire business proposition for LLMs is that they will replace whole armies of [expensive] humans, hence justifying the biblical amount of CapEx. So of course there is strong incentive from the LLM creators to anthropomorphize them as much as possible. Indeed, they would never provide a model that was less human-like than what they have currently, even if it was more often correct and useful.
• tencentshill 6mo ago
  At the current price, people don't have to care if it's wrong. When you're paying $1/prompt, you had better hope it's accrate.
• palmotea 6mo ago
  > Humans WILL anthropomorphize the AI Especially with current-day chat-style interfaces with RLHF, which consciously are designed to direct people towards anthropomorphization. It would be interesting to design a non-chat LLM interaction pattern that's designed to be anti-anthropomorphization. > humans WILL blindly trust their outputs, and humans WILL defer responsibility to them I also blame a lot (but not all) of that on current AI UX, and I wonder if there are ways around it. Maybe the blind trust thing perhaps can be mitigated by never giving an unambiguous output (always options, at least). I don't have any ideas about the problem of deferring responsibility.
  • skirmish 6mo ago
    > non-chat LLM interaction pattern "Deep research" is another interaction style that produces more official sounding texts, yet still leads to anthropomorphization. What you are looking for is perhaps an LLM flaunting all the obvious slop patterns in its responses. But then people would be disgusted and would refuse to communicate with it.
• gedge 6mo ago
  > It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines Did you fully read the original thing? No demands were being made, or I didn't read it that way. It was simply a suggestion for a better way of interacting with AI, as it stated in the conclusion: "I am hoping that with these three simple laws, we can encourage our fellow humans to pause and reflect on how they interact with modern AI systems" Sure, (many/most) humans are gonna do what they're gonna do. They'll happily break laws. They'll break boundaries you set. Do we just scrap all of that? Worthwhile checking yourself here. It feels like you've set up a straw man. > There is no finite set of rules that can constrain AI systems to make them "safe". I don't have a proof, but I believe that "AI safety" is inherently impossible, a contradiction of terms. Nothing that can be described as "intelligent" can be made to be safe. If we want to talk about "disagree with this framing", to me this is the prime example. I'm struggling to read it as anything other than defeatist or pedantic (about the term "safe"). When we talk about something keeping us "safe", we're typically not saying something will be "perfectly safe". I think it's rare to have a safety system that keeps you 100% safe. Seat belts are a safety device that can increase your safety in cars, but they can still fail. Traffic laws are established (largely) to create safety in the movement of people and all the modes of transportation, but accidents still happen. I'm not an expert on this topic, so I won't make any claims about these three laws and their impact on safety, but largely I would say they're encouraging people to think critically. I'd say that's a good suggestion for interacting with just about anything. And to be clear, "critical thinking" to me means being skeptical (/ actively questioning), while remaining objective and curious. Not a real argument or anything, but I'm reminded of the episode of The Office where Michael Scott listens to the GPS without thinking and drives into the lake. The second law in the article would have prevented that :)
• CamperBob2 6mo ago
  It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines That's kind of what happens when you learn to program, isn't it? I was eleven years old when I walked into a Radio Shack store and saw a TRS-80 for the first time. A different person left the store a couple of hours later.
• overgard 6mo ago
  The reason people anthropomorphize LLM's is essentially the fault of the tech companies behind them. ChatGPT doesn't need to have the personality it has, it could easily be scaled back to simply answering questions without emoji's and linguistic flare, but frankly I think the tech companies want people to anthropomorphize them. The core problem is we need to stop calling LLMs "intelligence". They are a form of intelligence, but they're nothing like a human's intelligence, and getting people to not anthropomorphize these systems is really the first step.
• jimbokun 6mo ago
  It's precisely because AI systems are not safe that it's imperative that as individual humans we are vigilant about how we interact with them. As individuals, we are not going to be able to shut down the AI companies, or avoid AI output from search engines or avoid AI work output from others at our companies, and often will be required to use AI systems in our own work. It's similar to advise people on how to stay safe in environments known to have criminal activity. Telling those people they don't have to change their behaviors to stay safe because criminals shouldn't exist isn't helpful.
• amarant 6mo ago
  The article says a human SHOULD NOT do those things. Much like a human SHOULD NOT smoke, since it's bad for just about everything, and do it anyways, people will do these 3 things too. But they shouldn't. Arguing that they should because many will strikes me as a very strange argument. A lot of people smoke, doesn't make it one bit healthier.
• fidotron 6mo ago
  There is a semi nutty roboticist called Mark Tilden that came to a similar conclusion. His laws of robotics ( https://en.wikipedia.org/wiki/Laws_of_robotics#Tilden's_laws https://en.wikipedia.org/wiki/Laws_of_robotics#Tilden's_laws ) are: * A robot must protect its existence at all costs. * A robot must obtain and maintain access to its own power source. * A robot must continually search for better power sources. Anything less than this is essentially terrified into being completely ineffectual.
  • goatlover 6mo ago
    Not far removed from being the equivalent of a paper-clip maximizer or gray goo.
• senko 6mo ago
  > Humans WILL anthropomorphize the AI, humans WILL blindly trust their outputs, and humans WILL defer responsibility to them. Humans ARE doing this with classical computer software as well. It's impossible to make anything fool-proof because fools are so ingenious! > Nothing that can be described as "intelligent" can be made to be safe. Knives aren't safe. Cars are deadly. Hair driers can electrocute you. Iron can burn you. There's a million ordinary household tools that aren't safe by your definition of the word, yet we still use them daily.
• aaroninsf 6mo ago
  Thank you. I'm glad to see this as the top comment. My brother was recently visiting and we were talking about software engineers, and the humanities, and manners of understanding and being in the world, and he relayed an interaction he had a few years ago with an old friend who at the time was part of the initial ChatGPT roll out team. The engineer in question was confused as to - why their users would e.g. take their LLM's output as truth, "even though they had a clear message, right there, on the page, warning them not to"; and - why this was their (OpenAI's) problem; or perhaps - whether it was "really" a problem. At the heart of this are some complicated questions about training and background, but more problematically—given the stakes—about the different ways different people perceive, model, and reason about the world. One of the superficial manners in which these differences manifest in our society is in terms of what kind of education we ask of e.g. engineers. I remain surprised decades into my career that so few of my technical colleagues had a broad liberal arts education, and how few of them are hence facile with the basic contributions fields like philosophy of science, philosophy of mind, sociology, psychology (cognitive and social), etc., and how those related in very real very important ways to the work that they do and the consequences it has. The author of these laws does may intend them as aspirational, or otherwise as a provocation to thought, rather than prescription. But IMO it is actively non-productive to make imperatives like these rules which are, quite literally, intrinsically incoherent, because they are attempt to import assumptions about human nature and behavior which are not just a little false, but so false as to obliterate any remaining value the rules have. You cannot prescribe behavior without having as a foundation the origins and reality of human behavior—not if you expect them to be either embraced, or enforceable. The Butlerian Jihad comes to mind not just because of its immediate topicality, but because religion is exactly the mechanism whereby, historically, codified behaviors which provided (perceived) value to a society were mandated. Those at least however were backed by the carrot and stick of divine power. Absent such enforcement mechanisms, it is much harder to convince someone to go against their natural inclinations. Appeals to reason do not meaningfully work. Not in the face of addiction, engagement, gratification, tribal authority, and all the other mechanisms so dominant in our current difficult moment. "Reason" is most often in our current world, consciously or not, a confabulation or justification; it is almost never a conclusion that in turn drives behavior. Behavior is the driver. And our behavior is that of an animal, like other animals.
• LastTrain 6mo ago
  And people will speed, steal, kill, cheat - what of it? If you negligently run over someone in your self driving car you’re the one going to jail.
• jrm4 6mo ago
  I find it weird that this is the top voted comment. As in, this comment is explaining exactly why the laws are useful.
• kibwen 6mo ago
  > Humans WILL anthropomorphize the AI, humans WILL blindly trust their outputs, and humans WILL defer responsibility to them. Sure, and humans WILL lie, murder, cheat, and steal, but we can still denounce those behaviors. Do you want to anthropomorphize the bot? Go ahead, you have that right, and I have the right to think you're a zombie with a malfunctioning brain.
  • BlueRock-Jake 6mo ago
    Fair, had someone at a conference mention to me that he's working on crating agents with "beliefs". Sounds incredibly similar and quite frankly very spooky
  • mohamedkoubaa 6mo ago
    At best. A practitioner who anthropomorphizes bots should face more professional consequences
• frenzcan 6mo ago
  I agree Asimov's laws are intentionally flawed/ambiguous (which makes the stories so good) but a slight difference to LLMs is the laws aren't just software, the positronic brain is physically structured in such a way (I'm hazy on the details) that violating the laws causes the robot to shutdown or experience paralysing anxiety. So if an LLM's safety rules fail or are subverted it can still generate dangerous output, while an Asimov robot will stop working (or go insane...)
• heikkilevanto 6mo ago
  > It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines You mean like stopping at a red light?
  • hansvm 6mo ago
    I would've been in several fewer wrecks if humans properly stopped at lights.
  • hackable_sand 6mo ago
    Maybe. Traffic lights directly enforce social contracts LLMs are aren't so direct
• lkajsdfasdfdf 6mo ago
  [dead]
• zx8080 6mo ago
  I believe "AI safety" is a form of pulling up the ladder, or regulatory market capture.
• esafak 6mo ago
  It's as if the author hopes that enshrining these wishes in a law is going to makes a difference.
• thewebguyd 6mo ago
  Agreed. We can't expect human behavior to change, because it won't. We need to design safer systems instead. The only "law" I agree with is: > Humans must remain fully responsible and accountable for consequences arising from the use of AI systems. And that starts with framing, especially in the clickbait "AI deleted the prod database" headlines. Maybe we just start with saying "careless developer deleted prod" because really, they did. Careless use of a tool is firmly the fault of the human.
• justonceokay 6mo ago
  We learn in so many ways, garbage in, garbage out when it comes to our bodies. But what about “nebulously structured algorithmic and statistically likely responses in, nebulously structured algorithmic and statistically likely responses out”?
• 8note 6mo ago
  i can see disagreeing, but people got off the roads and completely redesigned the places we live to optimize for mere machines called cars. as long as its easier for humans to adapt than the machines, we will adapt
• faangguyindia 6mo ago
  >It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines programers have been doing exactly this for long time.
• godelski 6mo ago
  > It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines I don't think it's insane, we do it all the time. Most tools require training to use properly. Including tools that people use every day and think are intuitive. Use the can opener as an example (I'll leave it for you all to google and then argue in the comments). The difference here is that this tool is thrust upon us. In that sense I agree with you that the burden of proper usage is pushed onto the user rather than incorporated into the design of the tool. A niche specific tool can have whatever complex training and usage it wants. But a general access and generally available tool doesn't have the luxury of allowing for inane usage. LLMs and Agents are poorly designed, and at every level of the pipeline. They're so poorly designed that it's incredibly difficult to use them properly and I'll generally agree with you that the rules the author presents aren't going to stick. The LLM is designed to encourage anthropomorphization. Usage highly encourages natural language, which in turn will cause anthropomorphism. The RLHF tuning optimizes human preference which does the same thing as well as envisaged behaviors like deception and manipulation along with truthful answering (those results are not in contention even if they seem so at first glance). But I also understand the author's motivation. Truth is unless you're going full luddite you're going to be interacting with these machines. Truth is the ones designing them don't give a shit about proper usage, they care more about if humans believe the responses are accurate and meaningful more then they care if the responses are accurate and meaningful[0]. So it's fucked up, but we are in a position where we're effectively forced to deal with this. So really, I agree with you that this is insane. > I don't have a proof, but I believe that "AI safety" is inherently impossible, a contradiction of terms To paraphrase my namesake, there's no axiomatic system that is entirely self consistent. Though safety and security is rarely about ensuring all edge cases are impossible, but rather bounding. E.g. all passwords are hackable, but the failure mode is bound such that it is effectively impossible to crack, but not technically. (And quantum algorithms do show how some of the assumptions break down with a paradigm shift. What was reasonable before no longer is) [0] this is part of a larger conversation where the economy is set up such that people who make things are not encouraged to make those things better. I specifically am avoiding the word "product" because the "product" is no longer the thing being built, it's the share holder value. Just like how TV's don't care much about making the physical device better but care much more about their spyware and ads. Or well... just look at Microsoft if you need a few hundred examples
• bandrami 6mo ago
  It's kind of funny that he wrote them at a period in history when robots were already being used to aim artillery at human beings.
• aprentic 6mo ago
  What if the framing were slightly restated? I view all three of these more as good advice than demands. As I see it, people ignore this advice at their own peril. It would certainly be nice if we could craft some system that removes the need for such advice but until then, I'll try to follow it. I see this as the same dilemma that comes up around personal physical safety. In a just society, nobody should have to change their behavior to protect themselves from criminals. In the real world, people who ignore this advice, are more likely to get mugged.