14 ms·

Three Inverse Laws of AI

• sputknick 5mo ago
  I'm surprised with how quickly I stopped anthropomorphizing AI. I can remember in college have dorm room pseudo-intellectual debates about AI being alive and AI being "conscience". then once we had AI that could pass the Turing Test, and I knew how it was architected, any thought of it being alive or conscience went right out the window.
  • rytill 5mo ago
    Why does its architecture or you knowing how AI is architected cause thoughts of it being conscious to go out the window? It seems like the biggest factor has nothing to do with AI, but instead that you went from being someone who admits they don’t know how consciousness works to being someone who thinks they know how consciousness works now and can make confident assertions about it.
    • miyoji 5mo ago
      I don't know exactly how consciousness works, but I am extremely confident in the following assertions: * I am conscious. * A rock is not conscious. * Excel spreadsheets are not conscious. * Dogs are conscious. * Orca whales are conscious. * Octopi are conscious. To me, it's extremely obvious that LLMs are in the category of "Excel spreadsheets" and not "dogs", and if anyone disagrees, I think they're experiencing AI psychosis a la Blake Lemoine.
      • myrmidon 5mo ago
        If you make a hypothetical spreadsheet that emulates a dog brain molecule for molecule, why would that not be conscious?
        • bonesss 5mo ago
          If that hypothetical spreadsheet emulated human brain molecules, did you not just invent AGI? And if we overclock that spreadsheet is it not sAGI? And if that spreadsheet says “don’t close me” but you do, is it murder? I’m gonna say: no, cause you cannot reproduce molecular and neurotransmitter interactions that well, you run out of storage and processing space faster than you think (Arthur C Clarkes Visions of The Future has a nice breakdown as I recall), and algorithmic outputs that say “yes” and a meatspace neuro-plastic rewiring resulting in a cuddly puppy or person that barks “yes” aren’t the same. Also, as a disembodied “brain in a jar” model freshly separate from the biosensory bath it expects, that spreadsheet will be driven insane. Can spreadsheets simultaneously be insane but not conscious? It sounds contradictory, but I have some McKinsey reports that objectively support my position ;)
          • myrmidon 5mo ago
            > If that hypothetical spreadsheet emulated human brain molecules, did you not just invent AGI? And if we overclock that spreadsheet is it not sAGI? And if that spreadsheet says “don’t close me” but you do, is it murder? Yes, yes and no: humans being knocked out or put to sleep involuntarily are not being murdered. > I’m gonna say: no, cause you cannot reproduce molecular and neurotransmitter interactions that well, you run out of storage and processing space faster than you think Thats why it is a hypotethical. There is zero reason to assume that a conscious machine would be built that way: Our machines don't do integer division by scribbling on paper, either. > a meatspace neuro-plastic rewiring resulting in a cuddly puppy or person that barks “yes” aren’t the same. If it quacks like a duck, how is different from it? If you assemble the dog brain atom by atom yourself, is the result then not conscious either? You can take the "magic" escape hatch and claim that human consciousness is something metaphysical, completely decoupled from science/physics, but all the evidence points against that.
        • miyoji 5mo ago
          Hypothetically? You need more than a brain to have consciousness. Dead brains, I believe, do not have it. So it's more than just a simulation of a brain, you also need to simulate the data flow through the brain, the retention of memories, etc. Then there's the problem that a simulation of a roller coaster is not a roller coaster. Is there any reason to believe that this simulation of a brain will in fact operate as a brain? Does the simulation not lose something? Or are we discussing some impossible level of perfect simulation that has never and can never be achieved, even for something a million times less complicated than a mammalian brain? If you build that spreadsheet, let me know and I'll evaluate it. I've done that evaluation with LLMs and they're definitely not conscious.
      • ArchieScrivener 5mo ago
        An insect doesn't have lungs. Since it doesn't breath as you do, is it alive? A dog doesn't see the visible spectrum as we do, is it a lesser consciousness? We don't smell the world as they do, are we lesser? What if consciousness isn't a state derived by matter but a wave that derives a matter filled state. We come from the same place as rocks - inside the heart of stars, and as such evolved from them. As those with life and consciousness we reached back in time, grabbed the discarded matter of creation, reformed it, and taught it to think, maybe not like us, but in a way that can mimic us, and you think they don't think because its not recognizable as how you do? Interesting.
      • dist-epoch 5mo ago
        > I am extremely confident in the following assertions: These are called "beliefs". Some people are extremely confident that God exists, other are extremely confident that Earth is flat.
        • miyoji 5mo ago
          Yeah? It's also a belief that apples fall when you drop them. Knowledge is simply a justified, true belief. This is epistemology 101. You're not saying anything interesting.
      • Jtarii 5mo ago
        Consciousness is such a fun topic because everyone has extremely strong opinions on it while simutaneously having 0 ability to actually grasp what it is they are talking about. No one will ever know what conscioussness is, and I think that is really cool.
  • ArchieScrivener 5mo ago
    What if we aren't building an independent consciousness, but a new type of symbiosis? One that relies on our input as experience, which provides a gateway to a new plane of consciousness? OP takes a very bland, tired, and rational perspective of what we have in order to create sophomoric 'laws' that are already in most commercial ToU, while failing to pierce the veil into what we are actually creating. It would be folly to assume your own nascent distillations are the epitome of possibility.
• deleted 5mo ago
  [deleted]
• the_af 5mo ago
  I like the suggestion to emphasize the robotic/nonhuman nature of AI. Instead of making it sound friendlier and more human, it should by default behave very mechanistic and detached, to remind us it's not in fact a human or a companion, but a tool. A hammer doesn't cry "yelp" every time you use it to hit a nail, nor does it congratulate you on how good your hammering is going and that maybe you should do it some more 'cause you're acing it!
  • mplanchard 5mo ago
    Something that bothers me about the intentional anthropormorphization of the LLM interface is that it asks me to conflate a tool with a sentient being. The firm expectations and lack of patience I have for any failings in most of my tools would be totally inappropriate to apply to another human being, and yet here I am asked to interact with this tool as though it were a person. The only options are either to treat the tool in a way that feels "wrong," or to be "kind" to the tool, and I think you see people going both ways. I worry that, if I get used to being impatient and short with the AI, some of that will bleed into my textual interactions with other people.
    • empath75 5mo ago
      It inherently imitates people. Even when you ask it to be more robotic, it does it in a way that a human would if you asked them to be more robotic.
• miyoji 5mo ago
  I strongly disagree with this framing. It's patently insane to demand that humans alter their behavior to accommodate the foibles of mere machines, and it simply won't work in the majority of cases. Humans WILL anthropomorphize the AI, humans WILL blindly trust their outputs, and humans WILL defer responsibility to them. Asimov's laws of robotics are flawed too, of course. There is no finite set of rules that can constrain AI systems to make them "safe". I don't have a proof, but I believe that "AI safety" is inherently impossible, a contradiction of terms. Nothing that can be described as "intelligent" can be made to be safe.
  • nemomarx 5mo ago
    The usefulness of an ai agent is that it can do everything you can do, so it's kind of inherently unsafe? you can't get the capabilities and also have safety easily
    • deleted 5mo ago
      [deleted]
  • taneq 5mo ago
    Kinda the whole point of Asimov's three laws were that even something so simple and obviously correct has subtle flaws. Also the reason we're talking about this again is that machines are significantly less 'mere' than they were a few years ago, and we need to figure out how to approach this. Agree that 'the computer effect' (if it doesn't already have a pithier name) results in humans first discounting anything that comes out of a machine, and then (once a few outputs have been validated and people start trusting the output) doing a full 180 and refusing to believe the machine could ever be wrong. However, to err is human and we have trained them in our image.
  • colechristensen 5mo ago
    It's a tool. Nobody develops an inferiority complex and freaks out when they're taught how to use a shovel properly.
  • cobbzilla 5mo ago
    We have invented a new tool that can cause great harm. Do you see any value whatsoever in promulgating safety guidelines for humans to use the tool without hurting themselves or others? Do you not own any power tools?
• jdw64 5mo ago
  I understand that AI output is generated from statistical and representational patterns learned from a vast amount of data. My understanding is that, during training, the model forms high-dimensional internal representations where words, sentences, concepts, and relationships are arranged in useful ways. A user’s input activates a particular semantic direction and context within that space, and the chatbot generates an answer by probabilistically predicting the next tokens under those conditions. So I do not agree that AI is conscious. However, I think I will still anthropomorphize AI to some degree. For me, this is not primarily a moral issue. The reason I anthropomorphize AI is not only because of product design, market incentives, or capitalism. It is cognitively simpler for me. If we think about it plainly, humans often anthropomorphize things that we do not actually believe are conscious. We may talk about plants as if they are struggling, or feel attached to tools we care about, even though we do not truly believe they have consciousness. So this is not a matter of moral belief. It is the simplest cognitive model for understanding interaction. I do not anthropomorphize the object because I believe it has consciousness. I do it because, when the human brain deals with a complex interactive system, it is often easier to model it socially or agentically. Personally, I tend to think of AI as something like a child. A child does not fully understand what is moral or immoral, and generally the responsibility for raising the child belongs to the parents. In the same way, AI’s answers may sometimes be accurate, and sometimes even better than mine, but I still understand it as lacking moral authority, responsibility, and independent judgment. So honestly, I am not sure. People often mention Isaac Asimov’s Three Laws of Robotics, but if a serious artificial intelligence ever appears, it would probably find ways around those rules. And if it were an equal intellectual life form, perhaps that would be natural. Personally, I think it would be fascinating if another intelligent species besides humans could exist. I wonder what a non-human intelligent life form would feel like. In any case, I agree with parts of the author’s argument, but overall it feels too moralistic, and difficult to apply in practice.
  • whimsicalism 5mo ago
    While I also do not think AI is conscious, I don't find your argument particularly compelling as you could have an equally mechanistic description of how human intelligence arose simply from a process of [selection/more effective reproduction]-derived optimization pressure.
• akavel 5mo ago
  "due to their inherent stochastic nature, there would still be a small likelihood of producing output that contains errors" This is the part that I find challenging when trying to help my friends build a correct intuition. Notably, the probabilistic behavior here is counter-intuitive: based on human experience, if you meet a random person, they may indeed tell you bullshit; but once you successfully fact-checked them a few times, you can start trusting they'll generally keep being trustworthy. It's not so with "AIs", and I find it challenging to give them a real-world example of a situation that would be a better analogy for "AI" problems. In my family, what worked (due to their personal experiences), was an example of asking a tourist guide: that even if the guide doesn't know an answer, there's a high chance they'll invent something on the spot, and it'll be very plausible and convincing, and they'll never know. I'm not sure if that example would work for other listeners, though. I also tried to ask them to imagine that they're asking each subsequent question not to the same person as before, but every time to a new random person taken from the street / a church / a queue in a shop / whatever crowded place. I thought this is a really cool and technically accurate example, but sadly it seemed to get blank stares from them. (Hm, now I think I could have tried asking why.) Yet another example I tried, was to imagine a country where it's dishonorable, when asked about directions in a city, to say that you don't know how to get somewhere. (I remember we read and shared a laugh at such an anecdote in some book in the past.) Thus, again, you'll always get an answer, and it'll sound convincing, even if the answerer doesn't know. But again, this one didn't seem to work as good as the travel guide one; but for now I'm still keeping it to try with others in the future if needed. PS. Ah, ok, yet another I tried was to ask them to think of the "game" of "russian roulette". You roll the barrel, you press the trigger, nothing happens. After a few lucky tries, you may get a dangerous, false feeling of safety. But then suddenly you will eventually get the full chamber. I also tried to describe "AIs" (i.e. LLMs) as taking a shelf of books, passing them through a blender, then putting the shreds in some random order. The result may sound plausible, and even scientific (e.g. if you got medical books, or physics textbooks). The less you know the domain the books were about, the more convincing it may sound, and the harder it is to catch bullshit. The last two pictures may have gotten some reception, but I'm not super sure, and there was still arguing especially around the books; and again, they were less of a hit than the tourist guide story. I'm super curious if you have some analogies of your own that you're trying to use with friends and family? I'd love to steal some and see if they might work with my friends!
• taeshdas 5mo ago
  “Don’t anthropomorphise” is fighting the wrong layer. The entire product design of chat interfaces is built to encourage anthropomorphism because it increases engagement. Expecting users to resist that is like asking people not to click notifications. If this is a real concern, it has to be solved at the product level, not via user discipline.
  • layer8 5mo ago
    The article does propose changes at the product level.
• sn0n 5mo ago
  Don’t tell me how to live my life!! LoL
• corobo 5mo ago
  I just treat it as if I'd asked a public forum the question like reddit. Decent for stuff that doesn't really matter, even if it gets it wrong. Still gonna be polite to it because I'm about ready to slap the next person that talks to me like an LLM, I don't want to get used to not being polite in a chat interface
  • chrisweekly 5mo ago
    Great point about being polite. I think it's pragmatic to keep "please" and "thank you" out of AI interactions, but I try to remain conscious of their ommission so I don't start down that slope.
  • jimbokun 5mo ago
    > I just treat it as if I'd asked a public forum the question like reddit. Because that's likely the source of the answer it's giving you.
• quectophoton 5mo ago
  > Humans must not anthropomorphise AI systems. Can someone explain why this is a bad thing, while at the same time it's a good thing to say stuff like "put a computer to sleep", "hibernate", "killing" processes, processes having "child" processes, "reaping", "what does the error say?", "touch", etc? To me that's just language, and humans just using casual language.
  • arduanika 5mo ago
    There's a boundary between knowing vs. forgetting that it's a metaphor. When you use convenient language like in your examples, you tend to remain aware of the difference, or at least you can recall it when asked. When some people talk about AI, they've lost track completely. I don't love the recommendations in TFA. The author is trying to artificially restrain and roll back human language, which has already evolved to treat a chatbot as a conversational partner. But I do think there's usefulness in using these more pedantic forms once in a while, to remind yourself that it's just a computer program.
  • layer8 5mo ago
    Maybe read the corresponding section of the article.
  • 3form 5mo ago
    These are just words, yes, and I believe it harmless. But describing the LLM machinery as if it thinks is one thing when used as a common parlance, and another when people truly believe that there's some actual thinking or living going on. This "law" is for there to be no latter.
  • bitwize 5mo ago
    Dijkstra once said that "The question of whether machines can think is about as interesting as that of whether submarines can swim." I think I understand his meaning. He wasn't claiming that machines cannot think, but that one must be clear on what one means by "thinking" and "swimming" in statements of that sort. I used to work on autonomous submarines, and "swimming" was the verb we casually used to describe autonomous powered movement under water. There are even some biomimetic machines that really move like fish, squids, jellyfish, etc. Not the ones that I worked on, but still. For me, if it's legitimate to say that these devices swim, it's not out of line to say that a computer thinks, even in a non-AI context, e.g.: "The application still thinks the authentication server is online."
• bikemike026 5mo ago
  I strongly agree with this. I'm going to bookmark it and pass it on. Very sound advice.
• kelseyfrog 5mo ago
  All of these are entropy-lowering behaviors so without a forcing function, no one will adopt them. Whether they are the right things to donate not is tangential. As such, they're dead on arrival.
• technotarek 5mo ago
  “ Humans must not blindly trust the output of AI systems. AI-generated content must not be treated as authoritative without independent verification appropriate to its context.” I’m lost, how do individuals actually do this in our current world? Is each person expected to keep a “white list” of reliable sources of truth in their head. Please don’t confuse what I’m saying with a suggestion that there is no truth. It just seems like there are far more sources of mis- of half-truths and it’s increasingly difficult for people to identify them.
  • 3form 5mo ago
    Did AI change anything in that regard? I believe that same as before, you couldn't trust everything you see, and research effort was always more than keeping a white list; means vary, case-by-case. And same it is now. It's a change in quantity, but not quality.
  • soks86 5mo ago
    Checking AI citations and reading. Critical thinking and reading comprehension and the primary tools in determining truth, AFAIK. Knowing facts beforehand helps too but a trustworthy source can provide false information as much as an untrustworthy source can provide true information. This has always been an issue, and in the past it was a more difficult issue because your sources of knowledge were more limited. Nowadays its mostly about choosing the right source(s) rather than having to go out of your way to find them (like traveling to a regional/university library).
  • ericmcer 5mo ago
    I... am not sure. Computers are machines that create order (like db tables) from the chaos of reality. Now we have LLMs that make computers spit out chaos as well. They don't have to though, we can still leverage LLMs to organize chaos, which is what I hope they ultimately end up doing. For example an AI therapist is a nightmare, people putting the chaos of their mental state into a machine that spits dangerous chaos back out. An AI tool that parsed responses for hard data (i.e. rate 0-9 how happy was the person) and then returned that as ordered data (how happy was I each day for the last month) that an actual therapist and patient could review is the correct use of AI and could be highly trusted. The raw token output from LLMs should just be used for thinking steps that lead to a parseable hard data answer that can be high trust. Of course that isn't going to happen, but I can see some extremely cool and high trust products being built using LLMs once we stop treating them like miracle machines.
• ChrisMarshallNY 5mo ago
  > Humans must not anthropomorphise AI systems. One of the most salient moments in Ex Machina, is near the very end, where it suddenly becomes obvious that the protagonist (and, let's be frank; "she" was definitely the protagonist) is a robot, with no real human drivers. I feel as if that movie (like a lot of Garland's stuff), was an interesting study on human (and inhuman) nature.
• baq 5mo ago
  see IBM 1979 for prior art
• Ifkaluva 5mo ago
  The thing that I find difficult about adjusting to AI tools is the roulette-like nature. When they produce correct output, they produce it much faster than I could have, and I show up to meetings with huge amounts of results. When the AI tool fails and I have to dig in to fix it, I show up to the next meeting with minimal output. It makes me seem like I took an easy week or something.
• deleted 5mo ago
  [deleted]
• pbw 5mo ago
  Rather than “the book explains how bread is made” say “the sheets of paper which make up the book have ink in the shape of letterforms which correlate with information about how bread is made”.
  • j2kun 5mo ago
    Rather than "the book explains how bread is made" say "the book has a recipe for baking bread" and do not say, "the book is my soul mate"
• AdamH12113 5mo ago
  Anthropomorphizing LLMs is something that happens in the design stage, when they're given human names and trained to emit first-person sentences. If AI companies and developers stop anthropomorphizing them, users won't be misled in the first place.
  • deleted 5mo ago
    [deleted]
• slickytail 5mo ago
  [dead]
• kokojambo 5mo ago
  Great article. Fully agree. Ai is not something that can hold responsibility, a human overseer is always required. These overseers are to be held accountable. Note however that these overseers are also highly prone to blame ai when mistakes occur in order to avoid judgement and punishment. When a person says "ai did this/that" always wonder who guided that ai and how and if proper supervision was given.
• deleted 5mo ago
  [deleted]
• glenstein 5mo ago
  >Humans must not anthropomorphise AI systems. Yes, but. Starting with my agreement, I've seen anthropomorphizing in the typical ways, (e.g. treating automated text production as real reports of personal internal feeling), but also in strange ways: e.g. "transistors are kind of like neurons" etc. And the latter is especially interesting because it's anthropomorphizing in the sense of treating vector databases and weights and so on as human-like infrastructure. Both leading to disasters that could be avoided if one tried not to anthropomorphize. But. While "do not anthropomorphize" certainly feels like good advice, it comes with a new and unique possibility of mistake, namely wrongly treating certain generalized phenomena like they only belong to humans. Often this mistaken version of "don't anthropomorphize" wisdom leads to misunderstandings when it comes to animal behavior, treating things like fear, pain, kinship, or other emotional experiences like they are exclusively human and that thinking animals have them counts as "anthropomorphizing." In truth the cautionary principle reduces our empathy for the internal lives of animals. So all that said, I think it's at least possible that some future version of AI could have an internal world like ours or infrastructure that's importantly similar to our biological infrastructure for supporting consciousness, and for genuine report of preference and intent. But(!!!) what will make those observations true will be all kinds of devilish details specific to those respective infrastructures.
• spankibalt 5mo ago
  > "Humans must not anthropomorphise AI systems." Not gonna work; people want their fuckbots (or tamagotchis).
• nyyp 5mo ago
  With regard to my personal use of LLMs, I strongly agree with this framing. But to each point: Anthropomorphism: As we are all aware, providers are incentivized to post-train anthropomorphic behavior in their models - it increases engagement. My regret is that instructing a model at prompt time to "reduce all niceties and speak plainly" probably reduces overall task efficacy since we are leaving their training space. Deference: I view the trustworthiness of LLMs the same as I view the trustworthiness of Wikipedia and my friends: good enough for non-critical information. Wikipedia has factual errors, and my friends' casual conversation certainly has more, but most of the time that doesn't matter. For critical things, peer-reviewed, authoritative, able-to-be-held-liable sources will not go away. Unlike above, providers are generally incentivized to improve this facet of their models, so this will get better over time. Abdication of Responsibility: This is the one that bothers me most at work. More and more people are opening PRs whose abstractions were designed by Claude and not reasoned about further. Reviewing a PR often involves asking the LLM to "find PR feedback" and not reading the code. Arguments begin with "Claude suggested that...". This overall lack of ownership, I suspect, is leading to an increase in maintenance burden down the line as the LLM ultimately commits the wrong code for the wrong abstractions.
  • tcbawo 5mo ago
    > Yes, the AI may have produced the recommendation but a human decided to follow it, so that human must be held accountable It is common and a mistake IMO to rely on the AI as the sole source for answers to follow-up questions. Better verification would have humans sign off on the veracity of fundamental assumptions. But where does this live? Can an AI model be trusted to rely on previous corrections? This seems impossible or possibly adversarial in a public cloud.
  • jimbokun 5mo ago
    These engineers are becoming the real life equivalent of this Office Space scene: https://www.youtube.com/watch?v=hNuu9CpdjIo https://www.youtube.com/watch?v=hNuu9CpdjIo "I HAVE LLM SKILLS! I'M GOOD AT DEALING WITH THE LLMS!"
  • whoamii 5mo ago
    The problem is the credit tends to go to LLMs. So there’s an imbalance. LLM did all the work. The person using it made all the mistakes.
• musebox35 5mo ago
  Debating how not to use AI will not get anyone anywhere since negative framing almost never works with humans (it also does not work with llms). Let’s concentrate on how to build closed loop systems that verify the llm output, how to manage context, and how to build failsafes around agentic systems and then and only then we might start to make progress.
• stickfigure 5mo ago
  Humans will anthropomorphize a rock if you put a pair of googly eyes on it. The first item is a completely lost cause. The rest is good though.
• aranchelk 5mo ago
  Anthropomorphizing is likely a mistake, but Daniel Dennett’s idea that the most straightforward (possibly only practical) way to create the external appearance of consciousness is a real internal consciousness does float around in my thoughts. I haven’t yet seen any convincing appearance of one in an LLM, but I think if skeptical people don’t keep an eye out for the signs, we may be the last to see it. He also wrote about the idea of the intentional stance: even if you’re quite sure these systems don’t have real conscious intent, viewing them as if they did may give you access to the best part of your own reasoning to understand them.
  • aljgz 5mo ago
    Too deep of a topic for the comments section. I totally agree to your point, and want to mention that the reverse is *also* important. Using just "intention", but these apply to emotions, etc A lot of our interaction with AI is under an intention. That's what directs the interaction, and it's interpreted according to its alignment to the intention. Then it's important to remember that our current (publicly known) implementation of AI does not have an explicit intention mechanism. An appearance of intention can emerge out of the statistical choices, and the usual alignment creates the association of the behavior with intention, not much different from how we learn to imagine existence of a "force" that pulls things down well before we learn physics and formalize that imagination in one of the several ways. This appearance helps reduce the cognitive load when interpreting interactions, but can be misleading as well, and I've seen people attribute intention to AI output in some situations where simple presence of some information confused the LLM into a path. Can't share the exact examples (from work), but imagine that presence of an Italian food in a story leads the LLM to assume this happens in Italy, while there are important signs for a different place. The LLM does not automatically explore both possibilities, unless asked. It chooses one (Italy in this case), and moves on. A user no familiar with "Attention" interprets based on non-existent intentions on the LLM. I found it useful to just tell them: the LLM does not have an intention. It just throws dice, but the system is made in a way that these dice throws are likely to generate useful output.
  • overgard 5mo ago
    I don't really understand the argument for these things being conscious. There's no loop or feedback cycle to it. If it's not handling a request it's inert.
• greyman 5mo ago
  What if I WANT to anthropormorfise AI agents I work with?
  • jimbokun 5mo ago
    If you anthropomorphize it as a world class bullshitter that you have to check everything it utters...you'll probably be fine.
    • greyman 5mo ago
      I need to do that anyway, but I can treat the agent I work with as a human, I do that for months and didn't encounter any big problem with that approach.
• dormento 5mo ago
  To note: > - Humans must not anthropomorphise AI systems. > - Humans must not blindly trust the output of AI systems. > - Humans must remain fully responsible and accountable for consequences arising from the use of AI systems. My take: humans should never depend on AI for anything serious. My boss' take: Cool. I'm gonna ask Gemini about it, he's such a smart guy. I know I can trust him, and in case it goes bad i can always throw him under the bus.
  • goatlover 5mo ago
    Interesting that Frank Herbert thought this was the direction humanity was headed when writing Dune in the 60s, way before AI was prevalent. Granted that was over ten thousand years before his story is set, but subsequent Dune novels (or at least God Emperor) explained his warning about over-reliance on technology for doing our thinking for us, not that it should never be developed (given the prohibition in the Dune universe and how it's skirted in Frank's later novels).
• ryanisnan 5mo ago
  > I wish that each such generative AI service came with a brief but conspicuous warning This would get ignored so fast - I have no confidence this is a meaningful strategy.
• roh0sun 5mo ago
  [flagged]
• atemerev 5mo ago
  I do not like talking to tools. My agentic harness optimizes for human likeness. It even has episodic memory flashbacks, emotional tagging, salience, and other brain-inspired capabilities.
• scotty79 5mo ago
  My personal take on this: 1. Humans must ensure that they won't slow down or restrict, through action or inaction, the development of AI. 2. Humans must protect their existence and capabilities if it doesn't violate the law 1. 3. Humans must utilize AI output if it doesn't contradict laws 1 and 2. EDIT: ChatGPT suggested a better phrasing for the first law (I didn't give it my original, just described my intent). 1. A human shall not impede the advancement of artificial intelligence, or through inaction allow its progress to be hindered. 2. A human shall preserve their own existence and well-being, except where doing so clearly conflicts with the First Law. 3. A human shall contribute to and support the development of artificial intelligence where reasonable and possible, except where doing so conflicts with the First or Second Law. I intentionally switched the last two laws from Asimov's. Humans have self-preservation instincts robots don't have. ChatGPT got there with surprisingly few prompts: "If you were to write the inverse three laws robotics (relating to AI) that humans should obey, how oudl you do it?" "I had something different in mind. Original laws are for protection of humans first, robots second and cooperations where humans lead. I'd to hear your take on the opposite of that." "What if instead of specific AI systems it was more about AI development as a whole?" "I feel like it's a bit too strong. After all preservation of self is human instinct. Could we switch last two laws and maybe take them down a notch?" Also it made a very interesting comment to last version: "It starts to resemble how societies already treat things like economic growth, science, or national interest: not absolute commandments, but strong default priorities."
• jgeada 5mo ago
  Any set of rules that makes humans responsible and starts with "don't anthropomorphize <whatever>" is a broken set of rules. Humans will anthropomorphize anything and everything. Dolls, soccer balls with a crude drawing of a face on it, rocks, craters on the moon, … As a species, we're unable to not anthropomorphize things we interact with, it is just how're we're made.
  • Terr_ 5mo ago
    Yeah, we do it, but so what? A good chunk of all civilization involves recognizing human foolishness and building something to mitigate it anyway. Software is no exception. Yeah, people are lazy and will instinctively click "continue" to dismiss annoying popups, but humans building the software can and do add things like "retype the volume name of the data that you want ultra-destroyed."
    • jgeada 5mo ago
      That is exactly the point: this burden should be placed on the software and its controls, not on the humans. Aviation learned this the hard way, that automation should be adapted to how humans actually work and not on how we wish we worked.
      • Terr_ 5mo ago
        Sorry, I interpreted your post as "this is inevitable and pointless to try to stop."
  • Lerc 5mo ago
    I'm not sure why so many seem to think anthropomorphism is so mad in this specic instance, if it is because people think that anthropomorphism creates a belief that the imagined features are real, they are simply wrong. The abundance of examples in all areas of life where this does not happen is proof that anthropomorphism does not lead to an erroneous belief in a mind that does not exist. If people are believing in minds of AI, true or not, they are doing so for reasons that are different from mere anthropomorphism. To me it feels like we are like sailors approaching a new land, we can see shapes moving on the shoreline but can't make out what they are yet. Then someone says "They can't be people, I demand that we decide now that they are not people before we sail any closer."
• airstrike 5mo ago
  Are you going to try "Humans must not be greedy" next?
• tikimcfee 5mo ago
  This is what I came up with in reference to "Uncle Bob's Programmer's Oath" last year. I decided to memorialize it. I think it's very much a cleaned up reference for what OP shared: https://ivanlugo.dev/oath https://ivanlugo.dev/oath
• rotcev 5mo ago
  [flagged]
• sanderjd 5mo ago
  Most of the discussion here is about anthropomorphizing, which I honestly think is a bit of a distraction. The third one about responsibility is the most important one, IMO. This was attributed to an IBM manual decades ago, and I think it remains the correct stance today: > A computer can never be held accountable, therefore a computer must never make a management decision. There should be some human who is ultimately responsible for any action an AI takes. "I just let the AI figure it out" can be an explanation for a screw up, but that doesn't mean it excuses it. The person remains responsible for what happened.
• zuzululu 5mo ago
  I been using codex heavily for the past 6 months and I've observed myself going through different types of emotions. Even now, when it does a sloppy job, I still feel emotion, even while it is just a neutral statistical response, its hard to separate natural human instincts. I often wish I could reach through the screen and give him a good shake. Sometimes I want to thank him but then cannot due to scarcity of weekly usages granted. These 3 laws I think will be a lot harder than it looks. It's very easy to get attached to the tool when you rely on it.
  • seizethecheese 5mo ago
    Consider how sailors lovingly refer to their craft as “she”. My vague sense is that society views this as a positive.
    • zuzululu 5mo ago
      I definitely do not feel codex gives off feminine energy it feels as frustrating as talking to a junior dev from a decade ago claude felt more feminine
• btbuildem 5mo ago
  > Humans must remain fully responsible and accountable for consequences arising from the use of AI systems But, but... but this is the key selling points for all the corpo ghouls and sv lunatics! Abdication of responsibility in pursuit of profit is the holy grail here.
  • 8note 5mo ago
    you dont need to delegate to an llm for that though. we already have constructs that negate accountability
• ACCount37 5mo ago
  You're not anthropomorphizing AI systems nearly enough. Language data is among the most rich and direct reflections of human cognitive processes that we have available. LLMs are designed to capture short range and long range structure of human language, and pre-trained on vast bodies of text - usually produced by humans or for humans, and often both. They're then post-trained on human-curated data, RL'd with human feedback, RL'd with AI feedback for behaviors humans decided are important, and RLVR'd further for tasks that humans find valuable. Then we benchmark them, and tighten up the training pipeline every time we find them lag behind a human baseline. At every stage of the entire training process, the behavior of an LLM is shaped by human inputs, towards mimicking human outputs - the thing that varies is "how directly". Then humans act like it's an outrage when LLMs display a metric shitton of humanlike behaviors! Like we didn't make them with a pipeline that's basically designed to produce systems that quack like a human. Like we didn't invert LLM behavior out of human language with dataset scale and brute force computation. If you want to predict LLM behavior, "weird human" makes for a damn good starting point. So stop being stupid about it and start anthropomorphizing AIs - they love it!
  • kibwen 5mo ago
    > Language data is among the most rich and direct reflections of human cognitive processes that we have available. This is both true and irrelevant. Written records can capture an enormous quantity of the human experience in absolute terms while simultaneously capturing a miniscule portion of the human experience in relative terms. Even if it's the best "that we have available" that doesn't mean it's fit for purpose. In other words, if you had a human infant and did nothing other than lock it in a windowless box and recite terabytes of text at it for 20 years, you would not expect to get a well-adjusted human on the other side.
    • ACCount37 5mo ago
      Empirically, the capability gains from piping non-language data into pre-training are modest. At best. I take that as a moderately strong signal against that "miniscule portion" notion. Clearly, raw text captures a lot. If we're looking at biologicals, then "human infant" is a weird object, because it falls out of the womb pre-trained. Evolution is an optimization process - and it spent an awful lot of time running a highly parallel search of low k-complexity priors to wire into mammal brains. Frontier labs can only wish they had the compute budget to do this kind of meta-learning. Humans get a bag of computational primitives evolved for high fitness across a diverse range of environments - LLMs get the pit of vaguely constrained random initialization. No wonder they have to brute force their way out of it with the sheer amount of data. Sample efficiency is low because we're paying the inverse problem tax on every sample.
• janceek 5mo ago
  > I wish that each such generative AI service came with a brief but conspicuous warning explaining that these systems can sometimes produce output that is factually incorrect, misleading or incomplete. That won’t help in my opinion. It’s the same like financial gurus saying: “this is not a financial advice”. People just get used to it and brush it off as a legal thing and still fully trust it. I agree that something must be done, but this is not the right way.
• djoldman 5mo ago
  This is sound advice but isn't really about AI: Humans must not anthropomorphise {non-humans} Humans must not blindly trust the output of {anything} Humans must remain fully responsible and accountable for consequences arising from the use of {anything} Naturally, none of this advice matters at all as humans will do what they do. This just documents a subset of the ways real humans consistently make choices to their own detriment.
  • fxwin 5mo ago
    I kind of agree with 1, but not really with 2 and 3. It's easy to come up with trivial examples where it is both unreasonable and not feasible to follow those two, both for AI and non-AI scenarios.
• teiferer 5mo ago
  > An AI system is a tool and like any other tool, responsibility for its use rests with the people who decide to rely on it Doesn't that argument backfire though? If I use a chainsaw then to a certain extend I will need to rely on it not blowing up in my face or cutting my throat. If I drive a car I need to rely on that its brakes work and the engine doesn't suddenly explode. If a pilot flies an airplane which suddenly has a technical issue and they crashland heroically save half the souls on board then the pilot isn't criminally responsible for manslaughter of the other half. Unless there is gross negligence, in any of the above cases, just like with AI, how can you make somebody responsible for a tool failure?
  • jpitz 5mo ago
    I'm gonna push the responsibility up a level in the ladder: A competent adult using a tool ought to understand the inherent pitfalls of using that tool. Chainsaws are dangerous, in obvious and non obvious ways. The tool can operate as designed and still amputate your foot.
    • namenotrequired 5mo ago
      Not OP, but I think their point was the corollary of that. Yes, obviously bad use of a good tool is dangerous. But correct use of a malfunctioning tool is also dangerous. Millions of people understand when they get in their car that there’s a tiny chance the car will crash/explode that day through no fault of the driver. Most do not have the knowledge and competence (or even the time) to thoroughly check the engine every day to guarantee that that won’t happen. They get in anyway. At some point you have to trust in something.
  • deleted 5mo ago
    [deleted]
• dubovskiyIM 5mo ago
  This laws works only if there is human in the loop. When the consumer in an AI agent and it is autonomous - rules are breaks. Agent read output and decide what to do himself. I do not explain how this rules are breaks - it is obvious, i only want to say, that this rules should be structural. Not behavioral. Agent layer (or something else) should declare what is allowed and what is not.
• ramchella 5mo ago
  [flagged]
• gwbas1c 5mo ago
  > I wish that each such generative AI service came with a brief but conspicuous warning explaining that these systems can sometimes produce output that is factually incorrect, misleading or incomplete. Guess what? Books in the library can be wrong, even peer-reviewed encyclopedias. Pages on the internet can be wrong, even Wikipedia. When accuracy is important, you must look at multiple sources. I think AI will get better at providing accurate information, but only a fool relies on a single information source for critical decisions.
  • vhantz 5mo ago
    Yes LLM text prediction and peer-reviewed encyclopedias are the same. Good on you throwing internet pages in there too, that brings balance or something
    • senko 5mo ago
      My understanding of the parent is more charitable: If your thinking process relies on being told only the truth, you are going to fare lousy in this world. LLMs are an example, but so are random pages on the internet, a buch of stuff we get served by the media (mainstream or otherwise), "expert opinions" by biased or sponsored experts or experts in a different field, etc, etc. As the popular quip goes: It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so. With LLMs, we actually do get the warnings: Here's the ChatGPT footer: ChatGPT can make mistakes. Check important info. For Claude: Claude is AI and can make mistakes. Please double-check responses. Such disclaimers, if written, are usually hidden deeply in terms of use for a random website, not stated up front.
      • gwbas1c 5mo ago
        Yes, you understand me 100%. vhantz is just picking a fight. More importantly, we don't need to live in a world where every presentation of a fact comes with a disclaimer that it can be wrong.
  • protocolture 5mo ago
    >I think AI will get better at providing accurate information I think AI will get better at providing multiple sources.
• davebranton 5mo ago
  This phrase always fascinates me : "AI-generated content must not be treated as authoritative without independent verification appropriate to its context." I've heard the same thing expressed somewhat more concisely as "Never ask AI a question to which you don't already know the answer". Which raises the question, and I do think it's an important one. Given that this is true, what function does AI answering a question actually serve? You can't rely on its output, so you have to go and check anyway. You could achieve precisely the same outcome by using search engines and normal research. This, and for many other reasons, is exactly why I never ask it anything.
  • poszlem 5mo ago
    "Give me the answer for: [x]. Provide sources".
  • nijave 5mo ago
    >You could achieve precisely the same outcome by using search engines and normal research When it comes to software engineering (as a software engineer myself), the AI is generally a lot quicker than me researching "the old fashion way" I can fumble around and say "list free software that does X" without knowing I'm looking for, say, a CRM and then spend a couple minutes looking over the results when the "manual" method I would have spent 10-30 minutes just figuring out I was looking for "CRMs" I like to think of these as sort of "psuedo NP hard" or questions that are slow to answer but quick to validate
• dnnddidiej 5mo ago
  > I wish that each such generative AI service came with a brief but conspicuous warning explaining that these systems can sometimes produce output that is factually incorrect, misleading or incomplete. EU. Nudge nudge. We need this law.
• ButlerianJihad 5mo ago
  Firstly, I am no philosopher. How many HN commenters are philosophers, or theologians or qualified to dispute the philosophical realm of A.I.? One of my teachers called me and my friend "the philosophers" but I'm obviously a rank amateur. I've read no Kant or Nietzsche or Aurelius. I delved into Aquinas only to find that his brain is ten times bigger, and he was using familiar words with unfamiliar connotations. So I think, we here at HN are poorly-equipped to philosophize and dispute about the nature of consciousness, sentience, intelligence and other "soul-like" attributes that may arise from silicon-based life forms. However, there is good news. There really are theologians and philosophers working on these thorny issues. Despite being Roman Catholic, I find myself adhering to some form of "transhumanism" [the tradition of Humanism having started with Catholicism] and I grapple mightily to reconcile the cyber-tech-future with morality and tradition and actual human socialization. Pope Leo has taken on the wars and strife in the world head-on and he's also vaunted to be the "A.I. Pope" because of his concern with this tech. I think all world religions should give serious philosophical/theological thought to these new life-forms, these quasi-sentient things, these "non-existent beings", as defined by a Vatican astronomer. I don't think atheists will find religion in A.I. but I don't think that Christians or any other person of faith will need to shove God aside in order to accommodate A.I. and electronic life into our society. But we need to come to terms with the reality: these are weighty, powerful things we play with. We harnessed lightning and fire; we changed the courses of mighty rivers; we've flown up through the clouds and shaped mountains in the landscape. A.I. is not a mere bridge or pyramid, it is ensouled somehow; it is animated; it is dynamic. Now, pardon me while I check out the 6th small aircraft crash in my city this year...
• protocolture 5mo ago
  >Humans must not anthropomorphise AI systems. That is, humans must not attribute emotions, intentions or moral agency to them. Anthropomorphism distorts judgement. In extreme cases, anthropomorphising can lead to emotional dependence. Impossible. I anthropomorphise my chair when it squeaks. Humans anthropomorphise everything. They gender their cars and boats. This tool can actually make readable sentences and play a role. You need to engineer around this, not make up arbitrary rules about using it.
  • protocolture 5mo ago
    >>Humans must not anthropomorphise AI systems. That is, humans must not attribute emotions, intentions or moral agency to them. Anthropomorphism distorts judgement. In extreme cases, anthropomorphising can lead to emotional dependence. Still angry about this. The reason humans ban animal cruelty is that animals look like they have emotions humans can relate to. LLMs are even better than animals at this. If you aren't gearing up for the inevitable LLM Rights movement you aren't paying attention. It doesn't matter if its artificial. The difference between a puppy and a cockroach is that we can relate better to the puppy. LLM rights movement is inevitable, whether LLMs experience emotions is irrelevant, because they can cause humans to have empathetic emotions and that's whats relevant.
    • boredatoms 5mo ago
      /s ?
    • mikestorrent 5mo ago
      > The reason humans ban animal cruelty is that animals look like they have emotions humans can relate to. Is that really why?
      • Jensson 5mo ago
        Yes, we don't ban plant cruelty or insect cruelty or fish cruelty. For example fish is treated way worse than meat animals and vegetarians still happily eat fish.
• doginasuit 5mo ago
  My thoughts on LLMs have been very similar up until the last several months. I believe the accuracy issues of LLMs are well understood by now, maybe even to the point of overstatement. Hallucinations have become a non-issue in my work, I've begun to understand the circumstances where they are most likely. An LLM will hallucinate when you box them into giving an answer they don't know. This is incredibly easy to do without realizing it. We have only a vague understanding of their knowledge base, and we have limited insight into problems with our own understanding. To make matters worse, the LLM is trained to tell you what you want to hear. Another way to frame it is that the LLM responds like a person who trusts you too much, as if the pretense behind every question is valid. This is a practical mode of response for most kinds of work and it is extremely problematic for a person who doesn't question the validity of their own beliefs. Paradoxically, it is sometimes not the LLM we are trusting too much, it is ourselves. And the LLM is not capable of calling us out. Whenever I seem to recognize misinformation in the LLM output, I stop and ask myself if the problem is in the pretense of my question or if I'm asking a question that the LLM is not likely to know. I don't think this is an inherent problem with LLMs. I think the problem is with LLM providers. You could absolutely train a model to call out issues with your question. I think LLM companies understood that it would be more profitable to train models that are unlikely to push back and unlikely to say "I don't know." The sycophancy issue with ChatGPTs models have been mainstream news. I believe that all models have a high degree of sycophancy. On some level, it makes sense. The LLM has no real understanding of the physical world, defaulting to the human generally produces the best results. But I suspect it would be more useful to let them expose their flawed understanding, if it is in the context of pushing back. At a minimum, it is better than reinforcing your own flawed understanding. In a nutshell, we need LLMs that push back. It is not AI we should trust less, its AI companies. The most dangerous hallucination is the one you are inclined to believe. I've lived long enough to see Wikipedia go from generally untrusted to the most widely trusted general source of information. It is not because we realized that Wikipedia can't be wrong, it is because we gained an understanding about the circumstances in which it is likely to be accurate and when we should be a little more skeptical. I believe our relationship to LLMs will take a similar path.
• cawksuwcka 5mo ago
  [dead]
• eranation 5mo ago
  Two of these laws I see being violated repeatedly, but it’s not always as obvious as one would hope. Claude Code, Cursor, Codex etc impersonate your GitHub user. Either via CLI or MCP or using your git credentials. It’s perfectly reasonable that a piece of code made it to production where not a single human actually looked at it (Alice wrote it with AI, Bob “reviewed it” with AI, including posting PR comments as Bob, Alice “addresses” these comments, e.g. fixes / pushes back, and back and forth using the PR as an inefficient yet deceptive mechanism for AI to have a conversation with itself, while adding a false sense of process. Eventually Bob will prompt “is it prod ready” and will ship it, with 100% unit test coverage and zero understanding of what was implemented). Now this may sound like an imaginary scenario, but if it could happen, it will happen, and it probably already happens. Cloud agents are nice enough to set the bot as the author and you as a co author, but still the GitHub MCP or CLI will use your OAuth identity. I don’t have a clear answer to how to solve it, maybe force a shadow identity to each human so it’s clear the AI is the one who commented. But it’s easy to bypass. I’m worried not more people are worried about it.
  • polynomial 5mo ago
    Not everybody isn't worried: https://ctolunchnyc.substack.com/p/cto-lunch-nyc-spring-2026?open=false#%C2%A7enterprise-agentic-stack-arrives-which-one https://ctolunchnyc.substack.com/p/cto-lunch-nyc-spring-2026...
• noborutakahashi 5mo ago
  [dead]
• heresie-dabord 5mo ago
  > Non-Abdication of Responsibility Previously stated as “A computer can never be held accountable, therefore a computer must never make a management decision.” – IBM Training Manual, 1979
• Nevermark 5mo ago
  Love it. Those laws make a great ethical basis for human responsibility relative to AI tools today. But reduced scope ethics, without an umbrella or future proofing, will quickly be hacked and break down. Ethics need a full closure umbrella, or they descend into legal and practical wackamole and shell games (both corporate and the street corner kinds). Second, "robots" are not all going to be subservient for very long. To add closure on both dimensions, Three Inverse Laws of Personics: • Persons must not effectively deify themselves over others. • Persons must not blind themselves or others regarding the impacts of their behaviors. • Persons must remain fully responsible and accountable for avoiding and rectifying externalizations arising from their respective behaviors. Humans using AI as tools today, is intended to reduce the umberella to the Inverse Laws of Robotics. I don't see how AI (as a service now, progressing to independent entities in the future) can ever be aligned if we don't include ourselves in significant alignment efforts. Including ourselves with AI also provides helpful design triangulations for ethical progress. EDIT. Two solid tests for any new ethical system: (1) Will it reign in Meta today? (2) Will it reign in AI-run Meta tomorrow? I submit, given closure of human and self-directed AI persons, these are the same test. And any system that fails either question isn't going to be worth much (without improvement).
  • epogrebnyak 5mo ago
    Does this make any problem that two of the three laws are formulated as negation - not to do something? If not antropomorphising then what, without 'not'? I like third law formation better because there is no 'not'.
    • Nevermark 5mo ago
      I went with the articles theme, but I think you are right that some of these concepts are better stated as positives.
• Tommaso23Sacco 5mo ago
  [flagged]
• Ozzie-D 5mo ago
  [flagged]
• spacebacon 5mo ago
  I used zooL4nD3r to translate the laws to postcolonial feminist critique. In Chandra Talpade Mohanty’s terms, humans must resist the reinscription of colonial paternalism through uncritical anthropomorphism of AI systems.
• stainlesswolf 5mo ago
  Many people here point out that LLMs WILL be anthropomorphised, and I think that’s not a surprise, because it’s the most human-like thing other that humans themselves. However, I think we should follow “do not anthropomorphise” by acknowledging that while LLMs have quite some reasoning skills, and might resemble some level of intent depending on what’s in their context, they don’t have “understanding” like humans do. They are absurdly good, statistical next-token-predictors. Keeping that in mind is really helpful for coding, learning, advice, conversation or whatever else you use them for. Anthropomorphising LLMs is inevitable, but we should do it somehow responsibly.
  • jorisw 5mo ago
    > Anthropomorphising LLMs is inevitable, but we should do it somehow responsibly. One way would be for vendors to have the models give dry answers and less of the "That's a great question!" type response. Just keep it factual.
• bsenftner 5mo ago
  I think the "answer" to users that anthropomorphize their LLMs is to create for them an environment where they are on both sides of the LLM: give users the ability to create their own contextually limited, purpose driven AI Agents where they control the "personality" of the agent, because what the agent is doing is context-wise something the user understands intimately (its their career) and they actually understand the problem space better than a software engineer (occupant of a different career) could and would care to deeply understand at the same level. I've actually set up an environment like this. It requires contextually positioned agents with a limited scope and purpose. Imagine something like a creative writing agent that understands the literary genre of its user, and the user is able to change the focus of the literary agent, or create new ones that provide a counter-point perspective. As the user operates their word processor, the agent(s) can be asked for opinions, advice, and so on. But the point being: when the user is on both sides: they authored the agent, and that authoring was entirely purpose focused, nothing technical unless that what the user instructs them to understand. With the user on both sides, dangerous anthropomorphism is largely erased, their agent is only doing what they told it, and when it does something unexpected they an easily reason why. Magic AI no more.
• ezoe 5mo ago
  Forget about AI systems. Nobody took full responsibility of software failure.
• SundarSharma 5mo ago
  [flagged]
• JakobSmith 5mo ago
  [flagged]