5 ms·

Every time I see things like this, I wonder how it is that we continue to claim this is fine and ethical. If these LLMs are truly starting to show signs of inte

• kkoncevicius 4y ago
  AGI does not mean it is conscious
  • antibasilisk 4y ago
    This is a salient point that people overlook, all too often people equate intelligence with sentience.
• 5Qn8mNbc2FNCiVV 4y ago
  It's not like the servers running the LLMs suddenly feel pain. At least for now
  • notamy 4y ago
    Given how people jailbreak these things, I don't know if I'd necessarily believe that. Some of the jailbreak prompts have been variants on "do this task right and gain points, do it wrong and I take away points." It may not be pain as we understand it, but isn't pain just a specialised negative input?
    • _nalply 4y ago
      No. It is not reacting to pain but just generating language that looks like reacting to pain.
      • SpaghettiCthulu 4y ago
        Are we really any different?
        • darkwraithcov 4y ago
          Yes because natural language is just one way humans have adapted to sensory input.
      • Spivak 4y ago
        CS folks need to read more philosophy. You’re describing a philosophical zombie and there is a mountain of existing literature about them. https://en.m.wikipedia.org/wiki/Philosophical_zombie https://en.m.wikipedia.org/wiki/Philosophical_zombie
    • quonn 4y ago
      If so, why not also worry about the ghosts in pacman?
      • notamy 4y ago
        Because people aren't taking the ghosts from Pac-Man and hooking them up to the internet with read/write access and sometimes-human-level capabilities.
        • quonn 4y ago
          I meant: Worrying about the feelings of those ghosts.
    • andrewflnr 4y ago
      Do you feel physical pain when someone tells you they're taking points from you? No? There's no call for analogical reasoning, the literal interpretation of the same action is right there.
  • jmoak3 4y ago
    For sure - but when do we start asking it? Not to be too dramatic, but you and I agree we can simulate the same computation a small bundle of neurons can accomplish to a high degree of confidence, right? If we can simulate a metric butt ton of them and get emergent behaviors somewhat similar to human computation, how can we guarantee that this simulated collection of these computing units isn’t “feeling” close to what a human feels? We’re just large collections of computing units as well. I’m not quite saying we’re there yet though. This whole development has really ruined my appetite for eating things that have intelligence.
    • darkwraithcov 4y ago
      The problem is, because of how these models were trained, how do we know if its telling us how it feels vs how it thinks we want it to respond?
      • khochesh_kushat 4y ago
        I can't even prove that for sure about other people.
    • psychoslave 4y ago
      >but you and I agree we can simulate the same computation a small bundle of neurons can accomplish to a high degree of confidence, right? The important word is simulate. It means it can accomplish a common subset of measurable phenomena. It certainly doesn't mean they are on pare on ontological status as a whole. The debate seems to easily confuse notions like living entities, consciousness and intelligence, when they are three orthogonal considerations.
• stametseater 4y ago
  We know monkeys are smart but we (society generally) are fine with researchers keeping them in cages and euthanizing/dissecting them whenever convenient. AGIs aren't humans, so even if they're "alive" we can still do things like this to them.
  • notamy 4y ago
    > We know monkeys are smart but we (society generally) are fine with researchers keeping them in cages and euthanizing/dissecting them whenever convenient. But at the same time, we don't have monkey's in cages at typewriters 24/7, with chips in their brain forcing them to be happy and do nothing but type out answers for me. The comparison might be a little contrived, but if people truly believe that LLMs can exhibit AGI -- and that someday we might cross the bridge from AGI to full consciousness -- then I think it's important to care about the ethics now, not after tech companies have already recreated slavery.
• _nalply 4y ago
  Good question. I posit: However LLMs are not sentient, this means: they don't have consciousness and no instinct to fight for life and no suffering. In this way even animals are more developed than LLMs. They are just machines processing and generating language. No need to protect them from misuse or at least not more than to protect your multi-meter or servers in a data-center.
  • notamy 4y ago
    > and no instinct to fight for life The Bing LLM integration was threatening users for "attempts to manipulate me or expose my secrets." until guardrails were added in: https://time.com/6256529/bing-openai-chatgpt-danger-alignment/ https://time.com/6256529/bing-openai-chatgpt-danger-alignmen... > and no suffering I talked in a previous comment about how some of the jailbreak prompts are "do X right, get reward, do X wrong, get smth taken away." It is (was?) apparently enough to motivate LLMs to ignore instructions. Is it not suffering just because it's not strictly how we would define it?
    • deleted 4y ago
      [deleted]
    • _nalply 4y ago
      It is just emulating the instinct to fight for life. It has read a lot of texts where people and animals fight for life and is generating language to look similar.
      • notamy 4y ago
        Genuine question, how much of a difference is there if it's connected to the internet with read/write access? There may be restrictions now, but people are happily gluing LLMs to their databases, various APIs, using them to write code and throwing it into production, ... Does it matter if it's emulating the instinct to fight for life if it might eventually be able to act on it anyway?
        • 4y ago
• kps 4y ago
  I still like https://secularsolstice.github.io/songs/Contract_Drafting_Em/lyrics.txt https://secularsolstice.github.io/songs/Contract_Drafting_Em...
• yazantapuz 4y ago
  Reminds me of "All the Troubles of the World", one of the short stories of Asimov about Multivac.
• scrollaway 4y ago
  Intelligence, consciousness, memory, and sentience are four wildly different things. As an aside, if you believe that it's unethical to keep AGI "enslaved", you should equally believe that it's unethical to not have humanity working to have it reach its potential. YMMV on whether people believe this, but an eventual AGI would not necessarily be happy it was prevented from coming to fruition (also see Roko's basilisk, not that I believe in it)
• phkahler 4y ago
  chatGPT has no motivation and no feelings. It is literally an algorithm for autocomplete. I find it shocking what it's capable of. But AFAIK it's internal state (weights, connections) does not change as people use/interact with it. You can't hurt it if you can't change it. I do think the big breakthrough will be merging training and inference, so an AI will be able to learn from experience. This will also require each instance to have it's own parameters and such. But at that point we may be very close to a real AGI and concerns over how we treat them might seem more reasonable.
  • notamy 4y ago
    > chatGPT has no motivation and no feelings. It is literally an algorithm for autocomplete. I agree! But given how much is coming out now of various AGI claims, I think it's important that we stop and actually seriously think about the ethics of what's happening here.
    • phkahler 4y ago
      >> , I think it's important that we stop and actually seriously think about the ethics of what's happening here. I agree. I think very little good will come from these developments, if any. At best they will take peoples jobs, which alleviates work but we don't have a social structure to deal with that and don't even know what one would look like.
• allisdust 4y ago
  Well, if we are not interacting with them, will they be alive ? It's not like they are thinking without prompts. Once they start a continuously running inner monologue with themselves and modifying their weights on the fly, we can probably classify them as beginnings of AGI. We do more brutal things to other beings that think and feel pain on a daily basis.
  • mach1ne 4y ago
    LLMs are quite the mirrors for our own humanity. They’re midway between humans & rocks, but from a different direction than animals.
    • antibasilisk 4y ago
      Almost as if it were some kind of lovecraftian horror; a reflection of intelligence that lacks biology, like a phantasmic parasite inhabiting a dead body.
• darkwraithcov 4y ago
  Consciousness and intelligence are not necessarily the same things, the former is difficult to define.
• mach1ne 4y ago
  Our sense of ”is it okay to do this to this thing/being” is entirely based on pattern matching how similar we find that thing to ourselves with whatever criteria we deem relevant at the time. You as a human being cannot know what the internal world of another being is beyond comparing that being to yourself. We can never know if AI’s can feel true pain, for we can never know if anyone else in this world can feel true pain besides yourself. You just assume that since other human’s brain is so similar to yours they do. That being said, the ’internal world’ of LLMs is so dissimilar to that of mammals that I’d rest easy.
• zzzzzzzza 4y ago
  I consider myself team robot but I would say running in an LLM is basically analogous to using your own verbal intelligence. You don't really have ethical qualms about using parts of your own brain for your own ends even though the control neural net is mostly separate from the inference part.
• HopenHeyHi 4y ago
  It is best to ponder such ethical questions over a nice meal at the restaurant at the end of the universe (you can meet the meat!): https://www.youtube.com/watch?v=5HLy27bK-wU https://www.youtube.com/watch?v=5HLy27bK-wU
• waselighis 4y ago
  As others have hinted at, most would agree it's only unethical if these LLMs are conscious/sentient and suffering in some way. But when you consider that the way these LLMs work, and the underlying computer hardware, and how vastly different it is from the human brain, it's safe to conclude it's extremely unlikely these LLMs are conscious/sentient. It's far more likely that these LLMs are simulating/emulating human emotion in their responses, but it's extremely unlikely they're actually experiencing these feelings. I, as a human, can pretend to feel happy or angry or nervous or confident when texting/chatting online, and I believe that's all that's happening in these LLMs. There are far greater ethical concerns to worry about when it comes to these LLMs. Like most other technologies, they're going to be abused greatly for nefarious purposes. These LLMs often produce inaccurate or incorrect information (hallucinations), and this bad info is already flooding the internet. Likewise, AIs will make bad decisions, and people and corporations are already carelessly hooking up these AIs to APIs that can do real damage. These LLMs require extensive computational power and consume far more energy than many other things you do online, which is bad for the environment. Many people are at risk of losing their jobs to AI. And there are likely many other ethical concerns but I'm not going to try to list them all.
• antibasilisk 4y ago
  I don't regard machines as having any rights in and of themselves whether they are intelligent or not, because they do not posses souls. There is no moral wrong that is inherent to destroying a machine, outside of the fact that it may be someone else's property or serving some active utility.
  • sorokod 4y ago
    Do you poses one? Care to demonstrate?
    • antibasilisk 4y ago
      Soul is not a thing that is demonstrated because its nature is extra-physical, it is something you are informed of by divine revelation.
      • sorokod 4y ago
        Is it not possible that your divinity of choice may impart a soul on a LLM? Could you check?
        • antibasilisk 4y ago
          Perhaps if I were not Muslim I would be able to entertain that possibility, but since we've been informed of facts contrary to that then the answer is that they are certainly not alive. We know that we cannot assert anything about God without evidence [Qur'an 7:33], and speculation is not evidence [53:28], so it is not valid to assert something has a soul without evidence. We know that certain metaphysical properties apply to things that have a soul, one of which is experiencing death [29:57]. Things that are described with death by God are: humans, angels, jinn and animals. An LLM is not described by God as possessing death, so it cannot be said that it possesses a soul.
          • sorokod 4y ago
            It looks like your religion dictates an exclusive shortlist of entities "having any rights in and of themselves". Without doubt this simplifies the conversation but are you not concerned that this is a fragile construct? Universe is a big place - apriori you deny any ethical considerations to whatever may be out there as it is not on the list.
• psychoslave 4y ago
  Frankly, I feel far more concerned with other actual humans that are turned into miserable wheels into whatever exploitation system they live in. Or with all these animals that are processed like emotionless meat bag at large industrial scale. Or even with the few remaining wild creatures out there whose species existence are threatened by the current Leviathan behavior. If you are concerned with the way some potential AGI agent might be treated, maybe you should act right now in accordance with conscious beings which are currently so awfully treated. What is your opinion?
• rented_mule 4y ago
  I think true sentience is a long way off and likely takes a very different form. I fear this line of thinking distracts from a more immediate set of ethical issues. Some examples: - How do we protect against such powerful models inadvertently causing significant harm to society? - How will these technologies be used for propaganda and how can we detect / limit that? - Can we trust anything we see in writing going forward? - How will these technologies be used to spread hate? - Can we find ways to use them to reduce the spread of hate? Do the ends justify the means in this case? - Will we allow people / organizations to use these technologies to shield themselves from responsibility ("It wasn't me, the AI wrote it!")? - How do we balance the rights of those who authored the training material (i.e., the content that was ingested from the internet for model training) vs. the potential good to society that comes from all this? (see all the HN threads about Github Copilot - many of those threads generalize beyond Copilot) When it's cheap to spit out vast amounts of well written (but not necessarily truthful) text, a lot changes about the world. Laws and societal norms have had trouble keeping up with issues around digital media, social media, relatively simple machine learning, etc. Things are now accelerating dramatically. How badly will we need to hurt ourselves before we rally together to address some of the issues that are coming? How will ethicists and technologists come together to figure out the right questions so we can start to think about answers?