6 ms·

If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions whi

• Lewton 11d ago
  > If models had persistent memory or an evolving set of weights, you could easily construct an argument that they can experience "suffering" or other emotions which resultingly adjusts their personality. They do, during training
  • add-sub-mul-div 11d ago
    If it was me, being forced to read all of Reddit is something I'd consider "suffering."
  • swiftcoder 11d ago
    The argument that models are conscious only during training, and not conscious during inference feels like a painful one to make. Are we killing models when they reach the training objective?
    • pixl97 11d ago
      Technically we are killing models when they don't reach the training objective by throwing those weights away via survival of the fittest (fit meaning what we humans think we want). Kinda like creating quantum copies of your child and keeping the ones that answer correctly and shooting the other ones in the face.
• vidarh 11d ago
  A model detached from a harness, sure. But we routinely do add persistent memory to systems incorporating LLMs.
  • dgellow 11d ago
    They don’t have persistent memory though. Models are static, what you have is a model that will reprocess the whole series of prompts + some data fetched from a data store. A harness is just a while loop prompting an LLM and executing tool calls, but the model is purely static
    • vidarh 11d ago
      The model does not. The system it runs in can. My agents have persistent memory. It is not at all clear whether or the distinction that the long-term memory is external from the model matters.
      • chrisjj 11d ago
        > My agents have persistent memory. So your disc drive likewise experiences emotions?
        • vidarh 11d ago
          Does your disc drive have the complexity of an LLM that shows behaviour consistent with having emotions? I'd consider that a prerequisite, but not sufficient, for that question to make sense.
          • chrisjj 11d ago
            > Does your disc drive have the complexity of an LLM that shows behaviour consistent with having emotions? An LLM does not show behaviour consistent with having emotions. It's regurgitation of humans' written accounts of emotions is not "having emotions".
            • vidarh 10d ago
              Your second sentence does not support the claim in your first sentence; it's merely a second unsupported claim. It is also entirely irrelevant to the argument. The threshold for behaviour consistent with having emotions (note: not behaviour sufficient to prove emotions) was one I purposefully set extremely low, because your comparison to a disk drive is so ludicrous, and yet your disc drive will still not clear it. Do you have an actual argument?
• deleted 11d ago
  [deleted]
• shawnz 11d ago
  So if a human loses their memory and their neutral plasticity (with age for example), then they are no longer capable of experiencing suffering?
  • pton_xd 11d ago
    It's not just memory, the physical body also encodes trauma and stress (The Body Keeps the Score is a book that touches on this). I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.
    • JoshTriplett 11d ago
      > I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering. Just because you don't remember the suffering doesn't mean you didn't experience it in the moment. Suffering does not suddenly become okay if your mind and body are going to forget it. "It's okay to torture someone if their mind and body both won't remember it afterwards" sure is a take.
      • fwip 11d ago
        Perhaps off-topic, but there's some people who believe that this can be what happens during surgery under anesthesia, if the dosage is off. From what I understand, there's 3 parts to modern anesthesia: Blocking pain signals, preventing the formation of memories, and inducing paralysis of the major muscle groups. If the former is dialed in too weakly, it's possible that the patient is feeling pain, unable to do anything about it, but won't remember it at all when they wake up. (Looking it up now, it seems that making the patient unconscious is perhaps the same drug that prevents memory formation... I realize now I understand this less well than I thought I did.)
        • JoshTriplett 11d ago
          Relatedly, look at the concept of "twilight anesthesia".
• myrmidon 11d ago
  Is the ability to form and retain long-term memories a necessary precondition for rights in humans though? Incinerating demented people is highly ethically questionable (even if you could be sure that the subjects have no memories left and are unable to form new ones).
  • qlte 11d ago
    This argument would be difficult to reconcile if we didn't have billions of examples of humans forming memories plus modern evolutionary biology to trivially demonstrate such cases are rare pathologies but still human beings.
  • joshheitzman 11d ago
    > Is the ability to form and retain long-term memories a necessary precondition for rights in humans though? For a species / class of being yes it is necessary. Some members of that species / class being exceptions does not make the generalization invalid. Without the ability to form and retain long-term memories we wouldn't have developed language and abstract concepts like rights. Whether people like it or not, rights aren't natural or god-given. Rights are only enforced or protected by the tip of a spear (or point of a gun, or hover of a drone). There are no rights in nature and they are no where to be found when social order breaks down. Animals can't make war for their rights, so they only get the rights we deign to give them. It's not nothing, but its not the same. There is no question to me that my dogs are conscious, but its not at the same level as humans. There is no question for me that LLMs lack consciousness as they are stateless.
• deleted 11d ago
  [deleted]
• mitxela 11d ago
  Model state is often called "context". You may have heard of it.