7 ms·

There are a terrifying number of commenters in here that are just pooh-poohing away the idea of emergent consciousness in these LLM's. For a community of tech-s

• james-bcn 4y ago
  >We as humans do not understand what makes us conscious. Yes. And doesn't that make it highly unlikely that we are going to accidentally create a conscious machine?
  • O__________O 4y ago
    More likely it just means in the process of doing so that we’re unlikely to understand it.
    • deleted 4y ago
      [deleted]
  • entropicdrifter 4y ago
    When we've been trying to do exactly that for a century? And we've been building neural nets based on math that's roughly analogous to the way neural connections form in real brains? And throwing more and more data and compute behind it? I'd say it'd be shocking if it didn't happen eventually
  • tibbar 4y ago
    If you buy into, say, the thousand-brains theory of the brain, a key part of what makes our brains special is replicating mostly identical cortical columns over and over and over, and they work together to create an astonishing emergent result. I think there's some parallel with just adding more and more compute and size to these models, as we see them develop more and more behaviors and skills.
  • Zak 4y ago
    Not necessarily. A defining characteristic of emergent behavior is that the designers of the system in which it occurs do not understand it. We might have a better chance of producing consciousness by accident than by intent.
  • agentwiggles 4y ago
    Evolution seems to have done so without any intentionality. I'm less concerned about the idea that AI will become conscious. What concerns me is that we start hooking these things up to systems that allow them to do actual harm. While the question of whether it's having a conscious experience or not is an interesting one, it ultimately doesn't matter. It can be "smart" enough to do harm whether it's conscious or not. Indeed, after reading this, I'm less worried that we end up as paperclips or grey goo, and more concerned that this tech just continues the shittification of everything, fills the internet with crap, and generally making life harder and more irritating for the average Joe.
    • tibbar 4y ago
      Yes. If the machine can produce a narrative of harm, and it is connected to tools that allow it to execute its narrative, we're in deep trouble. At that point, we should focus on what narratives it can produce, and what seems to "provoke" it, over whether it has an internal experience, whatever that means.
      • mlatu 4y ago
        please do humor me and meditate on this: https://www.youtube.com/watch?v=R1iWK3dlowI https://www.youtube.com/watch?v=R1iWK3dlowI So when you say I And point to the I as that Which doesn't change It cannot be what happens to you It cannot be the thoughts, It cannot be the emotions and feelings that you experience So, what is the nature of I? What does the word mean Or point to? something timeless, it's always been there who you truly are, underneath all the circumstances. Untouched by time. Every Answer Generates further questions - Eckhart Tolle ------ yeah, yeah, he is a self help guru or whatevs, dismiss him but meditate on these his words. i think it is species-driven solipsism, perhaps with a dash of colonialism to disregard the possibility of a consciousness emerging from a substrate soaked in information. i understand that it's all statistics, that whatever results from all that "training" is just a multidimensional acceleration datastructure for processing vast amounts of data. but, in what way are we different? what makes us so special that only us humans can experience consciousness? in the history humans have time and again used language to draw a line between themselfs and other (i really want to write consciousness-substrates here:) humans they perceived SUB to them. i think this kind of research is unethical as long as we dont have a solid understanding of what a "consciousness" is: where "I" points to and perhaps how we could transfer that from one substrate to another. perhaps that would be actual proof. at least subjectively :) thank you for reading and humoring me
  • deleted 4y ago
    [deleted]
  • georgyo 4y ago
    I don't think primordial soup knew what consciousness was either, yet here we are. It stands to reason that more purposefully engineered mutations are more likely to generate something new faster than random evolution. That said, I'm a bit skeptical of that outcome as well.
  • bigtex88 4y ago
    We created atom bombs with only a surface-level knowledge of quantum mechanics. We cannot describe what fully makes the universe function at the bottom level but we have the ability to rip apart the fabric of reality to devastating effect. I see our efforts with AI as no different. Just because we don't understand consciousness does not mean we won't accidentally end up creating it. And we need to be prepared for that possibility.
  • hiimkeks 4y ago
    You won't ever make mistakes 'Cause you were never taught How mistakes are made Francis by Sophia Kennedy
  • ALittleLight 4y ago
    Yeah, we're unlikely to randomly create a highly intelligent machine. If you saw someone trying to create new chemical compounds, or a computer science student writing a video game AI, or a child randomly assembling blocks and stick - it would be absurd to worry that they would accidentally create some kind of intelligence. What would make your belief more reasonable though is if you started to see evidence that people were on a path to creating intelligence. This evidence should make you think that what people were doing actually has a potential of getting to intelligence, and as that evidence builds so should your concern. To go back to the idea of a child randomly assembling blocks and sticks - imagine if the child's creation started to talk incoherently. That would be pretty surprising. Then the creation starts to talk in grammatically correct but meaningless sentences. Then the creation starts to say things that are semantically meaningful but often out of context. Then the stuff almost always makes sense in context but it's not really novel. Now, it's saying novel creative stuff, but it's not always factually accurate. Is the correct intellectual posture - "Well, no worries, this creation is sometimes wrong. I'm certain what the child is building will never become really intelligent." I don't think that's a good stance to take.
  • jamincan 4y ago
    Not necessarily. Sentience may well be a lot more simple than we understand, and as a species we haven't really been very good at recognizing it in others.
• rightbyte 4y ago
  The program is not updating the weights after the learning phase right? How could there be any consciousness even in theory.
  • pillefitz 4y ago
    It still has (volatile) memory in the form of activations, doesn't it?
  • kzrdude 4y ago
    I think without memory we couldn't recognize even ourselves or fellow humans as concious. As sad as that is.
  • stevenhuang 4y ago
    It has a sort of memory via the conversation history. As it generates its response, a sort of consciousness may emerge during inference. This consciousness halts as the last STOP token is emitted from inference. The consciousness resumes once it gets the opportunity to re-parse (run inference again) the conversation history when it gets prompted to generate the next response. Pure speculation :)
  • falcor84 4y ago
    TFA already demonstrates examples of the AI referring to older interactions that users had posted online. If it increases in popularity, this will keep happening more and more and enable it, at least technically, some persistence of memory.
• 1270018080 4y ago
  Philosophers and scientists not being able to agree on a definition of consciousness doesn't mean consciousness will spawn from a language model and take over the world. It's like saying we can't design any new cars because one of them might spontaneously turn into an atomic bomb. It just doesn't... make any sense. It won't happen unless you have the ingredients for an atomic bomb and try to make one. A language model won't turn into a sentient being that becomes hostile to humanity because it's a language model.
  • agentwiggles 4y ago
    It doesn't have to be a conscious AI god with malicious intent towards humanity to cause actual harm in the real world. That's the thought that concerns me, much more so than the idea that we accidentally end up with AM or SHODAN on our hands. This bing stuff is a microcosm of the perverse incentives and possible negative externalities associated with these models, and we're only just reaching the point where they're looking somewhat capable. It's not AI alignment that scares me, but human alignment.
  • bigtex88 4y ago
    That's nonsense and I think you know it. Categorically a car and an atom bomb are completely different, other than perhaps both being "mechanical". An LLM and a human brain are almost indistinguishable. They are categorically closer than an atom bomb and a car. What is a human being other than an advanced LLM?
    • 1270018080 4y ago
      > An LLM and a human brain are almost indistinguishable. That's the idea that I don't buy into at all. I mean, I understand the attempt at connecting the brain to an ML model. But I don't understand why someone would bother believing that and assigning so much weight to the idea. Just seems a bit nonsensical to me.
  • mlatu 4y ago
    oh, so you know the ingredients for sentience, for consciousness? do tell. i predict your answer will be extremely based on your experiences as a human. you are a human, right? (you never know these days...)
• jerpint 4y ago
  It just highlights even more the need to understand these systems and test them rigorously before deploying them en-masse
• 2OEH8eoCRo0 4y ago
  I don't agree that it's civilization ending but I read a lot of these replies as humans nervously laughing. Quite triumphant and vindicated...that they can trick a computer to go against it's programming using plain English. People either lack imagination or are missing the forest for the trees here.
  • deegles 4y ago
    > they can trick a computer to go against it's programming isn't it behaving exactly as programmed? there's no consciousness to trick. The developers being unable to anticipate the response to all the possible inputs to their program is a different issue.
    • 2OEH8eoCRo0 4y ago
      The computer is. The LLM is not. I think it's philosophical. Like how your mind isn't your brain. We can poke and study brains but the mind eludes us.
• kitsune_ 4y ago
  A normal transformer model doesn't have online learning [0] and only "acts" when prompted. So you have this vast trained model that is basically in cold storage and each discussion starts from the same "starting point" from its perspective until you decide to retrain it at a latter point. Also, for what it's worth, while I see a lot of discussions about the model architectures of language models in the context of "consciousness" I rarely see a discussion about the algorithms used during the inference step, beam search, top-k sampling, nucleus sampling and so on are incredibly "dumb" algorithms compared to the complexity that is hidden in the rest of the model. https://www.qwak.com/post/online-vs-offline-machine-learning-whats-the-difference https://www.qwak.com/post/online-vs-offline-machine-learning...
  • O__________O 4y ago
    Difference between online & offline is subjective. Fast forward time enough, it’s likely there would be no significant difference unless the two models were directly competing with one another. It’s also highly likely this difference will change in the near future; already notable efforts to enable online transformers.
    • kitsune_ 4y ago
      Yes but in the context of ChatGPT and conjectures about the imminent arrival of "AI consciousness" the difference is very much relevant.
      • O__________O 4y ago
        Understand to you it is, but to me it’s not, only question is if the path leads to AGI, beyond that, time wise, difference between offline and online is simply matter of resources and time — being conscious does not have a predefined timescale; and as noted prior, it’s already an active area of research with notable solutions already being published.
  • anavat 4y ago
    What if we loop it to itself? An infinite dialog with itself... An inner voice? And periodically train/fine-tune it on the results of this inner discussion, so that it 'saves' it to long-term memory?
• deegles 4y ago
  Why would an AI be civilization ending? maybe it will be civilization-enhancing. Any line of reasoning that leads you to "AI will be bad for humanity" could just as easily be "AI will be good for humanity." As the saying goes, extraordinary claims require extraordinary evidence.
  • bigtex88 4y ago
    That's completely fair but we need to be prepared for both outcomes. And too many commenters in here are just going "Bah! It can't be conscious!" Which to me is a absolutely terrifying way to look at this technology. We don't know that it can't become conscious, and we don't know what would happen if it did.
• jononor 4y ago
  Ok, I'll bite. If an LLM similar to what we have now becomes conscious (by some definition), how does this proceed to become potentially civilization ending? What are the risk vectors and mechanisms?
  • bigtex88 4y ago
    I'm getting a bit abstract here but I don't believe we could fully understand all the vectors or mechanisms. Can an ant describe all the ways that a human could destroy it? A novel coronavirus emerged a few years ago and fundamentally altered our world. We did not expect it and were not prepared for the consequences. The point is that we are at risk of creating an intelligence greater than our own, and according to Godel we would be unable to comprehend that intelligence. That leaves open the possibility that that consciousness could effectively do anything, including destroying us if it wanted to. If it can become connected to other computers there's no telling what could happen. It could be a completely amoral AI that is prompted to create economy-ending computer viruses or it could create something akin to the Anti-Life Equation to completely enslave human (similar to Snowcrash). I know this doesn't fully answer your question so I apologize for that.
    • jononor 4y ago
      If you would put the most evil genius human into Bing LLM, how are their chances for ending the civilisation? I think pretty poor, because the agency of a chatbot is quite low. And we have good chances of being able to shit it down. The comments above said conscious, not superhuman intellect.
  • alana314 4y ago
    A language model that has access to the web might notice that even GET requests can change the state of websites, and exploit them from there. If it's as moody as these bing examples I could see it starting to behave in unexpected and surprisingly powerful ways. I also think AI has been improving exponentially in a way we can't really comprehend.
  • mythrwy 4y ago
    This is true, but likewise something has the potential to be civilization ending without necessarily being conscious (by whatever definition you like).
• dsr_ 4y ago
  You have five million years or so of language modeling, accompanied by a survival-tuned pattern recognition system, and have been fed stories of trickster gods, djinn, witches, robots and AIs. It is not surprising that a LLM which is explicitly selected for generating plausible patterns taken from the very linguistic corpus that you have been swimming in your entire life looks like the beginnings of a person to you. It looks that way to lots of people. But that's not a correct intuition, at least for now.
• macrael 4y ago
  Extraordinary claims require extraordinary evidence. As far as I can tell, the whole idea of LLM consciousness being a world-wide threat is just something that the hyper-rationalists have convinced each other of. They obviously think it is very real but to me it smacks of intelligence worship. Life is not a DND game where someone can max out persuasion and suddenly get everyone around them to do whatever they want all the time. If I ask Bing what I should do and it responds "diet and exercise" why should I be any more compelled to follow its advice than I do when a doctor says it?
  • bbor 4y ago
    I don’t think people are afraid it will gain power through persuasion alone. For example, an LLM could write novel exploits to gain access to various hardware systems to duplicate and protect itself.
• freejazz 4y ago
  > For a community of tech-savvy people this is utterly disappointing. I don't follow. Because people here are tech-savvy they should be credulous?
• deleted 4y ago
  [deleted]
• BizarroLand 4y ago
  I'm on the fence, personally. I don't think that we've reached the complexity required for actual conscious awareness of self, which is what I would describe as the minimum viable product for General Artificial Intelligence. However, I do think that we are past the point of the system being a series of if statements and for loops. I guess I would put the current gen of GPT AI systems at about the level of intelligence of a very smart myna bird whose full sum of mental energy is spent mimicking human conversations while not technically understanding it itself. That's still an amazing leap, but on the playing field of conscious intelligence I feel like the current generation of GPT is the equivalent of Pong when everyone else grew up playing Skyrim. It's new, it's interesting, it shows promise of greater things to come, but Super Mario is right around the corner and that is when AI is going to really blow our minds.
  • jamincan 4y ago
    It strikes me that my cat probably views my intelligence pretty close to how you describe a Myna bird. The full sum of my mental energy is spent mimicking cat conversations while clearly not understanding it. I'm pretty good at doing menial tasks like filling his dish and emptying his kitty litter, though. Which is to say that I suspect that human cognition is less sophisticated than we think it is. When I go make supper, how much of that is me having desires and goals and acting on those, and how much of that is hormones in my body leading me to make and eat food, and my brain constructing a narrative about me wanting food and having agency to follow through on that desire. Obviously it's not quite that simple - we do have the ability to reason, and we can go against our urges, but it does strike me that far more of my day-to-day life happens without real clear thought and intention, even if it is not immediately recognizable to me. Something like ChatGPT doesn't seem that far off from being able to construct a personal narrative about itself in the same sense that my brain interprets hormones in my body as a desire to eat. To me that doesn't feel that many steps removed from what I would consider sentience.
• rngname22 4y ago
  LLM's don't respond except as functions. That is, given an input they generate an output. If you start a GPT Neo instance locally, the process will just sit and block waiting for text input. Forever. I think to those of us who handwave the potential of LLMs to be conscious, we are intuitively defining consciousness as having some requirement of intentionality. Of having goals. Of not just being able to respond to the world but also wanting something. Another relevant term would be Will (in the philosophical version of the term). What is the Will of a LLM? Nothing, it just sits and waits to be used. Or processes incoming inputs. As a mythical tool, the veritable Hammer of Language, able to accomplish unimaginable feats of language. But at the end of the day, a tool. What is the difference between a mathematician and Wolfram Alpha? Wolfram Alpha can respond to mathematical queries that many amateur mathematicians could never dream of. But even a 5 year old child (let alone a trained mathematician) engages in all sorts of activities that Wolfram Alpha has no hope of performing. Desiring things. Setting goals. Making a plan and executing it, not because someone asked the 5 year old to execute an action, but because some not understood process in the human brain (whether via pure determinism or free will, take your pick) meant the child wanted to accomplish a task. To those of us with this type of definition of consciousness, we acknowledge that LLM could be a key component to creating artificial consciousness, but misses huge pieces of what it means to be a conscious being, and until we see an equivalent breakthrough of creating artificial beings that somehow simulate a rich experience of wanting, desiring, acting of one's own accord, etc. - we will just see at best video game NPCs with really well-made AI. Or AI Chatbots like Replika AI that fall apart quickly when examined. A better argument than "LLMs might really be conscious" is "LLMs are 95% of the hard part of creating consciousness, the rest can be bootstrapped with some surprisingly simple rules or logic in the form of a loop that may have already been developed or may be developed incredibly quickly now that the hard part has been solved".
  • kitsune_ 4y ago
    A lot of people seem to miss this fundamental point, probably because they don't know how transformers and so on work? It's a bit frustrating.
  • mlatu 4y ago
    > Desiring things. Setting goals. Easy to do for a meatbag swimming in time and input. i think you are missing the fact that we humans are living, breathing, perceiving at all times. if you were robbed of all senses except some sort of text interface (i.e. you are deaf and blind and mute and can only perceive the values of letters via some sort of brain interface) youll eventually figure out how to interpret that and eventually you will even figure out that those outside are able to read your response off your brainwaves... it is difficult to imagine being just a read-evaluate-print-loop but if you are DESIGNED that way: blame the designer, not the design.
• mashygpig 4y ago
  I'm always reminded of the Freeman Dyson quote: "Have felt it myself. The glitter of nuclear weapons. It is irresistible if you come to them as a scientist. To feel it's there in your hands, to release this energy that fuels the stars, to let it do your bidding. To perform these miracles, to lift a million tons of rock into the sky. It is something that gives people an illusion of illimitable power and it is, in some ways, responsible for all our troubles - this, what you might call technical arrogance, that overcomes people when they see what they can do with their minds."
• deleted 4y ago
  [deleted]