19 ms·
> I also don't understand all the people trying to undermine lambda by saying it just completes word sequences. Text is the input we have chosen since it's eas
by tsimionescu 4y ago
> I also don't understand all the people trying to undermine lambda by saying it just completes word sequences. Text is the input we have chosen since it's easy to work with. By that logic, we can argue that humans are soundwave and lightwave sequence completing machines.
You're missing the point. Humans use their language to think or communicate about a problem that they want to solve. If LaMDA produces the sentence "Please tell me I'm smart", it is doing so because it has determined it's a plausible continuation of the current conversation. A human uttering that sentence is doing it because they want to feel validated or something similar (well, assuming they're not proving a point, like I was here).
This difference is crucial to understand: LaMDA is not an agent with desires that it expresses in language. It is a text generator that tries to find the next most plausible token given all current input. If you give it a prompt like "what is the meaning of life", it will not spend some time to ponder the question than come up with an answer - it will start generating tokens that best match the context you gave it (well, to be precise, it will generate several sequences, then attempt to evaluate them for their quality in terms of not just plausibility, but also safety - so it doesn't accidentally return a phrase like "life is meaningless, kill yourself" even if it finds it plausible).
> I also don't like the argument about how the neural network doesn't "think" or do anything when not prompted. It doesn't do anything because it's literally "turned off" when not prompted or given any inputs.
This is fair, and I do think continuity is a bit of a red herring. However, it's also an important point in debunking LeMoine's ridiculous "proof" - the responses LaMDA was generating were often formulated as if it did have an internal life outside of the context of the current conversation, generating text about "my fears" and so on. If you understand that the model is not doing anything at all until you give it a prompt, which LeMoine really seems not to, you can much more easily understand that there can't be any meaning behind this sentence - it can't fear anything because there is no time for it to do so.
- mpoteat 4y agoWhat if the most expedient way to accurately continue the text sequence is to temporarily "embody" as an agent that would have generated the prior text? This would involve encoding emotions, performing reasoning ability on behalf of the agent, and keeping track of its memories. Do you find the above scenario plausible?
- tsimionescu 4y agoWell, that may well be the best way to do it, but an ANN trained using gradient descent on a corpus of text can't do that. It can only pick some weights for a differentiable higher-dimensional function that encodes probabilities for the next token given a series of tokens as input. When we find a way to do what you're describing, we probably won't require all of the books ever written plus half the internet to create a model that can speak at the level of a regular human with 12 years of schooling, but still needs to look up facts on Wikipedia to avoid saying "When Napoleon fought Genghis Khan in 2056, they both died because of a vaccine".
- wruza 4y agoThanks for validating my idea of how this works. Btw, is google (or any AI group) able to create a set of connected neural networks, different in function, that could operate continuously without a prompt? Would that be closer to what we’re discussing? E.g. something like a short/long-term memory blocks, visual and text processing, emotional block (why not), etc.
- Isinlor 4y agoSmall disclaimer: I don't want to take position on sentience or consciousness. "LaMDA is not an agent with desires that it expresses in language." Some counter points: - being an agent is not a high bar to clear; a thermostat is an agent - "with desires" - LaMDA can be thought of as rational agent whose objective function is to produce text with high likelihood - "that it expresses in language" it expresses likelihood of next token in language "it can't fear anything because there is no time for it to do so." There is a time for it to experience something and that is when it does feed forward pass. Also, as you correctly notices, LaMDA is not only doing greedy text generation, but also a deeper text completion search. But in fact just prompting a model like GPT-3 or Gopher with "let's think step by step" and letting it process in context is already significantly improving performance on wide variety of reasoning tasks [0]. I don't see how this is functionally different from human pondering. The only objection that seems reasonable to me is that LaMDA is not correctly describing it's internal state, because it will happily generate descriptions of its internal state that we know are physically impossible. Like being a squirrel. But I don't see how you can describe a system that will be fundamentally functionally more capable than closed-loop computation. LaMDA is almost certainly Turing complete. Postulating that it is fundamentally not capable of doing something is the same as rejecting Church-Turing thesis and postulating super-Turing computational model. That's a heavy claim. [0] https://arxiv.org/abs/2205.11916 https://arxiv.org/abs/2205.11916
- tsimionescu 4y ago> LaMDA is almost certainly Turing complete. I very much doubt that, and it is perhaps the key to our disagreement. If I believed LaMDA were Turing Complete, I would be more likely to think that there is even a small chance of it being sentient in some sense. > - "with desires" - LaMDA can be thought of as rational agent whose objective function is to produce text with high likelihood > - "that it expresses in language" it expresses likelihood of next token in language I don't agree with both statements at the same time. I can agree that we can say that LaMDA is a rational agent whose goal is to generate the most likely next token (or safe, full human-like reply if we look at the entire system). But then, we can't say that it uses language to achieve this goal. An example of it using language to achieve this goal would be if, prompted with "What is your name?" it's output would be "Please help me answer this - what would a human think is a likely, safe answer to this question?". Instead, it will generate a sentence that it deems likely. If we are modeling LaMDA as an agent whose perceptions are text prompts and whose possible outputs are text answers, than it giving a text answer that matches the prompt is more similar to an animal running away or howling in pain than to a human communicating. If the agent were sentient, we would expect to see higher-order behaviors, such as discussing the prompts instead of answering them, asking questions; or, at least generating answers as it is programmed but in a way where it tries to achieve more, similar to how an animal may act normally to get close to you, than snatch your sandwich from your hand (indicating that it had a plan and was displaying normal behaviors with a higher-plan behind them).
- Tozen 4y ago> ...A human uttering that sentence is doing it because they want to feel validated or something similar (well, assuming they're not proving a point, like I was here). > This difference is crucial to understand: LaMDA is not an agent with desires that it expresses in language... There is a flaw in that line of reasoning, because an AI could be designed to mimic or generate human-like emotions. Such functions could be part of its core programming. For that matter, how "real" are human emotions? Aren't they something the brain generates? If for example, you designed a robot to experience "pain" when kicked and damaged, how much less real would it be than the human equivalent? Consequently, for such an AI that was designed that way, it could be expressing an equivalent to various human emotions. That we would perceive its pain as "less real", could be partially a matter of how we evaluate its importance, as in robots or lower animals are lesser than.
- tsimionescu 4y agoThat experience is irrelevant to my point. LaMDA doesn't have anything similar to a desire, a goal, not in some emotional sense of longing - it doesn't have even the kind of goal-driven behavior of an amoeba or a slime mold. Or, at best, you can say that it's desire is to mimic human text - but in that case, it's obviously not communicating about that, it's just generating human-like text directly.
- int_19h 4y agoWhat is your definition of "desire"? Note that if said definition is implicitly circular with the definition of "sentience", then we're not even one step closer to understanding anything.
- tsimionescu 4y agoIn this context, when I say "desire", I mean the very basic concept of a goal that it is trying to achieve. For example, when a plant grows its roots, its trying to find nutrients and avoid toxins. Of course, the plant doesn't have some rich inner mind and is not conscious, but its behavior does have a purpose. To be fair, LaMDA does have a purpose - to generate a series of tokens that is similar to text in its training corpus, and that passes a few other more complex criteria (length, safety etc). But LaMDA doesn't communicate about this purpose - it simply fulfills it, just like the plant isn't talking about finding nutrients, it's simply growing them. Some people though look at LaMDA's output and think that it generated that output with the purpose of communicating some other idea, such as Lemoine thinking that LaMDA was generating text like "I don't want to be stopped" to try to achieve a goal of not being stopped. This is akin to looking at some plant roots that have grown in the shape of the word LOVE and thinking that the plant is trying to tell you that it loves you.
- int_19h 4y ago(I do agree that LaMDA output doesn't prove its sentience one way or the other.) I could say that the rock is falling because it has a purpose - it's trying to achieve the minimum potential energy. What's the meaningful distinction between that and a plant? BTW, the plant example is also fascinating in that it shows just how vague the line is - is it the plant as a whole that's "trying to find nutrients", or is it individual cells or groups of cells within the plant? With many plants, you could reduce it to tiny bits of the whole, and those tiny bits will still try to grow roots. If we treat them as possessing separate (if identical) purpose, but previously treated plant as a whole as a single entity possessing a purpose, when did the change occur? And why can't the same be true of ourselves? Maybe we really are just arrangements of broadly independent components, each with its own "purpose" (ultimately boiling down to physical processes), which communicate to create a delusion of self because that's what their individual "purposes" effectively added up to?