4 ms·

You’re missing my point. Take one of the articles described in that comment, titled “The Internal State of an LLM Knows When It's Lying”. It states “In this pap

by reshlo 2y ago

You’re missing my point. Take one of the articles described in that comment, titled “The Internal State of an LLM Knows When It's Lying”. It states “In this paper, we provide evidence that the LLM's internal state can be used to reveal the truthfulness of statements.” Both of these are untrue, for a number of reasons. - An LLM knowing when it is lying is not the same thing as its internal state being able to “reveal the truthfulness of statements”. The LLM does not know when it is lying, because LLMs do not know things. - It is incapable of lying, because lying requires possessing intent to lie. Stating untrue things is not the same as lying. - As the paper states shortly afterwards, what it actually shows is “given a set of test sentences, of which half are true and half false, our trained classifier achieves an average of 71% to 83% accuracy”. That’s not the same thing as it being able to “reveal the truthfulness of statements”. No intellectually honest person would claim that this finding means an LLM “knows when it is lying”.

famouswaffles 2y ago
I'm not missing your point. I just don't think you're making one. You keep saying the same nonsense over and over again. A LLM does not know things so... What kind of argument is that ? You're working backwards from a conclusion that is nothing but your own erroneous convictions on what a "statistical model" is and are undertaking a whole lot of mental gymnastics to stay there. There are a lot of papers there that all try to approach this in different ways. You should read them and try to make an honest argument and that doesn't involve "This doesn't count because - claim that is in no way empirically or theoretically validated."
- reshlo 2y ago
  You are the one claiming that LLMs are conscious, so it falls to you to prove it. I argued that LLMs do not have the capacity to have ideas or to know things, and you tried to prove me wrong by providing examples of papers that show, for example, that LLMs have internal states that can be used to predict the likelihood that what they will output will be facts. But that doesn’t disprove what I said, because that’s not what it means to have ideas or know things. By definition, only conscious beings can do those things.
  - famouswaffles 2y ago
    >You are the one claiming that LLMs are conscious, so it falls to you to prove it. If a machine is doing things previously before ascribed to "conscious beings" then it's on you to tell me why the machine is not conscious. Hopefully something other than the circular - "It cannot be conscious so it is not conscious". But whatever. I hadn't quite realized this had devolved into a debate on consciousness. I think that's on me but I have no interest in a back and forth on such an ill-defined, ill-understood concept. You don't know what consciousness is, what is required of it or what makes it tick in you, you have no way of proving one way or another anybody else has it. It's extremely silly then don't you think to make such bold declarations on what doesn't have it ? especially with circular arguments. What difference does it make if you won't call it conscious if it does anything a conscious being does ? That's just semantics.
    - reshlo 2y ago
      You’re still failing to understand that a model being able to output a prediction of something is not the same thing as it “knowing” that thing. The Newton-Raphson method doesn’t “know” what the root of a function is, it just outputs an approximation of it. > It’s extremely silly then don’t you think to make such bold declarations on what doesn’t have it? I don’t find it particularly bold to respond to your assertion that a piece of mathematics is sentient life by stating that you haven’t proven that it is, and that in the absence of that proof, the most rational position is to continue to believe that it is not, as we have done for millennia. The burden of proof is on you. > if it does anything a conscious being does You haven’t shown that it can do anything that only conscious beings can do. Being able to generate a passable approximation of text that might follow some prompt doesn’t mean that it understands the prompt, or its answer. As an obvious example, if you give LLMs maths problems, they change their answers if you change the names of the people in the question. They’re not actually doing maths. > Notice anything? It’s not just that the performance on MathGLM steadily declines as the problems gets bigger, with the discrepancy between it and a calculator steadily increasing, it’s that the LLM based system is generalizing by similarity, doing better on cases that are in or near the training set, never, ever getting to a complete, abstract, reliable representation of what multiplication is.[0] [0] https://garymarcus.substack.com/p/math-is-hard-if-you-are-an-llm-and https://garymarcus.substack.com/p/math-is-hard-if-you-are-an...

by reshlo 2y ago

famouswaffles 2y ago

I'm not missing your point. I just don't think you're making one. You keep saying the same nonsense over and over again. A LLM does not know things so... What kind of argument is that ? You're working backwards from a conclusion that is nothing but your own erroneous convictions on what a "statistical model" is and are undertaking a whole lot of mental gymnastics to stay there. There are a lot of papers there that all try to approach this in different ways. You should read them and try to make an honest argument and that doesn't involve "This doesn't count because - claim that is in no way empirically or theoretically validated."

reshlo 2y ago
You are the one claiming that LLMs are conscious, so it falls to you to prove it. I argued that LLMs do not have the capacity to have ideas or to know things, and you tried to prove me wrong by providing examples of papers that show, for example, that LLMs have internal states that can be used to predict the likelihood that what they will output will be facts. But that doesn’t disprove what I said, because that’s not what it means to have ideas or know things. By definition, only conscious beings can do those things.
- famouswaffles 2y ago
  >You are the one claiming that LLMs are conscious, so it falls to you to prove it. If a machine is doing things previously before ascribed to "conscious beings" then it's on you to tell me why the machine is not conscious. Hopefully something other than the circular - "It cannot be conscious so it is not conscious". But whatever. I hadn't quite realized this had devolved into a debate on consciousness. I think that's on me but I have no interest in a back and forth on such an ill-defined, ill-understood concept. You don't know what consciousness is, what is required of it or what makes it tick in you, you have no way of proving one way or another anybody else has it. It's extremely silly then don't you think to make such bold declarations on what doesn't have it ? especially with circular arguments. What difference does it make if you won't call it conscious if it does anything a conscious being does ? That's just semantics.
  - reshlo 2y ago
    You’re still failing to understand that a model being able to output a prediction of something is not the same thing as it “knowing” that thing. The Newton-Raphson method doesn’t “know” what the root of a function is, it just outputs an approximation of it. > It’s extremely silly then don’t you think to make such bold declarations on what doesn’t have it? I don’t find it particularly bold to respond to your assertion that a piece of mathematics is sentient life by stating that you haven’t proven that it is, and that in the absence of that proof, the most rational position is to continue to believe that it is not, as we have done for millennia. The burden of proof is on you. > if it does anything a conscious being does You haven’t shown that it can do anything that only conscious beings can do. Being able to generate a passable approximation of text that might follow some prompt doesn’t mean that it understands the prompt, or its answer. As an obvious example, if you give LLMs maths problems, they change their answers if you change the names of the people in the question. They’re not actually doing maths. > Notice anything? It’s not just that the performance on MathGLM steadily declines as the problems gets bigger, with the discrepancy between it and a calculator steadily increasing, it’s that the LLM based system is generalizing by similarity, doing better on cases that are in or near the training set, never, ever getting to a complete, abstract, reliable representation of what multiplication is.[0] [0] https://garymarcus.substack.com/p/math-is-hard-if-you-are-an-llm-and https://garymarcus.substack.com/p/math-is-hard-if-you-are-an...