6 ms·

What is the difference between a google map style application that shows pixels that are "right" for a road, and pixels that are "wrong" for a road? A pixel is

by Drakim 2y ago

What is the difference between a google map style application that shows pixels that are "right" for a road, and pixels that are "wrong" for a road? A pixel is a pixel, colors cannot be true or false. The only way we can say a pixel is right or wrong in google maps is whether the act of the human utilizing that pixel to understand geographical information results in the human correctly navigating the road. In the same way, an LLM can tell me all kinds of things, and those things are just words, which are just characters, which are just bytes, which are just electrons. There is no truth or false value to an electron flowing in a specific pattern in my CPU. The real question is what I, the human, get out of reading those words, if I end up correctly navigating and understanding something about the world based on what I read.

deleted 2y ago
[deleted]
_boffin_ 2y ago
But then again, that road, according to a neighboring country who thinks that it’s their land and it isn’t a road. Depending on what country you’re in, it makes a difference
tines 2y ago
> The real question is what I, the human, get out of reading those words So then you agree with the OP that an LLM is not intelligent in the same way that Google Maps is not intelligent? That seems to be where your argument leads but you're replying in a way that makes me think you are disagreeing with the op.
- contravariant 2y ago
  Any reply will always sound like someone is disagreeing, even if they claim not to. Though in this case I'm not even sure what the comment they're supposedly disagreeing with is even claiming. Is it even claiming anything?
  - tines 2y ago
    > Any reply will always sound like someone is disagreeing, even if they claim not to. Disagree! :) > Though in this case I'm not even sure what the comment they're supposedly disagreeing with is even claiming. Is it even claiming anything? It's offering support for the claim that LLMs hallucinate 100% of the time, even when their hallucinations happen to be true.
    - williamcotton 2y ago
      > It's offering support for the claim that LLMs hallucinate 100% of the time, even when their hallucinations happen to be true. Well this makes the term "hallucinate" completely useless for any sort of distinction. The word then becomes nothing more than a disparaging term for an LLM.
      - tines 2y ago
        Not really. It distinguishes LLM output from human output even though they look the same sometimes. The process by which something comes into existence is a valid distinction to make, even if the two processes happen to produce the same thing sometimes.
        williamcotton 2y ago
        Why is it a valid distinction to make? For example, does this distinction affect the assessed truthfulness of a signal? Does process affect the artfulness of a painting? “My five year old could draw that?”
        tines 2y ago
        It makes sense to do so in the same way that it’s useful to distinguish quantum mechanics from classical mechanics, even if they make the same predictions sometimes.
        2y ago
- Drakim 2y ago
  I guess I am both agreeing and disagreeing. The exact same problem is true for words in a book. Are the words in a lexicon true, false, or do they not have a truth value?
  - tines 2y ago
    The words in the book are true or false, making the author correct or incorrect. The question being posed is whether the output of an LLM has an "author," since it's not authored by a single human in the traditional sense. If so, the LLM is an agent of some kind; if not, it's not. If you're comparing the output of an LLM to a lexicon, you're agreeing with the person you originally replied to. He's arguing that an LLM is incapable of being true or false because of the manner in which its utterances are created, i.e. not by a mind.
    - williamcotton 2y ago
      So only a mind is capable of something making signs that are either true or false? Is a properly calibrated thermometer that reads "true" if the temperature is over 25C incapable of being true? But isn't this question ridiculous in the first place? Isn't a mind required to judge whether or not something is true, regardless of how this was signaled?
      - tines 2y ago
        Read again; I said he’s arguing that the LLM (i.e. thermometer in your example) is the thing that can’t be true or false. Its utterances (the readings of your thermometer) can be. This would be unlike a human, who can be right or wrong independently of an utterance, because they have a mind and beliefs.
        williamcotton 2y ago
        A human can be categorically wrong? Please explain. And of course a thing in and of itself, be it an apple, a dog or an LLM, can’t be true or false.
pessimizer 2y ago
> The real question is what I, the human, get out of reading those words, if I end up correctly navigating and understanding something about the world based on what I read. No, the real question is how you will end up correctly navigating and understanding something about the world from a falsehood crafted to be optimally harmonious with the truths that happen to surround it?
surfingdino 2y ago
Unfortunately, we do not want LLMs to tell us "all sorts of things," we want them to tell us the truth, to give us the facts. Happy to read how this is the wrong way to use LLMs, but then please stop shoving them into every facet of our lives, because whenever we talk about real-life applications of this tech it somehow is "not the right fit".
- stalked_why 2y ago
  [dead]
- basch 2y ago
  That's a round peg in a square hole. As ive seen them called elsewhere today, these "plausible text generators" can create a pseudo facsimile of reasoning, but they don't reason, and they don't fact check. Even when they use sources to build consensus, its more about volume than authoritativeness.
  - theturtletalks 2y ago
    I was watching the show, 3 Body Problem, and there was a great scene where a guy tells a woman to double check another man’s work. Then goes to the man and tells him to triple check the woman’s work. MoE seems to work this way, but maybe we can leverage different models that have different randomness and maybe we can get to a more logical answer. We have to start thinking about LLM hallucination differently. When it’s follows logic correctly and provides factual information, that is also a hallucination, but one that fits our flow of logic.
  - dougabug 2y ago
    Sure, but if we label the text as “factually accurate” or “logically sound” (or “unsound”) etc., then we can presumably greatly increase the probability of producing text with targeted properties
    - salawat 2y ago
      What on Earth makes you think that training a model on all factual information is going to do a lick in terms of generating factual outputs? At that point, clearly our only problem has been we've done it wrong all along by not training these things only on academic textbooks! That way we'll only probabilistically get true things out, right? /s
genman 2y ago
> What is the difference between a google map style application that shows pixels that are "right" for a road, and pixels that are "wrong" for a road? Different training methodology and objective and possibility to correct obviously wrong outcomes by comparing it against reality.
dougabug 2y ago
A pixel (in the context of an image) could be “wrong” in the sense that its assigned value could lead to an image that just looks like a bunch of noise. For instance, suppose we set every pixel in an image to some random value. The resulting would look like total noise and we humans wouldn’t recognized it as a sensible image. By providing a corpus of accepted images, we can train a model to generate images (arrays of pixels) which look like images and not, say, random noise. Now it could still generate an image of some place or person that doesn’t actually exist, so in that sense the pixels are collectively lying to you.

by Drakim 2y ago

deleted 2y ago

[deleted]

_boffin_ 2y ago

But then again, that road, according to a neighboring country who thinks that it’s their land and it isn’t a road. Depending on what country you’re in, it makes a difference

tines 2y ago

> The real question is what I, the human, get out of reading those words So then you agree with the OP that an LLM is not intelligent in the same way that Google Maps is not intelligent? That seems to be where your argument leads but you're replying in a way that makes me think you are disagreeing with the op.

contravariant 2y ago
Any reply will always sound like someone is disagreeing, even if they claim not to. Though in this case I'm not even sure what the comment they're supposedly disagreeing with is even claiming. Is it even claiming anything?
- tines 2y ago
  > Any reply will always sound like someone is disagreeing, even if they claim not to. Disagree! :) > Though in this case I'm not even sure what the comment they're supposedly disagreeing with is even claiming. Is it even claiming anything? It's offering support for the claim that LLMs hallucinate 100% of the time, even when their hallucinations happen to be true.
  - williamcotton 2y ago
    > It's offering support for the claim that LLMs hallucinate 100% of the time, even when their hallucinations happen to be true. Well this makes the term "hallucinate" completely useless for any sort of distinction. The word then becomes nothing more than a disparaging term for an LLM.
    - tines 2y ago
      Not really. It distinguishes LLM output from human output even though they look the same sometimes. The process by which something comes into existence is a valid distinction to make, even if the two processes happen to produce the same thing sometimes.
      - williamcotton 2y ago
        Why is it a valid distinction to make? For example, does this distinction affect the assessed truthfulness of a signal? Does process affect the artfulness of a painting? “My five year old could draw that?”
        tines 2y ago
        It makes sense to do so in the same way that it’s useful to distinguish quantum mechanics from classical mechanics, even if they make the same predictions sometimes.
        2y ago
Drakim 2y ago
I guess I am both agreeing and disagreeing. The exact same problem is true for words in a book. Are the words in a lexicon true, false, or do they not have a truth value?
- tines 2y ago
  The words in the book are true or false, making the author correct or incorrect. The question being posed is whether the output of an LLM has an "author," since it's not authored by a single human in the traditional sense. If so, the LLM is an agent of some kind; if not, it's not. If you're comparing the output of an LLM to a lexicon, you're agreeing with the person you originally replied to. He's arguing that an LLM is incapable of being true or false because of the manner in which its utterances are created, i.e. not by a mind.
  - williamcotton 2y ago
    So only a mind is capable of something making signs that are either true or false? Is a properly calibrated thermometer that reads "true" if the temperature is over 25C incapable of being true? But isn't this question ridiculous in the first place? Isn't a mind required to judge whether or not something is true, regardless of how this was signaled?
    - tines 2y ago
      Read again; I said he’s arguing that the LLM (i.e. thermometer in your example) is the thing that can’t be true or false. Its utterances (the readings of your thermometer) can be. This would be unlike a human, who can be right or wrong independently of an utterance, because they have a mind and beliefs.
      - williamcotton 2y ago
        A human can be categorically wrong? Please explain. And of course a thing in and of itself, be it an apple, a dog or an LLM, can’t be true or false.

pessimizer 2y ago

> The real question is what I, the human, get out of reading those words, if I end up correctly navigating and understanding something about the world based on what I read. No, the real question is how you will end up correctly navigating and understanding something about the world from a falsehood crafted to be optimally harmonious with the truths that happen to surround it?

surfingdino 2y ago

Unfortunately, we do not want LLMs to tell us "all sorts of things," we want them to tell us the truth, to give us the facts. Happy to read how this is the wrong way to use LLMs, but then please stop shoving them into every facet of our lives, because whenever we talk about real-life applications of this tech it somehow is "not the right fit".

stalked_why 2y ago
[dead]
basch 2y ago
That's a round peg in a square hole. As ive seen them called elsewhere today, these "plausible text generators" can create a pseudo facsimile of reasoning, but they don't reason, and they don't fact check. Even when they use sources to build consensus, its more about volume than authoritativeness.
- theturtletalks 2y ago
  I was watching the show, 3 Body Problem, and there was a great scene where a guy tells a woman to double check another man’s work. Then goes to the man and tells him to triple check the woman’s work. MoE seems to work this way, but maybe we can leverage different models that have different randomness and maybe we can get to a more logical answer. We have to start thinking about LLM hallucination differently. When it’s follows logic correctly and provides factual information, that is also a hallucination, but one that fits our flow of logic.
- dougabug 2y ago
  Sure, but if we label the text as “factually accurate” or “logically sound” (or “unsound”) etc., then we can presumably greatly increase the probability of producing text with targeted properties
  - salawat 2y ago
    What on Earth makes you think that training a model on all factual information is going to do a lick in terms of generating factual outputs? At that point, clearly our only problem has been we've done it wrong all along by not training these things only on academic textbooks! That way we'll only probabilistically get true things out, right? /s

genman 2y ago

> What is the difference between a google map style application that shows pixels that are "right" for a road, and pixels that are "wrong" for a road? Different training methodology and objective and possibility to correct obviously wrong outcomes by comparing it against reality.

dougabug 2y ago

A pixel (in the context of an image) could be “wrong” in the sense that its assigned value could lead to an image that just looks like a bunch of noise. For instance, suppose we set every pixel in an image to some random value. The resulting would look like total noise and we humans wouldn’t recognized it as a sensible image. By providing a corpus of accepted images, we can train a model to generate images (arrays of pixels) which look like images and not, say, random noise. Now it could still generate an image of some place or person that doesn’t actually exist, so in that sense the pixels are collectively lying to you.