3 ms·

> humans aren't "optimized" to be correct either Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when c

by pdonis 3y ago

> humans aren't "optimized" to be correct either Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when communicating. LLMs don't even have a concept of "being correct". That would require having a concept of an "external world" that text refers to, which LLMs don't have. All they have is the text in their training data.

famouswaffles 3y ago
>LLMs don't even have a concept of "being correct" They clearly do as plenty research indicates. You've just decided not to accept this. Pure confirmation bias in action.
- pdonis 3y ago
  > They clearly do as plenty research indicates. I have seen plenty of researchers claim this. What I have not seen is actual support for such a claim.
  - famouswaffles 3y ago
    There's plenty support. You just need to know how to read.
    - yladiz 3y ago
      Please link to published papers that prove what you’re positing.
      - happytiger 3y ago
        To tack on, I would love to read more research on this topic if there is relevant research to read. I have not read much compelling evidence though I have seen a lot of researchers conjecturing and some inappropriately reaching (and anthropomorphising the living hell out of ai in the process sometimes). There’s a lot to learn and I don’t think anyone can keep up with all the papers coming out so let’s keep things positive and work together to learn as much as we can og_kalu. Thanks for letting me tag this comment on the end of thread. This is HN. We’re all nerds here anyways and nobody can keep up with everything in a field moving as fast at AI is these days. :)
        ckyngcybra 3y ago
        [flagged]
  - jshen 3y ago
    I'd ask this question the other way. What research indicates that humans care primarily, or even to a high degree, about being objectively right versus deluding ourselves into believing we're right regardless of whether we are or not? I've seen a lot of evidence for the latter, but I'm not familiar with any for the former.
nuancebydefault 3y ago
Humans do not _know_ the external world either. All humans have is perceptions from media like sounds, images and text! Internally we try to make a model of the world based on those perceptions. Any blanks are filled in by creativity ! Think about it, when we write or speak we start from some internalized model (which is mostly imperfect) and then we build text around it that feels fluent and consistent. I would argue, human reasoning is conceptually not so much different from LLMs as one might think.
- pdonis 3y ago
  > Humans do not _know_ the external world either. All humans have is perceptions Wrong. Human beings can act on the world as well as perceive it, and humans have rich internal models of the world that get updated continously as a result of those rich two-way interactions. > I would argue, human reasoning is conceptually not so much different from LLMs as one might think. I would argue that this is either an incredible over-estimate of LLMs, or an incredibly impoverished view of humans.
  - nuancebydefault 3y ago
    > Human beings can act on the world as well as perceive it Conceptually this is the same for LLMs. Input/perceive -> human/LLM -> output/act. You might say the acting part for LLMs is not autonomous but that is only dependent on how much agency is given to LLMs. LLMs outputs are massively used, hence their outputs lead to actions. > and humans have rich internal models of the world that get updated continously as a result of those rich two-way interactions. What is different from LLMs: rich, continuous. I would argue these are quantative and qualitative traits that fall outside of what I meant with "conceptually".
    - pdonis 3y ago
      > Conceptually this is the same for LLMs. No, it isn't. LLMs don't look at the results of their actions and update a world model. Humans do.
happytiger 3y ago
I love this point. One of the things that has been on my mind of late is the idea of how much of what humans call “correct” or “right” is actually socially generated pressures to conform. Berns and Asch’s various research on social conformity around what we normally would consider objectively correct answers seems to have some potential applications worth testing in AI land. AIs may be much more performant with AI “social inputs” (and of course all the long term memory improvements currently lacking required to use it as a tool). I am very much excited about the network AI and pipeline work (small teams!) currently coming into vogue. Feels like the “right direction” to make incredible progress. https://www.psychologytoday.com/us/blog/am-i-right/201404/the-astonishing-power-social-pressure https://www.psychologytoday.com/us/blog/am-i-right/201404/th... Otherwise agree with your points exactly. Good comment.
- pdonis 3y ago
  > One of the things that has been on my mind of late is the idea of how much of what humans call “correct” or “right” is actually socially generated pressures to conform And the only reason that is even a thing is that we humans have a concept of actually being right, which we can distinguish from "responding to socially generated pressures to conform".
  - jshen 3y ago
    I'm not sure I'd frame it that way. We seem to make decisions with our subconscious brains and our "rational" brains primarily rationalize a justification for that decision. Our brains are less concerned with being right than they are with justifying our subconscious behavior and fitting in with our tribe. Another example, our brains care far more about achieving and maintaining status than about being right. We seem barely capable of objective thought at best.
    - pdonis 3y ago
      > We seem to make decisions with our subconscious brains and our "rational" brains primarily rationalize a justification for that decision. There is research that is claimed to show this, but it doesn't. All it actually shows is that the process of making a decision takes time, and involves many different parts of the brain. There is no separate "rationalization" that comes after the decision is made. It's all part of a single process that can't be decomposed into such separate parts. In any case, all this is irrelevant to what I said. I said we humans have the concept of "actually being right". We don't always live up to that concept, but even making that observation, as you have, shows that we have the concept. If we didn't, we could not even make statements like this one of yours: "our brains care far more about achieving and maintaining status than about being right." > We seem barely capable of objective thought at best. I think you have a sadly impoverished view of human thought.

by pdonis 3y ago

famouswaffles 3y ago

>LLMs don't even have a concept of "being correct" They clearly do as plenty research indicates. You've just decided not to accept this. Pure confirmation bias in action.

pdonis 3y ago
> They clearly do as plenty research indicates. I have seen plenty of researchers claim this. What I have not seen is actual support for such a claim.
- famouswaffles 3y ago
  There's plenty support. You just need to know how to read.
  - yladiz 3y ago
    Please link to published papers that prove what you’re positing.
    - happytiger 3y ago
      To tack on, I would love to read more research on this topic if there is relevant research to read. I have not read much compelling evidence though I have seen a lot of researchers conjecturing and some inappropriately reaching (and anthropomorphising the living hell out of ai in the process sometimes). There’s a lot to learn and I don’t think anyone can keep up with all the papers coming out so let’s keep things positive and work together to learn as much as we can og_kalu. Thanks for letting me tag this comment on the end of thread. This is HN. We’re all nerds here anyways and nobody can keep up with everything in a field moving as fast at AI is these days. :)
      - ckyngcybra 3y ago
        [flagged]
- jshen 3y ago
  I'd ask this question the other way. What research indicates that humans care primarily, or even to a high degree, about being objectively right versus deluding ourselves into believing we're right regardless of whether we are or not? I've seen a lot of evidence for the latter, but I'm not familiar with any for the former.

nuancebydefault 3y ago

Humans do not _know_ the external world either. All humans have is perceptions from media like sounds, images and text! Internally we try to make a model of the world based on those perceptions. Any blanks are filled in by creativity ! Think about it, when we write or speak we start from some internalized model (which is mostly imperfect) and then we build text around it that feels fluent and consistent. I would argue, human reasoning is conceptually not so much different from LLMs as one might think.

pdonis 3y ago
> Humans do not _know_ the external world either. All humans have is perceptions Wrong. Human beings can act on the world as well as perceive it, and humans have rich internal models of the world that get updated continously as a result of those rich two-way interactions. > I would argue, human reasoning is conceptually not so much different from LLMs as one might think. I would argue that this is either an incredible over-estimate of LLMs, or an incredibly impoverished view of humans.
- nuancebydefault 3y ago
  > Human beings can act on the world as well as perceive it Conceptually this is the same for LLMs. Input/perceive -> human/LLM -> output/act. You might say the acting part for LLMs is not autonomous but that is only dependent on how much agency is given to LLMs. LLMs outputs are massively used, hence their outputs lead to actions. > and humans have rich internal models of the world that get updated continously as a result of those rich two-way interactions. What is different from LLMs: rich, continuous. I would argue these are quantative and qualitative traits that fall outside of what I meant with "conceptually".
  - pdonis 3y ago
    > Conceptually this is the same for LLMs. No, it isn't. LLMs don't look at the results of their actions and update a world model. Humans do.

happytiger 3y ago

I love this point. One of the things that has been on my mind of late is the idea of how much of what humans call “correct” or “right” is actually socially generated pressures to conform. Berns and Asch’s various research on social conformity around what we normally would consider objectively correct answers seems to have some potential applications worth testing in AI land. AIs may be much more performant with AI “social inputs” (and of course all the long term memory improvements currently lacking required to use it as a tool). I am very much excited about the network AI and pipeline work (small teams!) currently coming into vogue. Feels like the “right direction” to make incredible progress. https://www.psychologytoday.com/us/blog/am-i-right/201404/the-astonishing-power-social-pressure https://www.psychologytoday.com/us/blog/am-i-right/201404/th... Otherwise agree with your points exactly. Good comment.

pdonis 3y ago
> One of the things that has been on my mind of late is the idea of how much of what humans call “correct” or “right” is actually socially generated pressures to conform And the only reason that is even a thing is that we humans have a concept of actually being right, which we can distinguish from "responding to socially generated pressures to conform".
- jshen 3y ago
  I'm not sure I'd frame it that way. We seem to make decisions with our subconscious brains and our "rational" brains primarily rationalize a justification for that decision. Our brains are less concerned with being right than they are with justifying our subconscious behavior and fitting in with our tribe. Another example, our brains care far more about achieving and maintaining status than about being right. We seem barely capable of objective thought at best.
  - pdonis 3y ago
    > We seem to make decisions with our subconscious brains and our "rational" brains primarily rationalize a justification for that decision. There is research that is claimed to show this, but it doesn't. All it actually shows is that the process of making a decision takes time, and involves many different parts of the brain. There is no separate "rationalization" that comes after the decision is made. It's all part of a single process that can't be decomposed into such separate parts. In any case, all this is irrelevant to what I said. I said we humans have the concept of "actually being right". We don't always live up to that concept, but even making that observation, as you have, shows that we have the concept. If we didn't, we could not even make statements like this one of yours: "our brains care far more about achieving and maintaining status than about being right." > We seem barely capable of objective thought at best. I think you have a sadly impoverished view of human thought.