2 ms·

/Some/ people bullshit themselves stating the plausible; others check their hypotheses. The difference is total in both humans and automated processes.

by mdp2021 2y ago

stavros 2y ago
How are you going to check your hypotheses for why you preferred that jacket to that other jacket?
- deleted 2y ago
  [deleted]
- DSingularity 2y ago
  Is that example representative for the LLM tasks for which we seek explainability ?
  - stavros 2y ago
    Are we holding LLMs to a higher standard than people?
    - f_devd 2y ago
      Ideally yes, LLMs are tools that we expect to work, people are inherently fallible and (even unintentionally) deceptive. LLMs being human-like in this specific way is not desirable.
      - stavros 2y ago
        Then I think you'll be very disappointed. LLMs aren't in the same category as calculators, for example.
        f_devd 2y ago
        I have no illusions on LLMs, I have been working with them since og BERT, always with these same issues and more. I'm just stating what would be needed in the future to make them reliably useful outside of creative writing & (human-guided & checked) search. If an LLM provides an incorrect/orthogonal rhetoric without a way to reliably fix/debug it it's just not as useful as it theoretically could be given the data contained in the parameters.
- mdp2021 2y ago
  Do not lose the original point: some systems have a goal to sound plausible, while some have a goal to say the truth. Some systems, when asked "where have you been", will reply "at the baker's" because it is a nice narrative in their "novel writing, re-writing of reality", some other will check memory and say "at the butcher's", where they have actually been. When people invent explicit reasons on why they turned left or right, those reasons remain hypotheses. The clumsy will promote those hypotheses to beliefs. The apt will keep the spontaneous ideas as hypotheses, until the ability to assess them comes.
  - famouswaffles 2y ago
    Everybody promotes these sorts of hypotheses to beliefs because it's not a conscious decision you are aware of. It's not about being clumsy or apt. You don't have much control over it. https://pmc.ncbi.nlm.nih.gov/articles/PMC3196841/ https://pmc.ncbi.nlm.nih.gov/articles/PMC3196841/ https://pure.uva.nl/ws/files/25987577/Split_Brain.pdf https://pure.uva.nl/ws/files/25987577/Split_Brain.pdf
    - mdp2021 2y ago
      It does not matter, that there may be a tendency towards bad thinking: what matters is the possibility of proper thinking and the training towards it (becoming more and more proficient at it and practicing it constantly, having it as your natural state; in automation, implementing it in the process). What you control is the intentional revision of thought. (I am acquainted with earlier studies about the corpus callosum but I do not know why you would mention that, what it would prove: maybe you could be clearer? I do not see how it could affect the notion of critical thinking.)
      - famouswaffles 2y ago
        I've explained it the best i can in the other comment. But you keep making the mistake that this is just a culprit of 'bad thinking' or 'intentional revision of thought' and while i'm not saying those things don't exist, It's not. Not only are the rationalizations i'm talking about and which some of these papers allude to not intentional, they often happen without your conscious awareness.
catskul2 2y ago
Everyone, every last one of us, does this every single day, all day, and only occasionally do we deviate to check ourselves, and often then it's to save face. A Nobel prize was given for related research to Daniel Kahneman. If you think it doesn't apply to you, you're definitely wrong.
- mdp2021 2y ago
  > occasionally Properly educated people do it regularly, not occasionally. You are describing a definite set of people. No, it does not cover all. Some people will output a pre-given answer; some people check. Edit: sniper... Find some argument.
  - famouswaffles 2y ago
    Your decisions shape your preferences just as much as your preferences shape your decisions and you're not even aware of it. Yes, everybody regularly confabulates plausible sounding things that they themselves genuinely believe to be the 'real reason'. You're not immune or special. https://pmc.ncbi.nlm.nih.gov/articles/PMC3196841/ https://pmc.ncbi.nlm.nih.gov/articles/PMC3196841/
    - mdp2021 2y ago
      I will check the article with more attention as soon as I will have the time, but: putting aside a question on how would a similar investigation prove that all people would function in the same way, that does not seem to counter that some people «check their hypotheses» - as duly. Some people do exercise critical thinking. It is an intentional process.
      - famouswaffles 2y ago
        You're not getting it. You ask A "Why did you choose that?" > He answers "I like the color blue" This makes sense. This is what everyone thinks and believes is the actual sequence of such events. But often, this is the actual sequence "Let's go with this" > "Now i like the color blue" 'A' didn't lie to you or try to trick you. He didn't consciously rationalize liking blue after the fact. He's not stupid or "prone to bad thinking". Altering your perceptions of events without your conscious awareness is just simply something that your brain does fairly regularly. Make no mistake. A genuinely likes blue now - the only difference is that he genuinely believes he made the choice because he liked blue instead of the brain having the tendency to make you favor your choices and giving him the like of blue so it sits better. This is not something you "check your hypotheses" out of. And it's something every human deals with everyday, including you.

/Some/ people bullshit themselves stating the plausible; others check their hypotheses. The difference is total in both humans and automated processes.

stavros 2y ago

How are you going to check your hypotheses for why you preferred that jacket to that other jacket?

deleted 2y ago
[deleted]
DSingularity 2y ago
Is that example representative for the LLM tasks for which we seek explainability ?
- stavros 2y ago
  Are we holding LLMs to a higher standard than people?
  - f_devd 2y ago
    Ideally yes, LLMs are tools that we expect to work, people are inherently fallible and (even unintentionally) deceptive. LLMs being human-like in this specific way is not desirable.
    - stavros 2y ago
      Then I think you'll be very disappointed. LLMs aren't in the same category as calculators, for example.
      - f_devd 2y ago
        I have no illusions on LLMs, I have been working with them since og BERT, always with these same issues and more. I'm just stating what would be needed in the future to make them reliably useful outside of creative writing & (human-guided & checked) search. If an LLM provides an incorrect/orthogonal rhetoric without a way to reliably fix/debug it it's just not as useful as it theoretically could be given the data contained in the parameters.
mdp2021 2y ago
Do not lose the original point: some systems have a goal to sound plausible, while some have a goal to say the truth. Some systems, when asked "where have you been", will reply "at the baker's" because it is a nice narrative in their "novel writing, re-writing of reality", some other will check memory and say "at the butcher's", where they have actually been. When people invent explicit reasons on why they turned left or right, those reasons remain hypotheses. The clumsy will promote those hypotheses to beliefs. The apt will keep the spontaneous ideas as hypotheses, until the ability to assess them comes.
- famouswaffles 2y ago
  Everybody promotes these sorts of hypotheses to beliefs because it's not a conscious decision you are aware of. It's not about being clumsy or apt. You don't have much control over it. https://pmc.ncbi.nlm.nih.gov/articles/PMC3196841/ https://pmc.ncbi.nlm.nih.gov/articles/PMC3196841/ https://pure.uva.nl/ws/files/25987577/Split_Brain.pdf https://pure.uva.nl/ws/files/25987577/Split_Brain.pdf
  - mdp2021 2y ago
    It does not matter, that there may be a tendency towards bad thinking: what matters is the possibility of proper thinking and the training towards it (becoming more and more proficient at it and practicing it constantly, having it as your natural state; in automation, implementing it in the process). What you control is the intentional revision of thought. (I am acquainted with earlier studies about the corpus callosum but I do not know why you would mention that, what it would prove: maybe you could be clearer? I do not see how it could affect the notion of critical thinking.)
    - famouswaffles 2y ago
      I've explained it the best i can in the other comment. But you keep making the mistake that this is just a culprit of 'bad thinking' or 'intentional revision of thought' and while i'm not saying those things don't exist, It's not. Not only are the rationalizations i'm talking about and which some of these papers allude to not intentional, they often happen without your conscious awareness.

catskul2 2y ago

Everyone, every last one of us, does this every single day, all day, and only occasionally do we deviate to check ourselves, and often then it's to save face. A Nobel prize was given for related research to Daniel Kahneman. If you think it doesn't apply to you, you're definitely wrong.

mdp2021 2y ago
> occasionally Properly educated people do it regularly, not occasionally. You are describing a definite set of people. No, it does not cover all. Some people will output a pre-given answer; some people check. Edit: sniper... Find some argument.
- famouswaffles 2y ago
  Your decisions shape your preferences just as much as your preferences shape your decisions and you're not even aware of it. Yes, everybody regularly confabulates plausible sounding things that they themselves genuinely believe to be the 'real reason'. You're not immune or special. https://pmc.ncbi.nlm.nih.gov/articles/PMC3196841/ https://pmc.ncbi.nlm.nih.gov/articles/PMC3196841/
  - mdp2021 2y ago
    I will check the article with more attention as soon as I will have the time, but: putting aside a question on how would a similar investigation prove that all people would function in the same way, that does not seem to counter that some people «check their hypotheses» - as duly. Some people do exercise critical thinking. It is an intentional process.
    - famouswaffles 2y ago
      You're not getting it. You ask A "Why did you choose that?" > He answers "I like the color blue" This makes sense. This is what everyone thinks and believes is the actual sequence of such events. But often, this is the actual sequence "Let's go with this" > "Now i like the color blue" 'A' didn't lie to you or try to trick you. He didn't consciously rationalize liking blue after the fact. He's not stupid or "prone to bad thinking". Altering your perceptions of events without your conscious awareness is just simply something that your brain does fairly regularly. Make no mistake. A genuinely likes blue now - the only difference is that he genuinely believes he made the choice because he liked blue instead of the brain having the tendency to make you favor your choices and giving him the like of blue so it sits better. This is not something you "check your hypotheses" out of. And it's something every human deals with everyday, including you.