4 ms·

Tangent: This is often true of humans as well. We often make a decision based on a gut feeling, and then backfill a logical reason supporting our feeling, with

by kentonv 2mo ago

Tangent: This is often true of humans as well. We often make a decision based on a gut feeling, and then backfill a logical reason supporting our feeling, without even realizing we're doing it -- rationalization.

angry_octet 2mo ago
And we all know some people that rationalise poor choices and misbehavior, hide their mistakes, etc, to an unacceptable degree. Sometimes the individual knows they are rationalising but continues anyway, other times they seem incapable of seeing that. When you ask people who are rationalising poor behaviour about the scenario, but it is someone else doing it, they may arrive at a better answer. Can we use multiple LLMs to achieve self criticism and critical thinking?
- efilife 2mo ago
  isn't this happening already? There's the concept called "thinking" where the models talks with itself before giving you the final answer
  - angry_octet 2mo ago
    It does multiple rounds of feedback which is called 'thinking' but whether that is critical thinking is unknown.
glenstein 2mo ago
Tangent on the tangent: I think that's true in a minority of cases and in a majority of AI cases. Though in principle I think it should be possible for an LLM to have access to and faithfully represent its own reasoning.
- margalabargala 2mo ago
  On the contrary, I would argue that it's true in a totality of AI cases. To your point, I agree that nominally there should be a way to give conceptual names to paths of weights, and when answering a question, notice which weights were and were not applied and retrospect on that. That's not what reasoning traces as they currently exist are, though.
5-0 2mo ago
Yes, humans in general. Scientists, perhaps less so. Is it unreasonable to hold LLMs to a higher standard?
- kentonv 2mo ago
  > Scientists, perhaps less so. I disagree. In fact, I find people who think of themselves as highly rational seem to be particularly prone to rationalization. Because being rational is integral to their whole identity, they are much more eager to accept their own rationalizations as sound and resist admitting that they are guided by instinct.

by kentonv 2mo ago

angry_octet 2mo ago

And we all know some people that rationalise poor choices and misbehavior, hide their mistakes, etc, to an unacceptable degree. Sometimes the individual knows they are rationalising but continues anyway, other times they seem incapable of seeing that. When you ask people who are rationalising poor behaviour about the scenario, but it is someone else doing it, they may arrive at a better answer. Can we use multiple LLMs to achieve self criticism and critical thinking?

efilife 2mo ago
isn't this happening already? There's the concept called "thinking" where the models talks with itself before giving you the final answer
- angry_octet 2mo ago
  It does multiple rounds of feedback which is called 'thinking' but whether that is critical thinking is unknown.

glenstein 2mo ago

Tangent on the tangent: I think that's true in a minority of cases and in a majority of AI cases. Though in principle I think it should be possible for an LLM to have access to and faithfully represent its own reasoning.

margalabargala 2mo ago
On the contrary, I would argue that it's true in a totality of AI cases. To your point, I agree that nominally there should be a way to give conceptual names to paths of weights, and when answering a question, notice which weights were and were not applied and retrospect on that. That's not what reasoning traces as they currently exist are, though.

5-0 2mo ago

Yes, humans in general. Scientists, perhaps less so. Is it unreasonable to hold LLMs to a higher standard?

kentonv 2mo ago
> Scientists, perhaps less so. I disagree. In fact, I find people who think of themselves as highly rational seem to be particularly prone to rationalization. Because being rational is integral to their whole identity, they are much more eager to accept their own rationalizations as sound and resist admitting that they are guided by instinct.