8 ms·

This is why "critical thinking" is often promoted in education. It's the idea that it's better to teach students "how to discern the truth" than teach them "th

This is why "critical thinking" is often promoted in education. It's the idea that it's better to teach students "how to discern the truth" than teach them "the truth". Is it possible to teach students critical thinking? Is it easier or harder to teach AI critical thinking? Curious to hear your thoughts.

lukev 3y ago
I don't know if it can be taught critical thinking. LLM token-generation is only vaguely analogous to "thinking" at all. Recursive self prompting with both internal and external calibration might be analogous to "critical thinking" though.
- ddmma 3y ago
  Worth trying to show some examples of critical thinking to LLM and then see what happens on new set of data points
  - lukev 3y ago
    This definitely works. There's lots of papers on this. Heck, you can verify this with ChatGPT yourself... just ask it a couple questions, then (in a separate thread) ask the same questions but tell it to "think step by step" as an instruction. It performs visibly better. The point is it fundamentally, at the end of the day, it's generating one token at a time with only its prompt and previously generated tokens for input. If you can inject other information/facts into the context, or even context from another model instance that have a different "train of thought", you can in theory get better results than the model itself generating a sequence. Look up Chain of Thought / Tree of Thought / Graph of Thought for an intro, as well as basically anything on retrieval augmented generation (RAG) or LLM tool usage.
    - ddmma 3y ago
      Thanks for pointing out the evaluation methods and actually being doing some metrics on the outcome from LLMs myself. I bet more on now adjusted knowledge on top of generated content based on facts/ nonsense scoring that can be harvested for feeding reasoning graphs with kind of auto balancing. Less resources and high impact will make the difference on long term.
WalterBright 3y ago
> Is it possible to teach students critical thinking? I doubt it. Universities claim to teach it, but I don't see it in the graduates. Consider designing airplanes. (I.e. engineering.) If the result doesn't fly, no amount of rhetoric will convince anyone otherwise. This is not true of the humanities, where educated people can and do convince themselves of endless bad ideas. I'll take the reality checks of engineering over "critical thinking".
- maxbond 3y ago
  Once I was riding a train, and the conductor announced that only some cars would open their doors at the upcoming station, because the train was longer than the platform. "I wonder why?" A stranger mused. "I know," her companion sneered, "it's because some engineer designed it." She wasn't wrong per say, an engineer had designed the platform - a hundred years ago, for a different train. The train we were riding had squeeked through the legislature on a shoestring budget, reusing the existing tracks and with only a few new platforms slated to be built. And many of those were axed along the way due to budget overruns and such. It was a political miracle the train existed at all. But this passenger wasn't really looking for context. She had stereotypes and narratives that explained the situation to her satisfaction. Are you quite sure that you are engaging your faculties of critical thinking here, rather than relying on narratives and stereotypes? Does it really seem likely to you that the people doing the work you're most familiar with are the only people capable of thinking critically? Or that the feedback loops that you've had in your career are the only ones available?
  - WalterBright 3y ago
    Let me put it this way. Marxism doesn't fly, and nobody has yet found an example of it flying. But academics continue to push Marxism. That's a massive failure of critical thinking by the people who teach it. As for engineers, they make mistakes all the time. And the mistakes are very hard to talk one's way out of. Our society is built by engineers making things that work. Your car doesn't move by talking it into moving.
    - 3y ago

lukev 3y ago

I don't know if it can be taught critical thinking. LLM token-generation is only vaguely analogous to "thinking" at all. Recursive self prompting with both internal and external calibration might be analogous to "critical thinking" though.

ddmma 3y ago
Worth trying to show some examples of critical thinking to LLM and then see what happens on new set of data points
- lukev 3y ago
  This definitely works. There's lots of papers on this. Heck, you can verify this with ChatGPT yourself... just ask it a couple questions, then (in a separate thread) ask the same questions but tell it to "think step by step" as an instruction. It performs visibly better. The point is it fundamentally, at the end of the day, it's generating one token at a time with only its prompt and previously generated tokens for input. If you can inject other information/facts into the context, or even context from another model instance that have a different "train of thought", you can in theory get better results than the model itself generating a sequence. Look up Chain of Thought / Tree of Thought / Graph of Thought for an intro, as well as basically anything on retrieval augmented generation (RAG) or LLM tool usage.
  - ddmma 3y ago
    Thanks for pointing out the evaluation methods and actually being doing some metrics on the outcome from LLMs myself. I bet more on now adjusted knowledge on top of generated content based on facts/ nonsense scoring that can be harvested for feeding reasoning graphs with kind of auto balancing. Less resources and high impact will make the difference on long term.

WalterBright 3y ago

> Is it possible to teach students critical thinking? I doubt it. Universities claim to teach it, but I don't see it in the graduates. Consider designing airplanes. (I.e. engineering.) If the result doesn't fly, no amount of rhetoric will convince anyone otherwise. This is not true of the humanities, where educated people can and do convince themselves of endless bad ideas. I'll take the reality checks of engineering over "critical thinking".

maxbond 3y ago
Once I was riding a train, and the conductor announced that only some cars would open their doors at the upcoming station, because the train was longer than the platform. "I wonder why?" A stranger mused. "I know," her companion sneered, "it's because some engineer designed it." She wasn't wrong per say, an engineer had designed the platform - a hundred years ago, for a different train. The train we were riding had squeeked through the legislature on a shoestring budget, reusing the existing tracks and with only a few new platforms slated to be built. And many of those were axed along the way due to budget overruns and such. It was a political miracle the train existed at all. But this passenger wasn't really looking for context. She had stereotypes and narratives that explained the situation to her satisfaction. Are you quite sure that you are engaging your faculties of critical thinking here, rather than relying on narratives and stereotypes? Does it really seem likely to you that the people doing the work you're most familiar with are the only people capable of thinking critically? Or that the feedback loops that you've had in your career are the only ones available?
- WalterBright 3y ago
  Let me put it this way. Marxism doesn't fly, and nobody has yet found an example of it flying. But academics continue to push Marxism. That's a massive failure of critical thinking by the people who teach it. As for engineers, they make mistakes all the time. And the mistakes are very hard to talk one's way out of. Our society is built by engineers making things that work. Your car doesn't move by talking it into moving.
  - 3y ago