3 ms·

I'm also a trained physicist, spent some time in industry, and taught a college freshman math class for a semester a couple decades ago. There's an interpretat

I'm also a trained physicist, spent some time in industry, and taught a college freshman math class for a semester a couple decades ago. There's an interpretation that makes the reference answer correct. I didn't realize this until halfway through teaching the math class: Based on the lectures, textbook, and homework problems, you match the "form" of the problem with similar problems in the textbook, and apply the same algorithms to solve it. The students have some vague idea of this, referring to it as finding the "trick," but it's never explicitly explained to them. I was glad for the interesting puzzles and good grades. But physics really came alive for me in the lab, where mother nature decides the conditions of the problem being solved.

amluto 9d ago
> Based on the lectures, textbook, and homework problems, you match the "form" of the problem with similar problems in the textbook, and apply the same algorithms to solve it. Is the ultimate goal here to train an LLM to do well at mediocre physics homework or to train it to be a high-quality tool that can do real physics? And yes, I'm well aware that, even in elementary school, this form-matching is a thing. It's kind of sad.
- analog31 9d ago
  The odd thing is that we do train people to do real physics, somehow despite my cynical take. I see the education process as following heuristics that are believed to have some relationship to developing real world abilities, even if we don't know why. We all worked these problems, and now we're somehow able to do physics. In the end we don't know what turns people into physicists. Or musicians, artists, etc. An analogy is making students write 5-paragraph essays. We don't really believe that the knowledge of how to write a 5-paragraph essay is a real-world ability, and the LLM's can write them all day long. But for some reason we believed that 5-paragraph essays were a good pedagogical tool. But whether the same heuristics apply to teaching an LLM is anybody's guess. If they learn differently than we do, then feeding them on our learning tools isn't necessarily going to help them.

amluto 9d ago

> Based on the lectures, textbook, and homework problems, you match the "form" of the problem with similar problems in the textbook, and apply the same algorithms to solve it. Is the ultimate goal here to train an LLM to do well at mediocre physics homework or to train it to be a high-quality tool that can do real physics? And yes, I'm well aware that, even in elementary school, this form-matching is a thing. It's kind of sad.

analog31 9d ago
The odd thing is that we do train people to do real physics, somehow despite my cynical take. I see the education process as following heuristics that are believed to have some relationship to developing real world abilities, even if we don't know why. We all worked these problems, and now we're somehow able to do physics. In the end we don't know what turns people into physicists. Or musicians, artists, etc. An analogy is making students write 5-paragraph essays. We don't really believe that the knowledge of how to write a 5-paragraph essay is a real-world ability, and the LLM's can write them all day long. But for some reason we believed that 5-paragraph essays were a good pedagogical tool. But whether the same heuristics apply to teaching an LLM is anybody's guess. If they learn differently than we do, then feeding them on our learning tools isn't necessarily going to help them.