3 ms·

ChatGPT (and other LLMs) still cannot do (and probably never will) well in any consistent manner in physics. I don't think physics departments are worrying much

ChatGPT (and other LLMs) still cannot do (and probably never will) well in any consistent manner in physics. I don't think physics departments are worrying much about the AI. The only thing that can help students in a more reliable way is some coding projects. Which is okay because in most of these classes (computational physics) students are encouraged to work together, seek help and even ask on the internet (before ChatGPT…etc.) It was always about how to explain and describe the thinking. AI (at least in its current form) is very weak at the problem-solving aspects and in understanding concepts. On the other hand, as non-native English speaker, it save me much time into paraphrasing my poorly thoughts and writing that I would need an hour to express in a good formal manner. It can guide you in some aspects of coding tasks, introduce you to some APIs …etc. This is actually a good tool that I agree that a good student (researcher) would use wisely to gain some knowledge and save sometime. It will not help much with solving a cart on an inclined plane with some friction and a pendulum hanging from the cart. No, it will not be able to give you the normal modes. This is just a personal experience and opinion, though. It might be completely different in other areas.

chaxor 3y ago
You should really think about making statements like "AI will probably never do X well". Many formal linguists made very strong statements about the impossibility of (__insert feature here__, such as pragmatic implicature) to be learned by AI, which they are now being shown to be wrong. For instance, Miles Cranmers work on using GNNs for symbolic regression is a start towards useful new discoveries in physics. Transformers are just GNNs with a specific message passing function and position embeddings. It's not hard to see that either by a different architecture, augmentation, or potentially even just more of the same, we can get to new discoveries in physics with AI. The GNN symbolic regression work is evidence that it's already happened. As for grounding knowledge in the LLMs we have exactly just this moment (a rather short-sighted view) there is plenty of interest and work in the area, for which I expect will be addressed in a multitude of ways. It's ability with grounded physics knowledge is not perfect, but it's very good w.r.t. the common knowledge of a human off the street. External sources alone make it much better, and that's just the exceedingly short-sighted analysis of what we have today.
witherk 3y ago
Really? Granted LLMs might be a little weaker in physics than other areas. If someone figures out get LLMs to use a mathmatica API, and train it some more I can imagine some rapid progress.
tipsytoad 3y ago
Really? The only reason that ChatGPT is more adept at coding problems is because there is vastly more training data. There's nothing fundamentally different between problem solving a coding problem and physics problem. Like all the others before it, I don't think this comment will age too well.

chaxor 3y ago

You should really think about making statements like "AI will probably never do X well". Many formal linguists made very strong statements about the impossibility of (__insert feature here__, such as pragmatic implicature) to be learned by AI, which they are now being shown to be wrong. For instance, Miles Cranmers work on using GNNs for symbolic regression is a start towards useful new discoveries in physics. Transformers are just GNNs with a specific message passing function and position embeddings. It's not hard to see that either by a different architecture, augmentation, or potentially even just more of the same, we can get to new discoveries in physics with AI. The GNN symbolic regression work is evidence that it's already happened. As for grounding knowledge in the LLMs we have exactly just this moment (a rather short-sighted view) there is plenty of interest and work in the area, for which I expect will be addressed in a multitude of ways. It's ability with grounded physics knowledge is not perfect, but it's very good w.r.t. the common knowledge of a human off the street. External sources alone make it much better, and that's just the exceedingly short-sighted analysis of what we have today.

witherk 3y ago

Really? Granted LLMs might be a little weaker in physics than other areas. If someone figures out get LLMs to use a mathmatica API, and train it some more I can imagine some rapid progress.

tipsytoad 3y ago

Really? The only reason that ChatGPT is more adept at coding problems is because there is vastly more training data. There's nothing fundamentally different between problem solving a coding problem and physics problem. Like all the others before it, I don't think this comment will age too well.