3 ms·

So, I'm a professor, and I have, um, really strong opinions about this. :-) Perhaps too strong and long for the current forum. But I'll see if I can be brief.

by dgacmu 2y ago

So, I'm a professor, and I have, um, really strong opinions about this. :-) Perhaps too strong and long for the current forum. But I'll see if I can be brief. ChatGPT is really, really good at providing solid answers of varying levels of detail and complexity to hyper-common questions such as those used in problem sets. This is one part of the skill set of a tutor, and it's a valuable one. When I interview TAs for my classes, however, I actually put a lot more emphasis on a different skill: The ability to get into a student's head and understand where their conceptual difficulty or misunderstanding is. This is a very different skill, and it's one that ChatGPT isn't as good at, because we've gone from "maximum likelihood answers from questions that are in the middle of the distribution" into a wide range of possible sources of confusion, which the student may lack the words to explain in a precise way. In the case of my kid, the PV=nRT question manifested as "I don't get it!" (with more exclamation points). Asking ChatGPT (well, copilot, since I have institutional access to that, but it uses ChatGPT) to help understand the problem: It digressed and introduced Boyle's Law, threw in a new symbol "I" (ok, the 12 year old) had never seen for "proportional to", and ... in some sense just added to the cognitive overload. The human approach was to ask a question: Have you ever seen this equation before? (No) Oh! Well, let's talk a little about gases.. Now, responding to ChatGPT and asking "No, that didn't help. Please ELI5 instead?" actually produced a much better answer: An analogy using a balloon. Which, amusingly, is exactly how I explained the behavior of gases to her. But even here, there's a bit of a difference: In explaining it to her, I did so socratically: "Ok, so imagine a balloon. If you heat the air inside the balloon, what happens?" "Um, it gets bigger, right?" "Yup, ..." (and now, knowing that she got that part, we could go on...) That's something you can absolutely imagine trying to program around an LLM, but it's not a native way of interacting with it. So ... I'd instead be a little more cautious here and say that ChatGPT potentially provides a really useful piece of what a human tutor offers, but it loses on the interactive exchange that helps much more rapidly zoom in on the source of confusion and correct it. Assuming that it's right. I think that for a particularly sophisticated consumer, it can be more valuable, but it requires knowing what you don't know, in some sense: The ability to already isolate what you're confused by. Once you know the question to ask, ChatGPT can often provide it -- again, assuming that some quirk of its training or your phrasing doesn't cause it to generate an answer that's wrong in some way.

CamperBob2 2y ago
I wonder if you are getting very different results with Copilot relative to what I've seen lately from ChatGPT4o. It did a pretty good job with the successive "Explain Galois theory like I am 5/16/133" prompts that I tried earlier. Seems likely that the chat system prompt is different from Copilot's, but who knows how. I will say that I have asked it to solve gas-law problems before, back when the base ChatGPT4 model was in use, and it didn't do so well there. It came up with the correct logic but I found it hard to get it to use consistent units, so the numbers kept coming out wrong. In general I find it super useful at improving my conceptual understanding of things -- it's at least as good as a good human tutor, at levels up to and including fairly-advanced signal processing topics -- but the details are less reliable. Feeding the same question to Gemini is a good way to double-check the numbers in my experience. If they agree, the numbers are probably right, while if they don't, then it's likely they're both wrong.
- silasdavis 2y ago
  Can you ask it to explain galois theory like he's 21?
  - CamperBob2 2y ago
    ---------------- "Let’s imagine Évariste Galois survived that fateful duel and continued his work. What would he have done next? If Galois had lived past 21, he would have likely expanded and deepened his revolutionary ideas. By that time, he had already laid down the foundation of what we now call Galois theory, but there were still many directions he could have explored further. Generalization of Galois Theory: Galois had just scratched the surface of the powerful connection between field theory and group theory. His next logical step might have been to generalize his ideas to more complex situations, such as understanding the symmetries of solutions to equations in multiple variables or extending his work to non-polynomial equations. Exploration of Solvable Groups: Galois introduced the concept of solvable groups to determine whether a polynomial equation could be solved by radicals (using addition, multiplication, and roots). If he had lived longer, he would likely have delved deeper into classifying different types of groups and exploring their properties. This could have led to a more detailed theory of group structures, potentially anticipating parts of what we now know as modern group theory. Linking Galois Theory with Other Areas: Galois might have explored connections between his theory and other areas of mathematics, such as geometry, number theory, or even early versions of what we now call topology. For instance, he could have investigated how the symmetries of geometric shapes relate to the symmetries of algebraic equations. Developing Further Applications: Beyond theory, Galois could have sought applications of his work in more practical areas. Perhaps he would have applied his ideas to solving problems in physics, mechanics, or other emerging fields, where symmetry plays a crucial role. Inspiring the Next Generation: If Galois had lived, he might have become a prominent teacher and mentor, influencing a new generation of mathematicians. His radical ideas would have challenged and inspired others to push the boundaries of mathematics even further. Galois could have spearheaded a new movement in algebra, similar to how Einstein revolutionized physics. Refining His Work: Galois' original manuscripts were hastily written under dire circumstances. Given more time, he would likely have polished and clarified his ideas, making them more accessible to other mathematicians of the time. This might have sped up the acceptance and expansion of his ideas in the mathematical community." ---------------- I'd give it a C for effort, I guess. I don't know enough about the topic to ask it for more specific predictions. At the end of the day it can only work with what it was trained with, so any genuine insights it comes up with will be due more to accident than inspiration.

by dgacmu 2y ago

CamperBob2 2y ago

I wonder if you are getting very different results with Copilot relative to what I've seen lately from ChatGPT4o. It did a pretty good job with the successive "Explain Galois theory like I am 5/16/133" prompts that I tried earlier. Seems likely that the chat system prompt is different from Copilot's, but who knows how. I will say that I have asked it to solve gas-law problems before, back when the base ChatGPT4 model was in use, and it didn't do so well there. It came up with the correct logic but I found it hard to get it to use consistent units, so the numbers kept coming out wrong. In general I find it super useful at improving my conceptual understanding of things -- it's at least as good as a good human tutor, at levels up to and including fairly-advanced signal processing topics -- but the details are less reliable. Feeding the same question to Gemini is a good way to double-check the numbers in my experience. If they agree, the numbers are probably right, while if they don't, then it's likely they're both wrong.

silasdavis 2y ago
Can you ask it to explain galois theory like he's 21?
- CamperBob2 2y ago
  ---------------- "Let’s imagine Évariste Galois survived that fateful duel and continued his work. What would he have done next? If Galois had lived past 21, he would have likely expanded and deepened his revolutionary ideas. By that time, he had already laid down the foundation of what we now call Galois theory, but there were still many directions he could have explored further. Generalization of Galois Theory: Galois had just scratched the surface of the powerful connection between field theory and group theory. His next logical step might have been to generalize his ideas to more complex situations, such as understanding the symmetries of solutions to equations in multiple variables or extending his work to non-polynomial equations. Exploration of Solvable Groups: Galois introduced the concept of solvable groups to determine whether a polynomial equation could be solved by radicals (using addition, multiplication, and roots). If he had lived longer, he would likely have delved deeper into classifying different types of groups and exploring their properties. This could have led to a more detailed theory of group structures, potentially anticipating parts of what we now know as modern group theory. Linking Galois Theory with Other Areas: Galois might have explored connections between his theory and other areas of mathematics, such as geometry, number theory, or even early versions of what we now call topology. For instance, he could have investigated how the symmetries of geometric shapes relate to the symmetries of algebraic equations. Developing Further Applications: Beyond theory, Galois could have sought applications of his work in more practical areas. Perhaps he would have applied his ideas to solving problems in physics, mechanics, or other emerging fields, where symmetry plays a crucial role. Inspiring the Next Generation: If Galois had lived, he might have become a prominent teacher and mentor, influencing a new generation of mathematicians. His radical ideas would have challenged and inspired others to push the boundaries of mathematics even further. Galois could have spearheaded a new movement in algebra, similar to how Einstein revolutionized physics. Refining His Work: Galois' original manuscripts were hastily written under dire circumstances. Given more time, he would likely have polished and clarified his ideas, making them more accessible to other mathematicians of the time. This might have sped up the acceptance and expansion of his ideas in the mathematical community." ---------------- I'd give it a C for effort, I guess. I don't know enough about the topic to ask it for more specific predictions. At the end of the day it can only work with what it was trained with, so any genuine insights it comes up with will be due more to accident than inspiration.