4 ms·

I dunno, I've found ChatGPT (and its clones) have been immensely useful as a learning tool for me. Having it summarize stuff in different ways and being able t

I dunno, I've found ChatGPT (and its clones) have been immensely useful as a learning tool for me. Having it summarize stuff in different ways and being able to immediately ask clarifying questions has been sort of a game-changing learning tool, at least for me. I don't think they should allow use of AI during tests because yeah, it'd just do all the work for them and they wouldn't learn anything, but I absolutely think there's value in being able to use it as a 24/7 instant tutor, particularly since ChatGPT can be coerced into explaining things in basically any style that you'd like. Something I've found fun and bizarrely useful is having ChatGPT explain stuff in terms of scripts from King of the Hill. It's fun to hear Hank Hill compare ZF Set Theory to something as reliable and efficient as propane while Dale goes into tirades about how the government imposed the axiom of choice to distract away from System F.

ben_w 2y ago
> I don't think they should allow use of AI during tests because yeah, it'd just do all the work for them and they wouldn't learn anything, but I absolutely think there's value in being able to use it as a 24/7 instant tutor, particularly since ChatGPT can be coerced into explaining things in basically any style that you'd like. For any test where there's an AI good enough to cheat with and cheap enough students can afford it, the test itself is no longer important — it is as pointless as testing how well students can use fountain pens when everyone uses a keyboard for actual work. Or how well they can play chess. I'm not sure the exact shape of what each model can and can't do, but I am confident that this means they're never be an important "cheating" risk, not even when they're technically capable of it.
- tombert 2y ago
  Well, no, I absolutely and fundamentally disagree with that. We didn't just stop teaching children how to read the second that text-to-speech stuff came about, and we didn't stop teaching basic arithmetic the second that calculators were invented, and we didn't stop teaching algebra the second that Wolfram Alpha was released. There is value in learning the fundamentals of things. If you never actually learn algebra, then it becomes effectively impossible to do the more advanced stuff.
  - ben_w 2y ago
    > We didn't just stop teaching children how to read the second that text-to-speech stuff came about TTS from visual input isn't good enough to be an example of that yet. It's good, but not that good. > we didn't stop teaching basic arithmetic the second that calculators were invented and we didn't stop teaching algebra the second that Wolfram Alpha was released. And yet, my A-level exams 22 years ago required me to have a graphic calculator.
    - tombert 2y ago
      > TTS from visual input isn't good enough to be an example of that yet. It's good, but not that good. Even if TTS were absolutely perfect, then I'd still maintain my point. > And yet, my A-level exams 22 years ago required me to have a graphic calculator. Doesn't that sort of prove my point? Calculators existed and were obviously useful, but they didn't just stop teaching you algebra, despite knowing you had a calculator.
      - ben_w 2y ago
        > Doesn't that sort of prove my point? Only if you missed mine about how it is no longer cheating to use one.
        tombert 2y ago
        I see, so you're claiming that if AI can be utilized to solve the problem, and it's determined to be useful, then it should be allowed to be on a test as a result, in the same way that a graphing calculator is no longer considered cheating? I think the disagreement I have is that I think part of the reason calculators are allowed on calculus tests is because they're actually not that useful for a lot of what they teach you in calculus. I had a graphing calculator in my math and physics classes in high school, but most of my work was still done with a pen and paper. The calculator was of course useful for the arithmetic stuff, and it could be useful to sanity-check your work (e.g. to see if the roots you calculated were correct), but I still had to do most of the actual calculus by hand. Even when I got a calculator with a CAS built in (HP 50G), I would still have to do most of the algebra by hand, and honestly if my teachers had known that my calculator had a CAS I suspect they wouldn't have allowed me to use it to begin with.
- llm_trw 2y ago
  Fountain pens are easier to use than most ball point pens. If you need to think deeply about a problem there are few better things than a binder of a3 paper and a fountain pen to think. That said the uselessness of education is the point. It's a signal that you're willing to do pointless work for future rewards. We may as well replace it with counting beans and it would be just as effective.
  - ben_w 2y ago
    As a southpaw, the last thing I want to do is touch a fountain pen. Well, unless I have another go at learning Arabic…
- miki123211 2y ago
  I used to agree with this point, but I actually changed my stance. Once you hit a problem that an AI cannot solve, it's good to be at least aware of what's going on in the field. You don't necessarily have to remember every formula and every fact off the top of your head, but having an intuition on what facts and formulas there are and where they can be used is immensely helpful. Imagine you're writing a program in an esoteric scripting language that Copilot doesn't understand, and you need to put a large array of events in order from oldest to newest. If you had to learn this sort of think before, you'll immediately have an intuition that this is called sorting, there are slow (n^2) sorting algorithms and faster (n log n) sorting algorithms and you want the latter, and that quicksort is probably a good candidate to start with. You don't necessarily have to be able to write quicksort by hand, you can look up an implementation online and port it to your weird proprietary language, but the intuition is important.
  - ben_w 2y ago
    Hmm. That sounds like a different thing than I'm saying, and I agree with what I think you're saying. So, if an AI can answer "what's a fast sort function" with "quicksort, here's some pseudocode: …", that's a related but different question to "write quicksort in [this brand new language you've never encountered before]". Is like how AI won't take our jobs, but it will take over for each individual task that our jobs are made from.
- deleted 2y ago
  [deleted]
nicklecompte 2y ago
Incredibly depressing to read this comment when I have tested GPT-4 extensively on simple finite group theory, and it could not reliably distinguish associativity from commutativity, either in prose or in computations, even for very small groups where I gave the multiplication table. The only simple abstract algebra problems it could solve were cliches it almost certainly memorized. I would never use an LLM for learning undergraduate mathematics. It is overwhelmingly likely that you are learning incorrect facts about mathematics from ChatGPT, especially with the distracting gimmick of using cartoon characters.
- tombert 2y ago
  First, just because there's issues with some misinformation with ChatGPT right now doesn't imply that it will always be there. Second, I know what fuck I'm doing, and I'm sorry to "depress" you, but having a high level summary of something in terms of a cartoon is generally reasonably accurate, and generally any information I learn is also mechanically checked with Isabelle. I agree that it shouldn't be the be-all-end all of everything but if you're a student who's already frustrated with math, having a high-level description of stuff in terms of something you understand can be valuable.
  - xhkkffbf 2y ago
    Another way to look at it is to recognize that all teachers have limitations. Humans often convey misinformation too.
    - tombert 2y ago
      Yeah, and I actually think that there can be some value in students being challenged to find some misinformation. I maintain that one of the very best teachers I ever had was my 9th grade biology teacher, purely because she understood absolutely nothing about biology and appeared to just be making shit up. You could argue that in a vacuum this might be benign, but part of the issue is that she would use tests provided by the textbook, written by competent biologists. As a result of this, I had to learn to ignore most of what my teacher said, and sometimes argue back with her, and I feel like ironically I learned biology better than most people in that class; if nothing else I did get an A on all the "real" tests from the textbooks. I think that arguing is actually a really underrated tool in education. Looking for and correcting bullshit is something that extraordinarily enlightening, at least for me, and I think AIs even in their current state can be useful for that.
RandomLensman 2y ago
Does ChatGPT give you good exercises in set theory to work through as well on that or is that back to other material?
- tombert 2y ago
  I've never really had it do exercises in set theory, so tough to say. Usually the thing I like about it most is getting it summarize stuff at a high level, and then giving me keywords than I can independently search for and play around with in Isabelle. Like, for me I think part of the value to ChatGPT is simply figuring out some of what I don't know that I don't know. When I was a lecturer I would use ChatGPT to generate Java homework assignment. It would do "ok" with that, but it usually required three or four iterations to get something that actually made sense, and usually it would require a bit of editing on my end to get results I was happy with. It still saved me a boatload of time though, particularly since it was even able to give me the assignment in pandoc-compatible markdown.
  - RandomLensman 2y ago
    I always thought it tough to really learn math without doing exercise to really understand how things work.
    - tombert 2y ago
      It is! In my case, that's why I usually try and figure out something to do in Isabelle. Trying to prove some kind of property based on something I learned from ChatGPT. I also will occasionally grab exercises from "real" textbooks once I know the keywords to search in Kagi.
      - RandomLensman 2y ago
        Not sure self-selecting on exercises is really sufficient (at least for me I doubt it).
        2y ago

ben_w 2y ago

> I don't think they should allow use of AI during tests because yeah, it'd just do all the work for them and they wouldn't learn anything, but I absolutely think there's value in being able to use it as a 24/7 instant tutor, particularly since ChatGPT can be coerced into explaining things in basically any style that you'd like. For any test where there's an AI good enough to cheat with and cheap enough students can afford it, the test itself is no longer important — it is as pointless as testing how well students can use fountain pens when everyone uses a keyboard for actual work. Or how well they can play chess. I'm not sure the exact shape of what each model can and can't do, but I am confident that this means they're never be an important "cheating" risk, not even when they're technically capable of it.

tombert 2y ago
Well, no, I absolutely and fundamentally disagree with that. We didn't just stop teaching children how to read the second that text-to-speech stuff came about, and we didn't stop teaching basic arithmetic the second that calculators were invented, and we didn't stop teaching algebra the second that Wolfram Alpha was released. There is value in learning the fundamentals of things. If you never actually learn algebra, then it becomes effectively impossible to do the more advanced stuff.
- ben_w 2y ago
  > We didn't just stop teaching children how to read the second that text-to-speech stuff came about TTS from visual input isn't good enough to be an example of that yet. It's good, but not that good. > we didn't stop teaching basic arithmetic the second that calculators were invented and we didn't stop teaching algebra the second that Wolfram Alpha was released. And yet, my A-level exams 22 years ago required me to have a graphic calculator.
  - tombert 2y ago
    > TTS from visual input isn't good enough to be an example of that yet. It's good, but not that good. Even if TTS were absolutely perfect, then I'd still maintain my point. > And yet, my A-level exams 22 years ago required me to have a graphic calculator. Doesn't that sort of prove my point? Calculators existed and were obviously useful, but they didn't just stop teaching you algebra, despite knowing you had a calculator.
    - ben_w 2y ago
      > Doesn't that sort of prove my point? Only if you missed mine about how it is no longer cheating to use one.
      - tombert 2y ago
        I see, so you're claiming that if AI can be utilized to solve the problem, and it's determined to be useful, then it should be allowed to be on a test as a result, in the same way that a graphing calculator is no longer considered cheating? I think the disagreement I have is that I think part of the reason calculators are allowed on calculus tests is because they're actually not that useful for a lot of what they teach you in calculus. I had a graphing calculator in my math and physics classes in high school, but most of my work was still done with a pen and paper. The calculator was of course useful for the arithmetic stuff, and it could be useful to sanity-check your work (e.g. to see if the roots you calculated were correct), but I still had to do most of the actual calculus by hand. Even when I got a calculator with a CAS built in (HP 50G), I would still have to do most of the algebra by hand, and honestly if my teachers had known that my calculator had a CAS I suspect they wouldn't have allowed me to use it to begin with.
llm_trw 2y ago
Fountain pens are easier to use than most ball point pens. If you need to think deeply about a problem there are few better things than a binder of a3 paper and a fountain pen to think. That said the uselessness of education is the point. It's a signal that you're willing to do pointless work for future rewards. We may as well replace it with counting beans and it would be just as effective.
- ben_w 2y ago
  As a southpaw, the last thing I want to do is touch a fountain pen. Well, unless I have another go at learning Arabic…
miki123211 2y ago
I used to agree with this point, but I actually changed my stance. Once you hit a problem that an AI cannot solve, it's good to be at least aware of what's going on in the field. You don't necessarily have to remember every formula and every fact off the top of your head, but having an intuition on what facts and formulas there are and where they can be used is immensely helpful. Imagine you're writing a program in an esoteric scripting language that Copilot doesn't understand, and you need to put a large array of events in order from oldest to newest. If you had to learn this sort of think before, you'll immediately have an intuition that this is called sorting, there are slow (n^2) sorting algorithms and faster (n log n) sorting algorithms and you want the latter, and that quicksort is probably a good candidate to start with. You don't necessarily have to be able to write quicksort by hand, you can look up an implementation online and port it to your weird proprietary language, but the intuition is important.
- ben_w 2y ago
  Hmm. That sounds like a different thing than I'm saying, and I agree with what I think you're saying. So, if an AI can answer "what's a fast sort function" with "quicksort, here's some pseudocode: …", that's a related but different question to "write quicksort in [this brand new language you've never encountered before]". Is like how AI won't take our jobs, but it will take over for each individual task that our jobs are made from.
deleted 2y ago
[deleted]

nicklecompte 2y ago

Incredibly depressing to read this comment when I have tested GPT-4 extensively on simple finite group theory, and it could not reliably distinguish associativity from commutativity, either in prose or in computations, even for very small groups where I gave the multiplication table. The only simple abstract algebra problems it could solve were cliches it almost certainly memorized. I would never use an LLM for learning undergraduate mathematics. It is overwhelmingly likely that you are learning incorrect facts about mathematics from ChatGPT, especially with the distracting gimmick of using cartoon characters.

tombert 2y ago
First, just because there's issues with some misinformation with ChatGPT right now doesn't imply that it will always be there. Second, I know what fuck I'm doing, and I'm sorry to "depress" you, but having a high level summary of something in terms of a cartoon is generally reasonably accurate, and generally any information I learn is also mechanically checked with Isabelle. I agree that it shouldn't be the be-all-end all of everything but if you're a student who's already frustrated with math, having a high-level description of stuff in terms of something you understand can be valuable.
- xhkkffbf 2y ago
  Another way to look at it is to recognize that all teachers have limitations. Humans often convey misinformation too.
  - tombert 2y ago
    Yeah, and I actually think that there can be some value in students being challenged to find some misinformation. I maintain that one of the very best teachers I ever had was my 9th grade biology teacher, purely because she understood absolutely nothing about biology and appeared to just be making shit up. You could argue that in a vacuum this might be benign, but part of the issue is that she would use tests provided by the textbook, written by competent biologists. As a result of this, I had to learn to ignore most of what my teacher said, and sometimes argue back with her, and I feel like ironically I learned biology better than most people in that class; if nothing else I did get an A on all the "real" tests from the textbooks. I think that arguing is actually a really underrated tool in education. Looking for and correcting bullshit is something that extraordinarily enlightening, at least for me, and I think AIs even in their current state can be useful for that.

RandomLensman 2y ago

Does ChatGPT give you good exercises in set theory to work through as well on that or is that back to other material?

tombert 2y ago
I've never really had it do exercises in set theory, so tough to say. Usually the thing I like about it most is getting it summarize stuff at a high level, and then giving me keywords than I can independently search for and play around with in Isabelle. Like, for me I think part of the value to ChatGPT is simply figuring out some of what I don't know that I don't know. When I was a lecturer I would use ChatGPT to generate Java homework assignment. It would do "ok" with that, but it usually required three or four iterations to get something that actually made sense, and usually it would require a bit of editing on my end to get results I was happy with. It still saved me a boatload of time though, particularly since it was even able to give me the assignment in pandoc-compatible markdown.
- RandomLensman 2y ago
  I always thought it tough to really learn math without doing exercise to really understand how things work.
  - tombert 2y ago
    It is! In my case, that's why I usually try and figure out something to do in Isabelle. Trying to prove some kind of property based on something I learned from ChatGPT. I also will occasionally grab exercises from "real" textbooks once I know the keywords to search in Kagi.
    - RandomLensman 2y ago
      Not sure self-selecting on exercises is really sufficient (at least for me I doubt it).
      - 2y ago