4 ms·

Incredibly depressing to read this comment when I have tested GPT-4 extensively on simple finite group theory, and it could not reliably distinguish associativi

Incredibly depressing to read this comment when I have tested GPT-4 extensively on simple finite group theory, and it could not reliably distinguish associativity from commutativity, either in prose or in computations, even for very small groups where I gave the multiplication table. The only simple abstract algebra problems it could solve were cliches it almost certainly memorized. I would never use an LLM for learning undergraduate mathematics. It is overwhelmingly likely that you are learning incorrect facts about mathematics from ChatGPT, especially with the distracting gimmick of using cartoon characters.

tombert 2y ago
First, just because there's issues with some misinformation with ChatGPT right now doesn't imply that it will always be there. Second, I know what fuck I'm doing, and I'm sorry to "depress" you, but having a high level summary of something in terms of a cartoon is generally reasonably accurate, and generally any information I learn is also mechanically checked with Isabelle. I agree that it shouldn't be the be-all-end all of everything but if you're a student who's already frustrated with math, having a high-level description of stuff in terms of something you understand can be valuable.
- xhkkffbf 2y ago
  Another way to look at it is to recognize that all teachers have limitations. Humans often convey misinformation too.
  - tombert 2y ago
    Yeah, and I actually think that there can be some value in students being challenged to find some misinformation. I maintain that one of the very best teachers I ever had was my 9th grade biology teacher, purely because she understood absolutely nothing about biology and appeared to just be making shit up. You could argue that in a vacuum this might be benign, but part of the issue is that she would use tests provided by the textbook, written by competent biologists. As a result of this, I had to learn to ignore most of what my teacher said, and sometimes argue back with her, and I feel like ironically I learned biology better than most people in that class; if nothing else I did get an A on all the "real" tests from the textbooks. I think that arguing is actually a really underrated tool in education. Looking for and correcting bullshit is something that extraordinarily enlightening, at least for me, and I think AIs even in their current state can be useful for that.
    - nicce 2y ago
      > Yeah, and I actually think that there can be some value in students being challenged to find some misinformation. The future generations do not have the same background as we do. For them it is very difficult to teach what is "misinformation", or have a thinking model like that, unless we make them read proper books and compare the content for the output of the AI. But if the AI is soon correct enough, they don't get it, and they don't feel it important, and they just take the output from AI as fact.
      - tombert 2y ago
        I agree with all that, and that's why I think we should still keep using traditional books for the foreseeable future. I think that AI can be a terrific supplement, particularly if the students are told to challenge it a bit. I think that just like human teachers, it'll be impossible to completely solve the misinformation problem, but I do think it'll get asymptotically close to being solved.
    - cmcconomy 2y ago
      finding misinformation is a critical skill, but you need a basis to suss out bullshit. diving into the LLM plausibility deepend is not the best way for most people to distinguish the two.
      - tombert 2y ago
        I mentioned in a sibling thread that I am absolutely not suggesting we throw away textbooks or anything like that. I think we should still have and verified work that can referenced as a source of truth, and I think those should be the primary source of learning. What I think is valuable is trying to figure out stuff where ChatGPT contradicts the "established" stuff, and having students figure it out.
    - xhkkffbf 2y ago
      A friend of mine who taught high school said he would tell his class each year that there would be one day when all he would spew would be made up baloney. But he wouldn't tell them which day that would be. It was up to them to discover it. A clever guy he was.
pnathan 2y ago
At this point, I am surprised that LLMs _can_ do code, given the volume of incorrect information they give on specific topics.
- nicklecompte 2y ago
  I think most LLM codegen successes is due to their translation abilities, which is what transformers were designed to do in the first place. Software developers usually solve problems in human language (or maybe a sketch) with general “white collar reasoning abilities” that most of us honed in college, regardless of our major. The translation to Python or whatever is often quite routine. A human developer’s software-specific problem-solving skills are needed for questions involving state, unfamiliar algorithms, “simple” quantitative reasoning, newer programming languages, etc... all of which LLM codegen is pretty bad at.

tombert 2y ago

First, just because there's issues with some misinformation with ChatGPT right now doesn't imply that it will always be there. Second, I know what fuck I'm doing, and I'm sorry to "depress" you, but having a high level summary of something in terms of a cartoon is generally reasonably accurate, and generally any information I learn is also mechanically checked with Isabelle. I agree that it shouldn't be the be-all-end all of everything but if you're a student who's already frustrated with math, having a high-level description of stuff in terms of something you understand can be valuable.

xhkkffbf 2y ago
Another way to look at it is to recognize that all teachers have limitations. Humans often convey misinformation too.
- tombert 2y ago
  Yeah, and I actually think that there can be some value in students being challenged to find some misinformation. I maintain that one of the very best teachers I ever had was my 9th grade biology teacher, purely because she understood absolutely nothing about biology and appeared to just be making shit up. You could argue that in a vacuum this might be benign, but part of the issue is that she would use tests provided by the textbook, written by competent biologists. As a result of this, I had to learn to ignore most of what my teacher said, and sometimes argue back with her, and I feel like ironically I learned biology better than most people in that class; if nothing else I did get an A on all the "real" tests from the textbooks. I think that arguing is actually a really underrated tool in education. Looking for and correcting bullshit is something that extraordinarily enlightening, at least for me, and I think AIs even in their current state can be useful for that.
  - nicce 2y ago
    > Yeah, and I actually think that there can be some value in students being challenged to find some misinformation. The future generations do not have the same background as we do. For them it is very difficult to teach what is "misinformation", or have a thinking model like that, unless we make them read proper books and compare the content for the output of the AI. But if the AI is soon correct enough, they don't get it, and they don't feel it important, and they just take the output from AI as fact.
    - tombert 2y ago
      I agree with all that, and that's why I think we should still keep using traditional books for the foreseeable future. I think that AI can be a terrific supplement, particularly if the students are told to challenge it a bit. I think that just like human teachers, it'll be impossible to completely solve the misinformation problem, but I do think it'll get asymptotically close to being solved.
  - cmcconomy 2y ago
    finding misinformation is a critical skill, but you need a basis to suss out bullshit. diving into the LLM plausibility deepend is not the best way for most people to distinguish the two.
    - tombert 2y ago
      I mentioned in a sibling thread that I am absolutely not suggesting we throw away textbooks or anything like that. I think we should still have and verified work that can referenced as a source of truth, and I think those should be the primary source of learning. What I think is valuable is trying to figure out stuff where ChatGPT contradicts the "established" stuff, and having students figure it out.
  - xhkkffbf 2y ago
    A friend of mine who taught high school said he would tell his class each year that there would be one day when all he would spew would be made up baloney. But he wouldn't tell them which day that would be. It was up to them to discover it. A clever guy he was.

pnathan 2y ago

At this point, I am surprised that LLMs _can_ do code, given the volume of incorrect information they give on specific topics.

nicklecompte 2y ago
I think most LLM codegen successes is due to their translation abilities, which is what transformers were designed to do in the first place. Software developers usually solve problems in human language (or maybe a sketch) with general “white collar reasoning abilities” that most of us honed in college, regardless of our major. The translation to Python or whatever is often quite routine. A human developer’s software-specific problem-solving skills are needed for questions involving state, unfamiliar algorithms, “simple” quantitative reasoning, newer programming languages, etc... all of which LLM codegen is pretty bad at.