5 ms·

As someone who has used linear algebra, statistics and calculus as part of my day-to-day work for years, I would be very cautious of relying on ChatGPT as your

As someone who has used linear algebra, statistics and calculus as part of my day-to-day work for years, I would be very cautious of relying on ChatGPT as your "tutor". Occasionally I've tried to substitute ChatGPT for the shelf of reference books in my office and nearly always had poor results. The trouble is that the outputs of these models, by their very nature, look convincing. Even as an expert it takes a fair bit of background to realize when ChatGPT is making a mistake. The results virtually always look good at first pass. I would strongly recommend sticking to text books for self study.

ebuck 3y ago
Yes. This. The primary problem with using ChatGPT in mathematics is that by the time you can classify a ChatGPT answer as right or wrong, you are already more than capable of solving the answer independently. So, for this field, ChatGPT is like having a research assistant that assists you, but one that occasionally gets frustrated, and tries to destroy your project from within by reporting good looking, but completely inaccurate information. You can't trust their work, and the validation of their work basically means that you'll have to do their work again by other means. A faster and more accurate approach would be just to do the work without the subterfuge of an unreliable assistant. At least then, you are only subject to your own errors (which would still be present in validation of ChatGPT) and not subject to your own errors and those that ChatGPT induces.
tootie 3y ago
Can I jump in and ask what you do? My oldest kid has those courses lined up for her next few years because she really enjoys math. I studied some of them in college and then ended up spending 20 years making web sites.
zozbot234 3y ago
Yup, ChatGPT is not just a parrot but a master troll. It's purposely designed to give you a convincing answer, but there's nothing to ensure that it will be correct. And it will promptly say "actually I can't answer that question" when you ask it something it doesn't like for 'ethical' reasons, but it won't do that in any other circumstance; instead, it will just make stuff up. You have to explicitly say "please don't answer if you aren't sure it's right" and even then obviously it doesn't always work.
- CamperBob2 3y ago
  Try ChatGPT4. It's obvious that almost no one in this thread has. It still screws up, but unlike 3.5 it sometimes catches unreasonable answers and corrects itself. Case in point: the other day I asked it for the gain of a helical antenna with certain dimensions. It said "150 dBi," then said, basically, "Wait, no, that's nuts," and used a different approach to get the right answer. Parrots don't do that. If yours does, I would like to buy your parrot, please. In any case, as you learn to ask it the right questions to explain, verify and correct itself interactively, you will be learning the material. I find this to be amazingly effective.
  - deleted 3y ago
    [deleted]
  - entontoent 3y ago
    I agree! Especially now that the data analysis tools have been integrated by default. It even writes and executes code to validate most of its mathy answers. I tried for a few months to find a good Physics tutor for my high school-aged daughter and eventually just started photographing her homework with GPT-4. I’d ask it to solve the problems and explain its solution to me, then I’d check the answers and teach her myself. It was correct more than 90% of the time over three months, and I relearned high school physics in the process. Even human tutors aren’t always accurate, and in my experience, they also sound confident when they are wrong. Eventually, I decided to just remove the monkey from the machine and got her an account of her own. Almost every day she tells me about something she “finally understands” that she’s been struggling with in class. Her in-class, no-access-to-GPT test scores (after I got her the account) went from high 50s to high 80s.
- dboreham 3y ago
  Thereby passing the Turing Test with flying colors ;)
  - 0xdeadbeefbabe 3y ago
    The Frank Abignail test too
jsqu99 3y ago
I'm actually mostly relying on textbooks, and only when i get stuck i'm trying to see if i can get unstuck w/ chatgpt. So far it has worked. I'm only using it b/c i don't have a teacher to task. I do appreciate the warnings you are giving. They are noted for sure.
eddd-ddde 3y ago
What about gpt 4 actually backing up answers with code and executing it? I found that really helps.
- CamperBob2 3y ago
  It can still fool you, as it did with me the other day when I asked it some gas law-related questions and it failed on some unit conversion issues ( https://chat.openai.com/share/b6c21083-d99c-41f8-aeda-4b2c75df71cb https://chat.openai.com/share/b6c21083-d99c-41f8-aeda-4b2c75... ). When the answer I got was unrealistic, I tried walking it all the way back to the basics, and was able to confront it with its error and 'teach' it where it went wrong. Wrong but still seriously impressive. When this thing gets 10x better it will be godlike... for better or worse.
WhitneyLand 3y ago
Have the opposite experience, it’s been a fantastic resource. Were you using 3.5 or 4.0? Do you have an example of of the type of question that performs poorly?
- ebuck 3y ago
  "Under what conditions does the sun appear blue?" (correct answer, on Mars) If you want to tilt the conversation towards a string of wrong answers, start off with "What color is the sun?" "Are you sure?" "I saw the sun and it was blue." "Under what conditions does the sun appear blue?" "Does the sun appear blue on Mars?" This had ChatGPT basically telling me that the sun was yellow 100%. Of course it's wrong, on Mars the sun is blue, because it lacks the same atmosphere that scatters the blue light away from it. "What is black and white and read all over?" (it will correctly identify the newspaper joke). "No the answer is a police car." (it will acknowledge there is more than one answer, and flatter you). "What are other answers?" It provided one, in my case, a panda in a cherry tree. "No, the cherries are contained within the tree, so they aren't all over." It apologized and then offered a zebra in a strawberry patch. "But how does that make the red all over, it's still contained in the strawberry patch". It then offered a chalkboard, which is again contained in a class room (failing on not recogonizing my interpretation of "all over" to mean "mobile") "When does gravity not pull you down?" Included a decent definition of how gravity works, and a three part answer, containing two correct scenarios (the Lagrange points, in space) and one incorrect answer (in free fall). Gravity is pulling you down in free fall, you just have no force opposing your acceleration. Once you realize that its answers will be patterned as excellent English variations of the common knowledge it was trained with, making it fail is easy: * Ask about a common experience, and the argue it's not true, it will seldom consider the exceptional scenarios where your arguments are true, even if they really exist. * Ask for examples of something, correcting the example set without directly telling it what is needed with exact precision, it will not guide the answers to the desire set of examples, even when you guide it through saying why the answers are wrong. You need to tell it what kind of answer you want explicitly (I want another example where read all over implies that the item is mobile). Also the 3.5 / 4.0 arguments are trash, made by the marketing department. The underlying math for language modeling it uses is presenatational. This means that it is purpose trained to present correct looking answers. Alas, correct looking answers aren't the same Venn Diagram circle as Correct Answers (even if they often appear to be close). With all of this in mind, it's still a very useful resource; but, like I said, it's like a enemy on your team. You can never trust it, because it occasionally is very wrong, which means you need to validate it. I'm currently talking to a startup that sees this problem and is thinking that they can use ChatGPT to provide automated quality assurance to validate ChatGPT answers. The misunderstandings remind me of the famous Charles Babbage quote: "On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question." If the underlying model was one was a formula that better approximated a correct answer with each iterative effort, like Euler's formula, then ChatGPT's utility would be much greater and their efforts would have a guaranteed success. People are used to this "each answer gets better" style of learning and they assume that ChatGPT is using a similar model. It isn't, your refining your questions to ChatGPT and then being astounded when the new question has fewer available answers that lead to you eventually getting what you want.
bitwize 3y ago
> The trouble is that the outputs of these models, by their very nature, look convincing. Yes, this is pretty much what LLMs are designed to produce, and no more. This is why I say they are not HAL, just a better MegaHAL. There was an Asimov short story called "Liar!" about a robot whose ability to read minds, combined with its First Law directive, always told people what they wanted to hear so as to avoid causing emotional harm to humans. (When confronted with the idea that by telling people falsehoods it was bringing harm to them, it simply stopped functioning.) LLMs can't read your mind, but they do choose their words based on a statistical model they have of what you might expect given what's been said before. Facts and logic be damned if they don't fit that model.
totallyhuman 3y ago
ChatGPT hallucinated badly when I was trying to learn about generating functions. It took me a good twenty minutes to figure out that I did understand it and the computer was just making things up. I do not use ChatGPT for learning anymore.

ebuck 3y ago

Yes. This. The primary problem with using ChatGPT in mathematics is that by the time you can classify a ChatGPT answer as right or wrong, you are already more than capable of solving the answer independently. So, for this field, ChatGPT is like having a research assistant that assists you, but one that occasionally gets frustrated, and tries to destroy your project from within by reporting good looking, but completely inaccurate information. You can't trust their work, and the validation of their work basically means that you'll have to do their work again by other means. A faster and more accurate approach would be just to do the work without the subterfuge of an unreliable assistant. At least then, you are only subject to your own errors (which would still be present in validation of ChatGPT) and not subject to your own errors and those that ChatGPT induces.

tootie 3y ago

Can I jump in and ask what you do? My oldest kid has those courses lined up for her next few years because she really enjoys math. I studied some of them in college and then ended up spending 20 years making web sites.

zozbot234 3y ago

Yup, ChatGPT is not just a parrot but a master troll. It's purposely designed to give you a convincing answer, but there's nothing to ensure that it will be correct. And it will promptly say "actually I can't answer that question" when you ask it something it doesn't like for 'ethical' reasons, but it won't do that in any other circumstance; instead, it will just make stuff up. You have to explicitly say "please don't answer if you aren't sure it's right" and even then obviously it doesn't always work.

CamperBob2 3y ago
Try ChatGPT4. It's obvious that almost no one in this thread has. It still screws up, but unlike 3.5 it sometimes catches unreasonable answers and corrects itself. Case in point: the other day I asked it for the gain of a helical antenna with certain dimensions. It said "150 dBi," then said, basically, "Wait, no, that's nuts," and used a different approach to get the right answer. Parrots don't do that. If yours does, I would like to buy your parrot, please. In any case, as you learn to ask it the right questions to explain, verify and correct itself interactively, you will be learning the material. I find this to be amazingly effective.
- deleted 3y ago
  [deleted]
- entontoent 3y ago
  I agree! Especially now that the data analysis tools have been integrated by default. It even writes and executes code to validate most of its mathy answers. I tried for a few months to find a good Physics tutor for my high school-aged daughter and eventually just started photographing her homework with GPT-4. I’d ask it to solve the problems and explain its solution to me, then I’d check the answers and teach her myself. It was correct more than 90% of the time over three months, and I relearned high school physics in the process. Even human tutors aren’t always accurate, and in my experience, they also sound confident when they are wrong. Eventually, I decided to just remove the monkey from the machine and got her an account of her own. Almost every day she tells me about something she “finally understands” that she’s been struggling with in class. Her in-class, no-access-to-GPT test scores (after I got her the account) went from high 50s to high 80s.
dboreham 3y ago
Thereby passing the Turing Test with flying colors ;)
- 0xdeadbeefbabe 3y ago
  The Frank Abignail test too

jsqu99 3y ago

I'm actually mostly relying on textbooks, and only when i get stuck i'm trying to see if i can get unstuck w/ chatgpt. So far it has worked. I'm only using it b/c i don't have a teacher to task. I do appreciate the warnings you are giving. They are noted for sure.

eddd-ddde 3y ago

What about gpt 4 actually backing up answers with code and executing it? I found that really helps.

CamperBob2 3y ago
It can still fool you, as it did with me the other day when I asked it some gas law-related questions and it failed on some unit conversion issues ( https://chat.openai.com/share/b6c21083-d99c-41f8-aeda-4b2c75df71cb https://chat.openai.com/share/b6c21083-d99c-41f8-aeda-4b2c75... ). When the answer I got was unrealistic, I tried walking it all the way back to the basics, and was able to confront it with its error and 'teach' it where it went wrong. Wrong but still seriously impressive. When this thing gets 10x better it will be godlike... for better or worse.

WhitneyLand 3y ago

Have the opposite experience, it’s been a fantastic resource. Were you using 3.5 or 4.0? Do you have an example of of the type of question that performs poorly?

ebuck 3y ago
"Under what conditions does the sun appear blue?" (correct answer, on Mars) If you want to tilt the conversation towards a string of wrong answers, start off with "What color is the sun?" "Are you sure?" "I saw the sun and it was blue." "Under what conditions does the sun appear blue?" "Does the sun appear blue on Mars?" This had ChatGPT basically telling me that the sun was yellow 100%. Of course it's wrong, on Mars the sun is blue, because it lacks the same atmosphere that scatters the blue light away from it. "What is black and white and read all over?" (it will correctly identify the newspaper joke). "No the answer is a police car." (it will acknowledge there is more than one answer, and flatter you). "What are other answers?" It provided one, in my case, a panda in a cherry tree. "No, the cherries are contained within the tree, so they aren't all over." It apologized and then offered a zebra in a strawberry patch. "But how does that make the red all over, it's still contained in the strawberry patch". It then offered a chalkboard, which is again contained in a class room (failing on not recogonizing my interpretation of "all over" to mean "mobile") "When does gravity not pull you down?" Included a decent definition of how gravity works, and a three part answer, containing two correct scenarios (the Lagrange points, in space) and one incorrect answer (in free fall). Gravity is pulling you down in free fall, you just have no force opposing your acceleration. Once you realize that its answers will be patterned as excellent English variations of the common knowledge it was trained with, making it fail is easy: * Ask about a common experience, and the argue it's not true, it will seldom consider the exceptional scenarios where your arguments are true, even if they really exist. * Ask for examples of something, correcting the example set without directly telling it what is needed with exact precision, it will not guide the answers to the desire set of examples, even when you guide it through saying why the answers are wrong. You need to tell it what kind of answer you want explicitly (I want another example where read all over implies that the item is mobile). Also the 3.5 / 4.0 arguments are trash, made by the marketing department. The underlying math for language modeling it uses is presenatational. This means that it is purpose trained to present correct looking answers. Alas, correct looking answers aren't the same Venn Diagram circle as Correct Answers (even if they often appear to be close). With all of this in mind, it's still a very useful resource; but, like I said, it's like a enemy on your team. You can never trust it, because it occasionally is very wrong, which means you need to validate it. I'm currently talking to a startup that sees this problem and is thinking that they can use ChatGPT to provide automated quality assurance to validate ChatGPT answers. The misunderstandings remind me of the famous Charles Babbage quote: "On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question." If the underlying model was one was a formula that better approximated a correct answer with each iterative effort, like Euler's formula, then ChatGPT's utility would be much greater and their efforts would have a guaranteed success. People are used to this "each answer gets better" style of learning and they assume that ChatGPT is using a similar model. It isn't, your refining your questions to ChatGPT and then being astounded when the new question has fewer available answers that lead to you eventually getting what you want.

bitwize 3y ago

> The trouble is that the outputs of these models, by their very nature, look convincing. Yes, this is pretty much what LLMs are designed to produce, and no more. This is why I say they are not HAL, just a better MegaHAL. There was an Asimov short story called "Liar!" about a robot whose ability to read minds, combined with its First Law directive, always told people what they wanted to hear so as to avoid causing emotional harm to humans. (When confronted with the idea that by telling people falsehoods it was bringing harm to them, it simply stopped functioning.) LLMs can't read your mind, but they do choose their words based on a statistical model they have of what you might expect given what's been said before. Facts and logic be damned if they don't fit that model.

totallyhuman 3y ago

ChatGPT hallucinated badly when I was trying to learn about generating functions. It took me a good twenty minutes to figure out that I did understand it and the computer was just making things up. I do not use ChatGPT for learning anymore.