12 ms·

> told me my answer (to a problem it had given me) was wrong, even though I was right. This is a fatal flaw. It absolutely cannot make mistakes like this if it

> told me my answer (to a problem it had given me) was wrong, even though I was right. This is a fatal flaw. It absolutely cannot make mistakes like this if it's going to succeed because that will undermine the confidence that teachers and students have in the system.

rahimnathwani 2y ago
Here's a screenshot of the conversation: https://imgur.com/a/MYOQRK3 https://imgur.com/a/MYOQRK3 Aside from the wrong answer, I also didn't like the fact that it just told me to multiply the numbers, instead of giving me a smaller hint that might allow me to realize this for myself.
- gnicholas 2y ago
  Amazing that it made such a basic mistake, so many times. I thought they wrapped the AI in enough layers of meta-cognition that it wouldn't make mistakes like this. Maybe AI in education really is like FSD, as another commenter said (so close, yet so far).
  - rahimnathwani 2y ago
    AIUI Khanmigo uses GPT-4 under the hood, so honestly I'm perplexed.
ianbicking 2y ago
Khanmigo is a surprisingly naive product, but this isn't a particularly hard problem: 1. It's not necessary to invent all these problems from scratch. At least for any "normal" math problem the world doesn't need new problems on demand, you just have to fetch one. Khan Academy has a perfectly good bank of problems. 2. If you have a baked problem you should hang onto knowledge about the correct answer. Khanmigo just seems to put it in the chat history where it is easily forgotten. 3. If you want customized problems you might need to invent problems instead of reading problems from a bank. In that case you might want to create a more "formal" problem and an "informal" problem, where the formal problem is expressed clearly and solvable (e.g., by Wolfram or SageMath or something), and therefore can be verified. Translating the formal problem into a word problem is generally much more reliable than inventing a problem and correct solution. That said, "right answers" aren't IMHO that interesting. Helping the student attack a problem in the right way, or adapt their knowledge to be able to apply it to a problem, is more interesting and helpful.

rahimnathwani 2y ago

Here's a screenshot of the conversation: https://imgur.com/a/MYOQRK3 https://imgur.com/a/MYOQRK3 Aside from the wrong answer, I also didn't like the fact that it just told me to multiply the numbers, instead of giving me a smaller hint that might allow me to realize this for myself.

gnicholas 2y ago
Amazing that it made such a basic mistake, so many times. I thought they wrapped the AI in enough layers of meta-cognition that it wouldn't make mistakes like this. Maybe AI in education really is like FSD, as another commenter said (so close, yet so far).
- rahimnathwani 2y ago
  AIUI Khanmigo uses GPT-4 under the hood, so honestly I'm perplexed.

ianbicking 2y ago

Khanmigo is a surprisingly naive product, but this isn't a particularly hard problem: 1. It's not necessary to invent all these problems from scratch. At least for any "normal" math problem the world doesn't need new problems on demand, you just have to fetch one. Khan Academy has a perfectly good bank of problems. 2. If you have a baked problem you should hang onto knowledge about the correct answer. Khanmigo just seems to put it in the chat history where it is easily forgotten. 3. If you want customized problems you might need to invent problems instead of reading problems from a bank. In that case you might want to create a more "formal" problem and an "informal" problem, where the formal problem is expressed clearly and solvable (e.g., by Wolfram or SageMath or something), and therefore can be verified. Translating the formal problem into a word problem is generally much more reliable than inventing a problem and correct solution. That said, "right answers" aren't IMHO that interesting. Helping the student attack a problem in the right way, or adapt their knowledge to be able to apply it to a problem, is more interesting and helpful.