3 ms·

Should we also try to teach children geometry and test them on calculus?

by red75prime 2y ago

IIAOPSW 2y ago
For centuries we only taught children geometry and one of them invented calculus.
- Filligree 2y ago
  Now that’s setting a high bar. If AI could reliably invent calculus, then I’d be briefly impressed and then terrified.
  - eru 2y ago
    One AI in a couple hundred years might be able to do it by luck?
nico 2y ago
This illustrates two ways of teaching I’ve experienced both, each at a different university In one, professors would teach one thing then ask very different (and much harder) questions on tests In the other, tests were more of a recap of the material up to that point I definitely learned a lot more in the second case and was a lot more motivated. It also required more effort from the professors The two methods also test different things. The recap one tests effort and dedication, if you do the work, you get the grade The difficult tests measure either luck and/or creativity and problem solving under pressure. It’s not about doing the work, it’s about either being lucky or good at testing
- Pet_Ant 2y ago
  > it’s about either being lucky or good at testing I think you are misunderstanding the experience. The first (harder questions) is testing your understanding of the material and problem. Can you applying the material to solve a novel problem? Do you understand the material not just the mechanics. Do you understand how it would interelate it with other problems? Do you understand the limitations? The second is just regurgitation. This is great for rote skills, but this isn't really learning. This is grinding until you can reproduce. These are the kinds of skills that are easily automated. This is not what we should be testing our kids.
  - godelski 2y ago
    Fwiw, I think both of you are on the same page. And yes, to bring back to ML it is the difference of generalization and memorization (compression). I wrote a longer response to a different response to my initial comment to help clarify because I think this chain is a bit obtuse and aggressive for no reason :/ (I mean you can check the Wiki page to verify what I said)
feoren 2y ago
I think the authors are responding to a claim that AI is doing this: look, we taught them geometry, and now they know calculus! GP is saying that it's not a true zero-shot to have a separate test and training set, because the classes overlap. Similarly, the authors are saying "true zero-shot" is basically not happening, at least not nearly to the extent some are claiming. So everyone here, including TFA, are all kinda doubting the same claim (our AI models can perform zero-shot generalizations) in different ways, I think?
godelski 2y ago
I think you're being overly obtuse, and I'd request you try to work in good faith. If you doubt what I claimed, you can quickly verify on the wiki page[0]. In essence you aren't wrong, but that's not what we'd typically do in a zero (or few) shot setting. We'd be focusing on things that are more similar. If you want to understand this a bit better in what we might do in a ML context I wrote more here[1]. And I like Nico's comment about how different professors test. Because it makes you think about what is actually being tested. Are you being tested on memorization or generalization? You can argue both these kinds of tests are testing "if you learned the material" but we'd understand that these two types of tests are fundamentally different and let's be real, are not reasonably fair to compare scores to. I'm sure many of us have experienced this where someone that gets a C in professor A's class likely learned more than than someone who got an A in professor B's class. The thing is that the nuance is incredibly important here to really understand. And you can trivialize anything, but be careful when doing so, you may overlook the most important things ;) Now... you could make this argument about geometry -> calculus if we're not talking about the typical geometry (single) class most people will have taken in either middle school or high school. Because yes, at the end of the day there is a geometric interpretation and we have the Riemann sum. But we'd need to ensure those kids have the understanding of infinities (which aren't numbers btw). We'd have to be pretty careful about formulating this kind of test if we're going to take away any useful information from it. Though the naive version might give us clues about information leakage (in our case with children this might be "child who has a parent that's a mathematician" or something along those lines). It really all depends on what the question behind the test is. Scores only mean things when we have nuanced clear understandings of what we're measuring (so again, tread carefully because "here be dragons" and you're likely to get burned before, or even without, knowing it) And truth be told, we actually do this a bit. There's a reason you take geometry before calculus. Because the skills build up. But you're right that they don't generalize. [0] https://en.wikipedia.org/wiki/Zero-shot_learning https://en.wikipedia.org/wiki/Zero-shot_learning [1] https://news.ycombinator.com/item?id=40313501 https://news.ycombinator.com/item?id=40313501

Should we also try to teach children geometry and test them on calculus?

IIAOPSW 2y ago

For centuries we only taught children geometry and one of them invented calculus.

Filligree 2y ago
Now that’s setting a high bar. If AI could reliably invent calculus, then I’d be briefly impressed and then terrified.
- eru 2y ago
  One AI in a couple hundred years might be able to do it by luck?

nico 2y ago

This illustrates two ways of teaching I’ve experienced both, each at a different university In one, professors would teach one thing then ask very different (and much harder) questions on tests In the other, tests were more of a recap of the material up to that point I definitely learned a lot more in the second case and was a lot more motivated. It also required more effort from the professors The two methods also test different things. The recap one tests effort and dedication, if you do the work, you get the grade The difficult tests measure either luck and/or creativity and problem solving under pressure. It’s not about doing the work, it’s about either being lucky or good at testing

Pet_Ant 2y ago
> it’s about either being lucky or good at testing I think you are misunderstanding the experience. The first (harder questions) is testing your understanding of the material and problem. Can you applying the material to solve a novel problem? Do you understand the material not just the mechanics. Do you understand how it would interelate it with other problems? Do you understand the limitations? The second is just regurgitation. This is great for rote skills, but this isn't really learning. This is grinding until you can reproduce. These are the kinds of skills that are easily automated. This is not what we should be testing our kids.
- godelski 2y ago
  Fwiw, I think both of you are on the same page. And yes, to bring back to ML it is the difference of generalization and memorization (compression). I wrote a longer response to a different response to my initial comment to help clarify because I think this chain is a bit obtuse and aggressive for no reason :/ (I mean you can check the Wiki page to verify what I said)

feoren 2y ago

I think the authors are responding to a claim that AI is doing this: look, we taught them geometry, and now they know calculus! GP is saying that it's not a true zero-shot to have a separate test and training set, because the classes overlap. Similarly, the authors are saying "true zero-shot" is basically not happening, at least not nearly to the extent some are claiming. So everyone here, including TFA, are all kinda doubting the same claim (our AI models can perform zero-shot generalizations) in different ways, I think?

godelski 2y ago

I think you're being overly obtuse, and I'd request you try to work in good faith. If you doubt what I claimed, you can quickly verify on the wiki page[0]. In essence you aren't wrong, but that's not what we'd typically do in a zero (or few) shot setting. We'd be focusing on things that are more similar. If you want to understand this a bit better in what we might do in a ML context I wrote more here[1]. And I like Nico's comment about how different professors test. Because it makes you think about what is actually being tested. Are you being tested on memorization or generalization? You can argue both these kinds of tests are testing "if you learned the material" but we'd understand that these two types of tests are fundamentally different and let's be real, are not reasonably fair to compare scores to. I'm sure many of us have experienced this where someone that gets a C in professor A's class likely learned more than than someone who got an A in professor B's class. The thing is that the nuance is incredibly important here to really understand. And you can trivialize anything, but be careful when doing so, you may overlook the most important things ;) Now... you could make this argument about geometry -> calculus if we're not talking about the typical geometry (single) class most people will have taken in either middle school or high school. Because yes, at the end of the day there is a geometric interpretation and we have the Riemann sum. But we'd need to ensure those kids have the understanding of infinities (which aren't numbers btw). We'd have to be pretty careful about formulating this kind of test if we're going to take away any useful information from it. Though the naive version might give us clues about information leakage (in our case with children this might be "child who has a parent that's a mathematician" or something along those lines). It really all depends on what the question behind the test is. Scores only mean things when we have nuanced clear understandings of what we're measuring (so again, tread carefully because "here be dragons" and you're likely to get burned before, or even without, knowing it) And truth be told, we actually do this a bit. There's a reason you take geometry before calculus. Because the skills build up. But you're right that they don't generalize. [0] https://en.wikipedia.org/wiki/Zero-shot_learning https://en.wikipedia.org/wiki/Zero-shot_learning [1] https://news.ycombinator.com/item?id=40313501 https://news.ycombinator.com/item?id=40313501