4 ms·

> 1. They don’t do what they say they do (assess retained learning, track progress) So getting an A or a C in arithmetic doesn't reveal anything about retained

by qiskit 5y ago

> 1. They don’t do what they say they do (assess retained learning, track progress) So getting an A or a C in arithmetic doesn't reveal anything about retained learning or track progress? > 2. They create perverse incentives that diminish students desire to learn No. They create incentives where none exists. The perversion is not the grade, testing, etc. The perversion is the system itself. Where kids are taken from their families to given to stranger to educate in an educational factory. > 3. They emphasize evaluation over iteration. Iteration is by far the more important skill - both pragmatically, for building knowledge, and for emotional development (grit). How do you determine the need for iteration unless you get some sort of feedback ( aka grade ). If you get a C in a test, that should indicate to the parent/student/teacher that the student needs to go over the material. > * progress is non-linear & child specific * learning isn’t about task completion I agree with the former. But task completion is most definitely part of learning. > Grade do serve one specific kind of student persona - those who optimize getting tasks done & getting good grades for it. You conveniently left out "learning". Those who optimize getting tasks done and learning and therefore getting good grades. How do you get good grades? By learning the subject matter.

alexvoda 5y ago
> So getting an A or a C in arithmetic doesn't reveal anything about retained learning or track progress? Yes, because Goodhart's law is applied. "When a measure becomes a target, it ceases to be a good measure". Many of those getting an A in arithmetics will have learned "how to get an A" instead of "how to practice and use arithmetics". In Romania, where school is very focused on grades at all levels, pupils really do actually learn how to get the grade instead of how to think. And the actual content is forgotten after a week or a month.
- qiskit 5y ago
  > Yes, because Goodhart's law is applied. "When a measure becomes a target, it ceases to be a good measure". It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. > Many of those getting an A in arithmetics will have learned "how to get an A" instead of "how to practice and use arithmetics". That's just a reflection of corruption in the system and not grading itself. > In Romania, where school is very focused on grades at all levels, pupils really do actually learn how to get the grade instead of how to think. No school anywhere exists to teach you how to think. But that's besides the point. Does the grade reflect the student's understanding of the material? > And the actual content is forgotten after a week or a month. Most content is forgotten regardless of the education system. So how do kids in romania learn to get good grades? Bribery? That doesn't show that grades are bad, but that the corruption is bad.
  - olliej 5y ago
    > It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. This is objectively false in the US education system at least. If you look at the training for standardized tests (the training that costs money, but is fundamentally necessary if you want to get into a good school) a huge amount of it has nothing to do with the material, and everything to do with how to rapidly exclude incorrect answers. All of that exclusion is based on various numeric tricks or filters with no actual understanding of why. Moreover standardized tests in the US explicitly encourage that behaviour by being ridiculously time constrained, to the extent that if you do not get training in how to *quickly* answer questions, no amount of understanding the material matters.
    - solveit 5y ago
      How much time do people actually spend on this training? I am dead certain that the bulk of the time is spent on actually learning the material and not on gaming the test. This is because there are only so many tricks you can learn, and you can learn them all in a week. > (the training that costs money, but is fundamentally necessary if you want to get into a good school This is objectively false. Source: I went to multiple good schools and met a boatload of people who also got in without said training. Also, that test-taking training is an overhyped and overpriced collection of cheap common sense strategies. You can learn how to do well on a multiple choice test in 20 seconds. You always pick the most likely answer. Strategies specific to things like the SATs boil down to memorizing vocab and doing a hundred practice tests. You can do this for a hundred bucks by buying a couple test prep books over Amazon. The part of the test-taking training that costs thousands of dollars is where they get a teacher with a fancy degree to babysit unmotivated children into actually doing the work. > Moreover standardized tests in the US explicitly encourage that behaviour by being ridiculously time constrained, to the extent that if you do not get training in how to quickly answer questions, no amount of understanding the material matters. There is actual academic research studying this, and shockingly understanding correlates strongly with being able to quickly answer questions. It's unfortunate that we can't assess millions of children by having them take time-unconstrained oral exams or projects, but the effect of training is grossly overstated. Show me any intervention (other than actually learning the material) that boosts your SAT score by more than ~30 points.
- solveit 5y ago
  When a measure becomes a target, the measure will be optimized at the cost of the actual target if the two diverge. If the measure is close enough to the target that the easiest way to optimize the measure is to optimize the target, then you'll be fine. If the easiest way to get an A in arithmetic is to know arithmetic, then students will learn arithmetic. There is a reason the previous poster chose arithmetic as an example. It is very difficult to pretend to know arithmetic without knowing arithmetic. Can Romanians add, subtract, multiply, and divide?
  - alexvoda 5y ago
    Thinking you can keep the two aligned is naive. There absolutely are ways to learn to pass arithmetic tests without learning to use arithmetic. It all depends on the type of test. During an oral test, 1-1 with the teacher, is close to impossible to fake knowledge. But with written tests, there are plenty of ways. Cheating (rampant, wide spread, by the way) in its various forms (asking a colleague, having a cheat sheet, getting the answers from someone who took the test earlier, etc.) is the most obvious and easy way but let's exclude it. A student can just memorize the exercises because the teacher didn't bother to create exercises unique to the test and reused them. Or reused them from a previous test. If it is a single answer multiple choice, a student might simply develop a bullshit detector and learn to spot suspect options. It is a wonderfully useful skill for test taking when the test is not antagonistic and intentionally trying to trip you up. There are so so so many options.
    - chii 5y ago
      all of the issues you mentioned pertain to the method of testing. It's pretty well known that you need to have some certain quality in the tests - after all, the result of an experiment is only as good as the design of the experiment, and this is true in a testing context. But grading via a test isn't a problem here. A good test will show a correct grade, and a student who dont know the material, and cannot cheat, will not get a good grade. This grade is a proxy to the learning that took place - memorization is a part of learning. Of course, there's more to learning than _just_ memorization, such as creativity and problem solving (novel problem solving). These areas are more difficult to test (not impossible), and the metric is difficult to define for these areas, but none of that invalidates the tests that grade a student.
- mattcwilson 5y ago
  I see your point about the possibility for a student getting an A to have perhaps gamed the system. Are you saying the same is true for (some of, all of) the C students? Are you saying that C students may in some cases be superior to the students who got As (including the “didn’t game the system” ones)?
  - alexvoda 5y ago
    First of all, I do not believe cramming for an exam and forgetting everything afterwards is "gaming the system". This is what the system encourages and the system is broken. Cheating might be called "gaming the system", but unfortunately it is equally encouraged by the system. There are students who will find ways to get a top grade and subsequently retain none of the knowledge. There are students who will actually learn everything and only bother to get a passing grade because the theatre of school is boring to them. There are also sudents who will find ways to get a passing grade, without having gained any knowledge at all, and they will be content with that. It is a mistake to lump any of these categories together. I know people in all of them, and depending on the subject, I was also in all of them. The affinity a student has for a teacher is a major, maybe even the most important, factor in determining the ammount of motivation and interest a student will show for a certain subject. And I believe this stays true at all levels.
neon_electro 5y ago
>How do you determine the need for iteration unless you get some sort of feedback ( aka grade ). Did you not read the article, where the author describes how taking grades off of her own plate allowed her to "[write] meaningful comments, [suggest] improvements, [ask] questions and [enter] into a dialogue with my students that felt more productive – that felt, in short, more like an extension of the classroom."
- mattcwilson 5y ago
  Strange to assume GP didn’t read the article. I did, and I have these similar questions: 1. How do I, as a student receiving this teacher feedback, know how seriously to take it? How do I know when it indicates I should spend an additional revision cycle on it, versus on another assignment (possibly from another teacher)? 2. Assuming I can figure out how to evaluate severity of the feedback and the relative amount of improvement I should do, how is this better than just a grade letter telling me how much further I can (or should) go? 3. The class still culminates in a letter grade, no? So doesn’t this create a potentially super-high-stakes environment, where I need to be damned sure I understand what the cumulative feedback I’ve received means, in terms of final letter grade? This honestly reminds me of the phenomenon of doing end-of-year self assessments on a 1-5 scale and having to turn that in to a manager before receiving the “real” 1-5 scale evaluation from them. Massively long feedback loops, massively high stakes and opportunity for misalignment. Why the heck would you not want early and frequent evaluation? Maybe it would help for me to understand specific examples of the assignment feedback given to students. I just see way too much risk for students to hear “you’re fine; a couple rough spots, but everyone has those, no big deal” only to flunk the class at the end of the year in a massive surprise.

by qiskit 5y ago

alexvoda 5y ago

> So getting an A or a C in arithmetic doesn't reveal anything about retained learning or track progress? Yes, because Goodhart's law is applied. "When a measure becomes a target, it ceases to be a good measure". Many of those getting an A in arithmetics will have learned "how to get an A" instead of "how to practice and use arithmetics". In Romania, where school is very focused on grades at all levels, pupils really do actually learn how to get the grade instead of how to think. And the actual content is forgotten after a week or a month.

qiskit 5y ago
> Yes, because Goodhart's law is applied. "When a measure becomes a target, it ceases to be a good measure". It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. > Many of those getting an A in arithmetics will have learned "how to get an A" instead of "how to practice and use arithmetics". That's just a reflection of corruption in the system and not grading itself. > In Romania, where school is very focused on grades at all levels, pupils really do actually learn how to get the grade instead of how to think. No school anywhere exists to teach you how to think. But that's besides the point. Does the grade reflect the student's understanding of the material? > And the actual content is forgotten after a week or a month. Most content is forgotten regardless of the education system. So how do kids in romania learn to get good grades? Bribery? That doesn't show that grades are bad, but that the corruption is bad.
- olliej 5y ago
  > It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. This is objectively false in the US education system at least. If you look at the training for standardized tests (the training that costs money, but is fundamentally necessary if you want to get into a good school) a huge amount of it has nothing to do with the material, and everything to do with how to rapidly exclude incorrect answers. All of that exclusion is based on various numeric tricks or filters with no actual understanding of why. Moreover standardized tests in the US explicitly encourage that behaviour by being ridiculously time constrained, to the extent that if you do not get training in how to *quickly* answer questions, no amount of understanding the material matters.
  - solveit 5y ago
    How much time do people actually spend on this training? I am dead certain that the bulk of the time is spent on actually learning the material and not on gaming the test. This is because there are only so many tricks you can learn, and you can learn them all in a week. > (the training that costs money, but is fundamentally necessary if you want to get into a good school This is objectively false. Source: I went to multiple good schools and met a boatload of people who also got in without said training. Also, that test-taking training is an overhyped and overpriced collection of cheap common sense strategies. You can learn how to do well on a multiple choice test in 20 seconds. You always pick the most likely answer. Strategies specific to things like the SATs boil down to memorizing vocab and doing a hundred practice tests. You can do this for a hundred bucks by buying a couple test prep books over Amazon. The part of the test-taking training that costs thousands of dollars is where they get a teacher with a fancy degree to babysit unmotivated children into actually doing the work. > Moreover standardized tests in the US explicitly encourage that behaviour by being ridiculously time constrained, to the extent that if you do not get training in how to quickly answer questions, no amount of understanding the material matters. There is actual academic research studying this, and shockingly understanding correlates strongly with being able to quickly answer questions. It's unfortunate that we can't assess millions of children by having them take time-unconstrained oral exams or projects, but the effect of training is grossly overstated. Show me any intervention (other than actually learning the material) that boosts your SAT score by more than ~30 points.
solveit 5y ago
When a measure becomes a target, the measure will be optimized at the cost of the actual target if the two diverge. If the measure is close enough to the target that the easiest way to optimize the measure is to optimize the target, then you'll be fine. If the easiest way to get an A in arithmetic is to know arithmetic, then students will learn arithmetic. There is a reason the previous poster chose arithmetic as an example. It is very difficult to pretend to know arithmetic without knowing arithmetic. Can Romanians add, subtract, multiply, and divide?
- alexvoda 5y ago
  Thinking you can keep the two aligned is naive. There absolutely are ways to learn to pass arithmetic tests without learning to use arithmetic. It all depends on the type of test. During an oral test, 1-1 with the teacher, is close to impossible to fake knowledge. But with written tests, there are plenty of ways. Cheating (rampant, wide spread, by the way) in its various forms (asking a colleague, having a cheat sheet, getting the answers from someone who took the test earlier, etc.) is the most obvious and easy way but let's exclude it. A student can just memorize the exercises because the teacher didn't bother to create exercises unique to the test and reused them. Or reused them from a previous test. If it is a single answer multiple choice, a student might simply develop a bullshit detector and learn to spot suspect options. It is a wonderfully useful skill for test taking when the test is not antagonistic and intentionally trying to trip you up. There are so so so many options.
  - chii 5y ago
    all of the issues you mentioned pertain to the method of testing. It's pretty well known that you need to have some certain quality in the tests - after all, the result of an experiment is only as good as the design of the experiment, and this is true in a testing context. But grading via a test isn't a problem here. A good test will show a correct grade, and a student who dont know the material, and cannot cheat, will not get a good grade. This grade is a proxy to the learning that took place - memorization is a part of learning. Of course, there's more to learning than _just_ memorization, such as creativity and problem solving (novel problem solving). These areas are more difficult to test (not impossible), and the metric is difficult to define for these areas, but none of that invalidates the tests that grade a student.
mattcwilson 5y ago
I see your point about the possibility for a student getting an A to have perhaps gamed the system. Are you saying the same is true for (some of, all of) the C students? Are you saying that C students may in some cases be superior to the students who got As (including the “didn’t game the system” ones)?
- alexvoda 5y ago
  First of all, I do not believe cramming for an exam and forgetting everything afterwards is "gaming the system". This is what the system encourages and the system is broken. Cheating might be called "gaming the system", but unfortunately it is equally encouraged by the system. There are students who will find ways to get a top grade and subsequently retain none of the knowledge. There are students who will actually learn everything and only bother to get a passing grade because the theatre of school is boring to them. There are also sudents who will find ways to get a passing grade, without having gained any knowledge at all, and they will be content with that. It is a mistake to lump any of these categories together. I know people in all of them, and depending on the subject, I was also in all of them. The affinity a student has for a teacher is a major, maybe even the most important, factor in determining the ammount of motivation and interest a student will show for a certain subject. And I believe this stays true at all levels.

neon_electro 5y ago

>How do you determine the need for iteration unless you get some sort of feedback ( aka grade ). Did you not read the article, where the author describes how taking grades off of her own plate allowed her to "[write] meaningful comments, [suggest] improvements, [ask] questions and [enter] into a dialogue with my students that felt more productive – that felt, in short, more like an extension of the classroom."

mattcwilson 5y ago
Strange to assume GP didn’t read the article. I did, and I have these similar questions: 1. How do I, as a student receiving this teacher feedback, know how seriously to take it? How do I know when it indicates I should spend an additional revision cycle on it, versus on another assignment (possibly from another teacher)? 2. Assuming I can figure out how to evaluate severity of the feedback and the relative amount of improvement I should do, how is this better than just a grade letter telling me how much further I can (or should) go? 3. The class still culminates in a letter grade, no? So doesn’t this create a potentially super-high-stakes environment, where I need to be damned sure I understand what the cumulative feedback I’ve received means, in terms of final letter grade? This honestly reminds me of the phenomenon of doing end-of-year self assessments on a 1-5 scale and having to turn that in to a manager before receiving the “real” 1-5 scale evaluation from them. Massively long feedback loops, massively high stakes and opportunity for misalignment. Why the heck would you not want early and frequent evaluation? Maybe it would help for me to understand specific examples of the assignment feedback given to students. I just see way too much risk for students to hear “you’re fine; a couple rough spots, but everyone has those, no big deal” only to flunk the class at the end of the year in a massive surprise.