7 ms·

> So getting an A or a C in arithmetic doesn't reveal anything about retained learning or track progress? Yes, because Goodhart's law is applied. "When a measu

> So getting an A or a C in arithmetic doesn't reveal anything about retained learning or track progress? Yes, because Goodhart's law is applied. "When a measure becomes a target, it ceases to be a good measure". Many of those getting an A in arithmetics will have learned "how to get an A" instead of "how to practice and use arithmetics". In Romania, where school is very focused on grades at all levels, pupils really do actually learn how to get the grade instead of how to think. And the actual content is forgotten after a week or a month.

qiskit 5y ago
> Yes, because Goodhart's law is applied. "When a measure becomes a target, it ceases to be a good measure". It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. > Many of those getting an A in arithmetics will have learned "how to get an A" instead of "how to practice and use arithmetics". That's just a reflection of corruption in the system and not grading itself. > In Romania, where school is very focused on grades at all levels, pupils really do actually learn how to get the grade instead of how to think. No school anywhere exists to teach you how to think. But that's besides the point. Does the grade reflect the student's understanding of the material? > And the actual content is forgotten after a week or a month. Most content is forgotten regardless of the education system. So how do kids in romania learn to get good grades? Bribery? That doesn't show that grades are bad, but that the corruption is bad.
- olliej 5y ago
  > It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. This is objectively false in the US education system at least. If you look at the training for standardized tests (the training that costs money, but is fundamentally necessary if you want to get into a good school) a huge amount of it has nothing to do with the material, and everything to do with how to rapidly exclude incorrect answers. All of that exclusion is based on various numeric tricks or filters with no actual understanding of why. Moreover standardized tests in the US explicitly encourage that behaviour by being ridiculously time constrained, to the extent that if you do not get training in how to *quickly* answer questions, no amount of understanding the material matters.
  - solveit 5y ago
    How much time do people actually spend on this training? I am dead certain that the bulk of the time is spent on actually learning the material and not on gaming the test. This is because there are only so many tricks you can learn, and you can learn them all in a week. > (the training that costs money, but is fundamentally necessary if you want to get into a good school This is objectively false. Source: I went to multiple good schools and met a boatload of people who also got in without said training. Also, that test-taking training is an overhyped and overpriced collection of cheap common sense strategies. You can learn how to do well on a multiple choice test in 20 seconds. You always pick the most likely answer. Strategies specific to things like the SATs boil down to memorizing vocab and doing a hundred practice tests. You can do this for a hundred bucks by buying a couple test prep books over Amazon. The part of the test-taking training that costs thousands of dollars is where they get a teacher with a fancy degree to babysit unmotivated children into actually doing the work. > Moreover standardized tests in the US explicitly encourage that behaviour by being ridiculously time constrained, to the extent that if you do not get training in how to quickly answer questions, no amount of understanding the material matters. There is actual academic research studying this, and shockingly understanding correlates strongly with being able to quickly answer questions. It's unfortunate that we can't assess millions of children by having them take time-unconstrained oral exams or projects, but the effect of training is grossly overstated. Show me any intervention (other than actually learning the material) that boosts your SAT score by more than ~30 points.
- alexvoda 5y ago
  > It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. Goodhart's law makes them become the end. It does not matter if the intended grades to only be a means. If they are what is measured, they will become the end. At least for some. > That's just a reflection of corruption in the system and not grading itself. I am not talking about cheating or bribing. I am talking about all ways to pass a test and not retain any knowledge after an hour/day/week/month. > Most content is forgotten regardless of the education system. It matters if that content (and especially the existance of that content) is forgotten in a hour to a month or if that content is forgotten in years. The longer the time, the higher the chance you will actually use it. And in many cases, if after years, only the knowledge about the existance of the content is retained, it is enough. You know it exists, so you know how to find it when you need it. But it is entirely possible to pass tests and remember nothing at all a short while after.
solveit 5y ago
When a measure becomes a target, the measure will be optimized at the cost of the actual target if the two diverge. If the measure is close enough to the target that the easiest way to optimize the measure is to optimize the target, then you'll be fine. If the easiest way to get an A in arithmetic is to know arithmetic, then students will learn arithmetic. There is a reason the previous poster chose arithmetic as an example. It is very difficult to pretend to know arithmetic without knowing arithmetic. Can Romanians add, subtract, multiply, and divide?
- alexvoda 5y ago
  Thinking you can keep the two aligned is naive. There absolutely are ways to learn to pass arithmetic tests without learning to use arithmetic. It all depends on the type of test. During an oral test, 1-1 with the teacher, is close to impossible to fake knowledge. But with written tests, there are plenty of ways. Cheating (rampant, wide spread, by the way) in its various forms (asking a colleague, having a cheat sheet, getting the answers from someone who took the test earlier, etc.) is the most obvious and easy way but let's exclude it. A student can just memorize the exercises because the teacher didn't bother to create exercises unique to the test and reused them. Or reused them from a previous test. If it is a single answer multiple choice, a student might simply develop a bullshit detector and learn to spot suspect options. It is a wonderfully useful skill for test taking when the test is not antagonistic and intentionally trying to trip you up. There are so so so many options.
  - chii 5y ago
    all of the issues you mentioned pertain to the method of testing. It's pretty well known that you need to have some certain quality in the tests - after all, the result of an experiment is only as good as the design of the experiment, and this is true in a testing context. But grading via a test isn't a problem here. A good test will show a correct grade, and a student who dont know the material, and cannot cheat, will not get a good grade. This grade is a proxy to the learning that took place - memorization is a part of learning. Of course, there's more to learning than _just_ memorization, such as creativity and problem solving (novel problem solving). These areas are more difficult to test (not impossible), and the metric is difficult to define for these areas, but none of that invalidates the tests that grade a student.
mattcwilson 5y ago
I see your point about the possibility for a student getting an A to have perhaps gamed the system. Are you saying the same is true for (some of, all of) the C students? Are you saying that C students may in some cases be superior to the students who got As (including the “didn’t game the system” ones)?
- alexvoda 5y ago
  First of all, I do not believe cramming for an exam and forgetting everything afterwards is "gaming the system". This is what the system encourages and the system is broken. Cheating might be called "gaming the system", but unfortunately it is equally encouraged by the system. There are students who will find ways to get a top grade and subsequently retain none of the knowledge. There are students who will actually learn everything and only bother to get a passing grade because the theatre of school is boring to them. There are also sudents who will find ways to get a passing grade, without having gained any knowledge at all, and they will be content with that. It is a mistake to lump any of these categories together. I know people in all of them, and depending on the subject, I was also in all of them. The affinity a student has for a teacher is a major, maybe even the most important, factor in determining the ammount of motivation and interest a student will show for a certain subject. And I believe this stays true at all levels.

qiskit 5y ago

> Yes, because Goodhart's law is applied. "When a measure becomes a target, it ceases to be a good measure". It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. > Many of those getting an A in arithmetics will have learned "how to get an A" instead of "how to practice and use arithmetics". That's just a reflection of corruption in the system and not grading itself. > In Romania, where school is very focused on grades at all levels, pupils really do actually learn how to get the grade instead of how to think. No school anywhere exists to teach you how to think. But that's besides the point. Does the grade reflect the student's understanding of the material? > And the actual content is forgotten after a week or a month. Most content is forgotten regardless of the education system. So how do kids in romania learn to get good grades? Bribery? That doesn't show that grades are bad, but that the corruption is bad.

olliej 5y ago
> It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. This is objectively false in the US education system at least. If you look at the training for standardized tests (the training that costs money, but is fundamentally necessary if you want to get into a good school) a huge amount of it has nothing to do with the material, and everything to do with how to rapidly exclude incorrect answers. All of that exclusion is based on various numeric tricks or filters with no actual understanding of why. Moreover standardized tests in the US explicitly encourage that behaviour by being ridiculously time constrained, to the extent that if you do not get training in how to *quickly* answer questions, no amount of understanding the material matters.
- solveit 5y ago
  How much time do people actually spend on this training? I am dead certain that the bulk of the time is spent on actually learning the material and not on gaming the test. This is because there are only so many tricks you can learn, and you can learn them all in a week. > (the training that costs money, but is fundamentally necessary if you want to get into a good school This is objectively false. Source: I went to multiple good schools and met a boatload of people who also got in without said training. Also, that test-taking training is an overhyped and overpriced collection of cheap common sense strategies. You can learn how to do well on a multiple choice test in 20 seconds. You always pick the most likely answer. Strategies specific to things like the SATs boil down to memorizing vocab and doing a hundred practice tests. You can do this for a hundred bucks by buying a couple test prep books over Amazon. The part of the test-taking training that costs thousands of dollars is where they get a teacher with a fancy degree to babysit unmotivated children into actually doing the work. > Moreover standardized tests in the US explicitly encourage that behaviour by being ridiculously time constrained, to the extent that if you do not get training in how to quickly answer questions, no amount of understanding the material matters. There is actual academic research studying this, and shockingly understanding correlates strongly with being able to quickly answer questions. It's unfortunate that we can't assess millions of children by having them take time-unconstrained oral exams or projects, but the effect of training is grossly overstated. Show me any intervention (other than actually learning the material) that boosts your SAT score by more than ~30 points.
alexvoda 5y ago
> It depends on whether the grade is the ends or not. But here the grades aren't the ends. It's just a means to achieve the ends - learning the material. Goodhart's law makes them become the end. It does not matter if the intended grades to only be a means. If they are what is measured, they will become the end. At least for some. > That's just a reflection of corruption in the system and not grading itself. I am not talking about cheating or bribing. I am talking about all ways to pass a test and not retain any knowledge after an hour/day/week/month. > Most content is forgotten regardless of the education system. It matters if that content (and especially the existance of that content) is forgotten in a hour to a month or if that content is forgotten in years. The longer the time, the higher the chance you will actually use it. And in many cases, if after years, only the knowledge about the existance of the content is retained, it is enough. You know it exists, so you know how to find it when you need it. But it is entirely possible to pass tests and remember nothing at all a short while after.

solveit 5y ago

When a measure becomes a target, the measure will be optimized at the cost of the actual target if the two diverge. If the measure is close enough to the target that the easiest way to optimize the measure is to optimize the target, then you'll be fine. If the easiest way to get an A in arithmetic is to know arithmetic, then students will learn arithmetic. There is a reason the previous poster chose arithmetic as an example. It is very difficult to pretend to know arithmetic without knowing arithmetic. Can Romanians add, subtract, multiply, and divide?

alexvoda 5y ago
Thinking you can keep the two aligned is naive. There absolutely are ways to learn to pass arithmetic tests without learning to use arithmetic. It all depends on the type of test. During an oral test, 1-1 with the teacher, is close to impossible to fake knowledge. But with written tests, there are plenty of ways. Cheating (rampant, wide spread, by the way) in its various forms (asking a colleague, having a cheat sheet, getting the answers from someone who took the test earlier, etc.) is the most obvious and easy way but let's exclude it. A student can just memorize the exercises because the teacher didn't bother to create exercises unique to the test and reused them. Or reused them from a previous test. If it is a single answer multiple choice, a student might simply develop a bullshit detector and learn to spot suspect options. It is a wonderfully useful skill for test taking when the test is not antagonistic and intentionally trying to trip you up. There are so so so many options.
- chii 5y ago
  all of the issues you mentioned pertain to the method of testing. It's pretty well known that you need to have some certain quality in the tests - after all, the result of an experiment is only as good as the design of the experiment, and this is true in a testing context. But grading via a test isn't a problem here. A good test will show a correct grade, and a student who dont know the material, and cannot cheat, will not get a good grade. This grade is a proxy to the learning that took place - memorization is a part of learning. Of course, there's more to learning than _just_ memorization, such as creativity and problem solving (novel problem solving). These areas are more difficult to test (not impossible), and the metric is difficult to define for these areas, but none of that invalidates the tests that grade a student.

mattcwilson 5y ago

I see your point about the possibility for a student getting an A to have perhaps gamed the system. Are you saying the same is true for (some of, all of) the C students? Are you saying that C students may in some cases be superior to the students who got As (including the “didn’t game the system” ones)?

alexvoda 5y ago
First of all, I do not believe cramming for an exam and forgetting everything afterwards is "gaming the system". This is what the system encourages and the system is broken. Cheating might be called "gaming the system", but unfortunately it is equally encouraged by the system. There are students who will find ways to get a top grade and subsequently retain none of the knowledge. There are students who will actually learn everything and only bother to get a passing grade because the theatre of school is boring to them. There are also sudents who will find ways to get a passing grade, without having gained any knowledge at all, and they will be content with that. It is a mistake to lump any of these categories together. I know people in all of them, and depending on the subject, I was also in all of them. The affinity a student has for a teacher is a major, maybe even the most important, factor in determining the ammount of motivation and interest a student will show for a certain subject. And I believe this stays true at all levels.