4 ms·

The Lesson to Unlearn

9 points by absolute100 5y ago

absolute100 5y ago
This, I think, is the incentive that will get generations to come to stop playing only for grades and diplomas: "I suspect many people implicitly assume that working in a field with bad tests is the price of making lots of money. But that, I can tell you, is false. It used to be true. In the mid-twentieth century, when the economy was composed of oligopolies, the only way to the top was by playing their game. But it's not true now. There are now ways to get rich by doing good work, and that's part of the reason people are so much more excited about getting rich than they used to be. When I was a kid, you could either become an engineer and make cool things, or make lots of money by becoming an executive. Now you can make lots of money by making cool things." -- how many of your software engineers are brilliant, eagerly learning new things and yet didn't complete (or start) their CS degree?
wayoverdone 5y ago
>> Hacking bad tests is becoming less important as the link between work and authority erodes. The erosion of that link is one of the most important trends happening now, and we see its effects in almost every kind of work people do. Startups are one of the most visible examples, but we see much the same thing in writing. Writers no longer have to submit to publishers and editors to reach readers; now they can go direct. This is extremely tech centric view of the world. If your work function cannot be sold with minimal marginal cost - e.g. technical service provider, consultants, or other service industry work - this strategy may not work for you. This is true for most of traditional engineering, or other non-tech STEM functions in orgs. As the cost of customer acquisition over net revenue will be significantly higher for you than the incumbent - so might as well join them - and to do so, you will likely need to hack the bad test.
Bostonian 5y ago
'All the tests most students take their whole lives are at least as bad, including, most spectacularly of all, the test that gets them into college.' If the SAT were so easily hackable by using the right test-taking techniques, rich parents would have hired tutors to teach their children the tricks, rather than paying people to actually take tests for them, which caused several famous people to go to jail.
- BeetleB 5y ago
  > rich parents would have hired tutors to teach their children the tricks, rather than paying people to actually take tests for them, which caused several famous people to go to jail. I would wager over 95% of rich people do hire tutors (or send children to expensive tutoring programs). Since they don't get sent to jail, we overestimate the percentage of the latter. The SAT is hackable. In my time, both the SAT and the GRE analogy questions had the same "flaw": You are given 5 choices. Two of them will always have identical relationships - so neither can be the answer. There is always a nonsensical choice (something like "Dog : mathematics"), so you can rule that out. You're left with two choices. Even if you don't understand the question, you now have a 50% chance of getting it right. Similarly, for quantitative questions, a number of problems are easily "solved" by working backwards. I don't need to know how to solve a quadratic: I just plug in the choices into the equation and see which gives the correct answer. This is so "common" that people consider the strategy as legitimate. However, the goal of the question is to see if you can solve a quadratic - not evaluate one. A lot of quantitative problems tend to have at least one nonsensical answer. You can rule it out. And so on.
  - tharkun__ 5y ago
    At my university someone ran all the available old exams of the standard first semester "filtering courses" in economics through some simple statistical tests. These were always 5 answer multiple choice exams where you had to copy your results from the actual test sheets onto a scanable matrix sheet. Long story short, if you simply checked answers A, C, E for every question you apparently had a very good chance to get a passing grade.
  - Blammar 5y ago
    Indeed. As a high school and college student, I was very good at "hacking" multiple choice tests. I did manage to crush my SATs (800's in the achievement tests), win local and state chemistry competitions, and came in second on the national college level math test (whose name I forgot, but it gets you an entry into the Actuary Guild.) I remember these, but I am actually most proud of being one of the top 100 scorers on the Putnam competition, definitely not a hackable test! Sadly, I didn't know Microsoft hired top Putnam scorers, no questions asked, at the time (1970), so I missed out on being a Word programmer. Oh, wait! That's a good thing!
innagadadavida 5y ago
There are a few key things here that seems to get conflated: 1. Usefulness - Is it a useful thing to study and remember - for an engineer medieval history is utterly useless. 2. Interest - is the student interested in the subject beyond grades and usefulness? 3. Signaling - you need good grades to get to the next stage in your life. 4. Energy and Time - students have other things that they want to do outside of school. Students are making a tradeoff on all these dimensions while optimizing for the grade - this in itself is no easy thing to do.

9 points by absolute100 5y ago

This, I think, is the incentive that will get generations to come to stop playing only for grades and diplomas: "I suspect many people implicitly assume that working in a field with bad tests is the price of making lots of money. But that, I can tell you, is false. It used to be true. In the mid-twentieth century, when the economy was composed of oligopolies, the only way to the top was by playing their game. But it's not true now. There are now ways to get rich by doing good work, and that's part of the reason people are so much more excited about getting rich than they used to be. When I was a kid, you could either become an engineer and make cool things, or make lots of money by becoming an executive. Now you can make lots of money by making cool things." -- how many of your software engineers are brilliant, eagerly learning new things and yet didn't complete (or start) their CS degree?

wayoverdone 5y ago

>> Hacking bad tests is becoming less important as the link between work and authority erodes. The erosion of that link is one of the most important trends happening now, and we see its effects in almost every kind of work people do. Startups are one of the most visible examples, but we see much the same thing in writing. Writers no longer have to submit to publishers and editors to reach readers; now they can go direct. This is extremely tech centric view of the world. If your work function cannot be sold with minimal marginal cost - e.g. technical service provider, consultants, or other service industry work - this strategy may not work for you. This is true for most of traditional engineering, or other non-tech STEM functions in orgs. As the cost of customer acquisition over net revenue will be significantly higher for you than the incumbent - so might as well join them - and to do so, you will likely need to hack the bad test.

Bostonian 5y ago

'All the tests most students take their whole lives are at least as bad, including, most spectacularly of all, the test that gets them into college.' If the SAT were so easily hackable by using the right test-taking techniques, rich parents would have hired tutors to teach their children the tricks, rather than paying people to actually take tests for them, which caused several famous people to go to jail.

BeetleB 5y ago
> rich parents would have hired tutors to teach their children the tricks, rather than paying people to actually take tests for them, which caused several famous people to go to jail. I would wager over 95% of rich people do hire tutors (or send children to expensive tutoring programs). Since they don't get sent to jail, we overestimate the percentage of the latter. The SAT is hackable. In my time, both the SAT and the GRE analogy questions had the same "flaw": You are given 5 choices. Two of them will always have identical relationships - so neither can be the answer. There is always a nonsensical choice (something like "Dog : mathematics"), so you can rule that out. You're left with two choices. Even if you don't understand the question, you now have a 50% chance of getting it right. Similarly, for quantitative questions, a number of problems are easily "solved" by working backwards. I don't need to know how to solve a quadratic: I just plug in the choices into the equation and see which gives the correct answer. This is so "common" that people consider the strategy as legitimate. However, the goal of the question is to see if you can solve a quadratic - not evaluate one. A lot of quantitative problems tend to have at least one nonsensical answer. You can rule it out. And so on.
- tharkun__ 5y ago
  At my university someone ran all the available old exams of the standard first semester "filtering courses" in economics through some simple statistical tests. These were always 5 answer multiple choice exams where you had to copy your results from the actual test sheets onto a scanable matrix sheet. Long story short, if you simply checked answers A, C, E for every question you apparently had a very good chance to get a passing grade.
- Blammar 5y ago
  Indeed. As a high school and college student, I was very good at "hacking" multiple choice tests. I did manage to crush my SATs (800's in the achievement tests), win local and state chemistry competitions, and came in second on the national college level math test (whose name I forgot, but it gets you an entry into the Actuary Guild.) I remember these, but I am actually most proud of being one of the top 100 scorers on the Putnam competition, definitely not a hackable test! Sadly, I didn't know Microsoft hired top Putnam scorers, no questions asked, at the time (1970), so I missed out on being a Word programmer. Oh, wait! That's a good thing!

innagadadavida 5y ago

There are a few key things here that seems to get conflated: 1. Usefulness - Is it a useful thing to study and remember - for an engineer medieval history is utterly useless. 2. Interest - is the student interested in the subject beyond grades and usefulness? 3. Signaling - you need good grades to get to the next stage in your life. 4. Energy and Time - students have other things that they want to do outside of school. Students are making a tradeoff on all these dimensions while optimizing for the grade - this in itself is no easy thing to do.