6 ms·

After reading yummyfajitas' comment, I wish I could take back my upvote, because he's right: it's one thing to say the structure of the process itself is so byz

After reading yummyfajitas' comment, I wish I could take back my upvote, because he's right: it's one thing to say the structure of the process itself is so byzantine and brittle that schools suck, but it's quite another to say we should remove measurable goals as well. If I didn't know better, I'd call this a disinformation piece: deliberately conflating two concepts into one large rant in order to try to join the two. The subtext goes something like this:we know Joe Sixpack has finally figured out how terrible the education system is, so let's throw together a big pile of things we don't like, stick a label on top (something like "bureaucracy") and that way we can get rid of pesky measurements at the same time we free up execution.) The author isn't good enough to pull that off, though. I think it's much more the case that it's just a populist rant from somebody who already has a big label he wants to stick on things that don't work. That is a non-starter. To continue the programming analogy, as a customer you guys are delivering crap all the time. I pay ten times as much and the crap just gets worse. Now we can sit around and dicker all day long about the process you have for making crap, but one thing we are not going to do is remove the testing at the end that makes it obvious to everybody just how shitty the whole thing is.

cynicalkane 15y ago
The thing about measurable goals in teaching is that all of them are awful. Economic psychology teaches us that bad incentives are worse than no incentives. The incentives being promulgated are things like "higher test scores" or "more kids graduating" so the system aligns itself towards teaching the test while simultaneously lowering the bar for everything else. So let's talk about the software analogy. In software, the bar for formal acceptance usually really low. Do test cases pass? Maybe there's an auto-integration test that also must pass, and some requirements checkout QA has to do. You can write terrible code and have it pass the formal process, which is why you see a lot of terrible programmers coasting along at some of the worse companies. Some teams might have more formal requirements but that goes team by team. If you're working anywhere worth working, there'll be a much higher non-formal bar set by your manager, your team, your customers if you're in contact with them. So the incentives are there but they're personal and informal. The 'waterfall' pattern is one example of a misguided attempt to formalize the software dev process and align it with goals. How would software devs feel if some perverse, untouchable bureaucracy measured their code according to ridiculous formal measures from the Central Bureaucracy and arbitrary rules that changed according political whims? That's what teachers are facing. I know some software devs face the same thing, but we always give them the same advice: find a better job. Teachers don't have that option because the bureaucracy is everywhere; but they can go to better (generally richer) schools, where it's a little mitigated, or they can drop out and leave that gaggle of little Kafkas behind forever.
- DanielBMarkham 15y ago
  "...The thing about measurable goals in teaching is that all of them are awful..." I learned to fly people around in airplanes using various instructional techniques, including at times studying for the test. So do most airline pilots. Seems to work for them. The military has been training people for over 200 years with measurable teaching goals. In fact, some of the best teachers are ex-military. I could continue if you'd like. "...if you're working anywhere worth working--there'll be a much higher non-formal bar set by your manager, your team, your customers if you're in contact with them..." You are throwing out all requirements from any outside bureaucratic structure and then assuming that some informal standard of quality is going to fill the void. I don't know what your experiences are, but this kind of "let's see what happens" mentality doesn't trend toward good things. It's a tragedy of the commons problem. Why do a little bit extra here if somebody else in the organization is going to look at it anyway? Instead, we find that really smart people are great at arguing on and on about technical stuff that has nothing to do with delivery. The only thing that naturally limits this is having testable work due on a fixed date. Otherwise, people would much rather create little complex systems that mirror their favorite technical book or management philosophy. There are no fights that are as vicious as academic fights over the smallest things. The answer here is simple: if it's not measurable, we do not pay for it. I am sure there are lots of teams of really great people leading fulfilling lives creating quality in an ad-hoc manner that will continue doing so. We call these folks artists. There are also people who live the same kind of lives but have measurable results at the end of their creative process. These folks are usually called artistic professionals. Teachers cannot be artists. The public cannot afford them to be so. They can certainly be artistic professionals, but like other artistic professionals there will be some kind of loose structure around their work product and it will be judged at regular intervals. This measure will be centered solely around students, which is natural because the only reason any teacher would be paid would be because of a student. There might be a great argument about how the current tests are bad. But I think that's another discussion. As a side note, and I want to make sure I hit this the right way, I do not care about teachers as such. Yes, I care about the dignity and humanity and kindness we show to all people, but teachers do not get some special place above, say, policemen. Or doctors. They're just like the rest of us. If you try to split up each profession and say we should treat them all in special ways due to the nature of their jobs, it gets much too complex for me to manage. Instead, when I meet somebody we pay our taxes to help us out, I usually try to say "thank you" It doesn't matter to me whether we employ one teacher or ten million. If we could teach all of the students of the country with one teacher, that would be fine with me. The same goes for those other professions.
lemmsjid 15y ago
If programming worked the way the education system worked, it would be more like this: As a customer, you pay a company to build software. Due to federal law the programmers must fill out paperwork for (arbitrarily) each class they build in Java (the universally mandated language accepted as the best compromise by the government, after no small amount of backdoor lobbying by Oracle). The resulting code must pass mandated federal unit testing standards of "100% code coverage" due to the No Line of Code Left Behind Act. (Various measures of quality were discussed on the federal level, but things like overall code complexity, terseness, flexibility, etc. didn't resonate with the public as much as the slogan, "100% Coverage--you want it for your car insurance, why not your software?", which got several freshmen congressmen elected) In order to ensure compliance the programmers must run a special tool that verifies the coverage. Programmers who don't achieve 100% coverage have their pay docked and/or face dismissal. Shops that consistently fail to achieve 100% coverage face closure. As a customer you're looking for a one-off administrative web application. In several years it's delivered (along with a warehouse of paperwork) in all of its 100% unit tested glory. As required by Federal Web Compliance 625, it uses EJB and JSP against an Oracle database. One of the tables has data duplication, requiring Federal Normalization Exemption 835. It's so late because there was a brief controversy during development. One of the developers had increased his productivity by using a code generator to achieve 100% coverage. Suddenly he had a lot of time on his hands--he spent it refactoring the code, building prototypes, knocking out requirements, etc. Due to management oversight his transgression wasn't identified for several months, and the mysterious increase in productivity was hailed by local politicians as a shining example of how the system is working. Once identified, he was escorted out of the building, and a new developer had to rewrite his code from scratch. There was a brief splash in the local papers but that blew over, thank god... During the meeting in which the resulting steaming pile of crap is delivered to you, the customer, one of the programmers breaks down crying. You can't put your arm around him and comfort him because it's against the law.
- Tloewald 15y ago
  It's a nice story, but in reality (a) the code would never be delivered and (b) the criteria would be poorly designed so that something other than the desired result would pass the criteria, e.g. you're required to write unit tests and the program has to pass them but the unit tests don't actually need to exercise the features of the program in a meaningful way.
ylem 15y ago
Perhaps it might be worth examining one metric: IMPACT--championed by Rhee in Washington DC public schools and also used by the LA times to judge teachers publicly. Here's the basic idea--one should try to determine the "value added" of a teacher. It's not fair to look at a teacher teaching the gifted class and the teacher teaching a remedial class and look at their absolute results. However, one could look at the test scores of students before they come into a class and after they leave a class and compare. If the teacher adds value, then one would expect the scores to go up. Because an individual student may have a bad day, you average the incremental gains across multiple students. What are the problems with this? It seems rather fair right? A few problems: 1) The teacher in the previous year cheated on the test results (or the test changed), so that the students scored unnaturally high in the previous years 2) Gains may be unevenly distributed--is it as easy to move a student from 90% to 95% as it is to move a student from 20% to 40%? 3) Stability of the measure--if you look at this metric over say 5-10 years, is it consistent for a given teacher, or does it fluctuate? 4) If you run the metric backwards in time, is it still predictive? That is, if we look at this year's results, can we predict what the student would have achieved in the past? 5) What about background issues? This in some sense assumes that the background of student achievement is flat in the absence of a teacher--but middle class parents will spend a lot of time working with their kids, that poorer parents (on average) do not. This means that even if the teacher does nothing, the middle class students will likely still perform better. If these students are not equally distributed (which they won't be), this will also effect the metric. I could go on, but my point is that this is one of the more objective methods of valuing student performance and it fails--I think there are several problems with schools--I think that professionalizing the teaching profession would go a long way in predominately middle class districts--for example, requiring teachers to obtain a masters, offering higher salaries to recruit more talented teachers into the field, requiring subject matter competency (my family has a friend who's background is teaching middle school math who got drafted into teaching geography! With relatively little time given in advance to prep.), etc. I think setting curriculum standards and tests that reflect them and allowing the teachers autonomy in how they teach the required subjects would be useful. Generally, promoting more understanding and less drill and kill--but just saying, "throw the whole thing out" is not terribly useful. There's a fun read called "Surpassing Shanghai" which offers a brief review of education systems in other countries that is worth reading...
mattchew 15y ago
> it's one thing to say the structure of the process itself is so byzantine and brittle that schools suck, but it's quite another to say we should remove measurable goals as well. What good is it to have measurable goals if you are measuring the wrong thing, or your measurements are corrupt, or there is no effective means of correction if goals are not met? > To continue the programming analogy, as a customer you guys are delivering crap all the time. I pay ten times as much and the crap just gets worse. Now we can sit around and dicker all day long about the process you have for making crap, but one thing we are not going to do is remove the testing at the end that makes it obvious to everybody just how shitty the whole thing is. Keep it in then. Your programming team can learn to game the system, and your management team can quietly lobby to change the metrics so the numbers look better, and you can even bring in a pair of Bobs for a development cycle to shake things up, but none of that actually gets you to working software.

cynicalkane 15y ago

The thing about measurable goals in teaching is that all of them are awful. Economic psychology teaches us that bad incentives are worse than no incentives. The incentives being promulgated are things like "higher test scores" or "more kids graduating" so the system aligns itself towards teaching the test while simultaneously lowering the bar for everything else. So let's talk about the software analogy. In software, the bar for formal acceptance usually really low. Do test cases pass? Maybe there's an auto-integration test that also must pass, and some requirements checkout QA has to do. You can write terrible code and have it pass the formal process, which is why you see a lot of terrible programmers coasting along at some of the worse companies. Some teams might have more formal requirements but that goes team by team. If you're working anywhere worth working, there'll be a much higher non-formal bar set by your manager, your team, your customers if you're in contact with them. So the incentives are there but they're personal and informal. The 'waterfall' pattern is one example of a misguided attempt to formalize the software dev process and align it with goals. How would software devs feel if some perverse, untouchable bureaucracy measured their code according to ridiculous formal measures from the Central Bureaucracy and arbitrary rules that changed according political whims? That's what teachers are facing. I know some software devs face the same thing, but we always give them the same advice: find a better job. Teachers don't have that option because the bureaucracy is everywhere; but they can go to better (generally richer) schools, where it's a little mitigated, or they can drop out and leave that gaggle of little Kafkas behind forever.

DanielBMarkham 15y ago
"...The thing about measurable goals in teaching is that all of them are awful..." I learned to fly people around in airplanes using various instructional techniques, including at times studying for the test. So do most airline pilots. Seems to work for them. The military has been training people for over 200 years with measurable teaching goals. In fact, some of the best teachers are ex-military. I could continue if you'd like. "...if you're working anywhere worth working--there'll be a much higher non-formal bar set by your manager, your team, your customers if you're in contact with them..." You are throwing out all requirements from any outside bureaucratic structure and then assuming that some informal standard of quality is going to fill the void. I don't know what your experiences are, but this kind of "let's see what happens" mentality doesn't trend toward good things. It's a tragedy of the commons problem. Why do a little bit extra here if somebody else in the organization is going to look at it anyway? Instead, we find that really smart people are great at arguing on and on about technical stuff that has nothing to do with delivery. The only thing that naturally limits this is having testable work due on a fixed date. Otherwise, people would much rather create little complex systems that mirror their favorite technical book or management philosophy. There are no fights that are as vicious as academic fights over the smallest things. The answer here is simple: if it's not measurable, we do not pay for it. I am sure there are lots of teams of really great people leading fulfilling lives creating quality in an ad-hoc manner that will continue doing so. We call these folks artists. There are also people who live the same kind of lives but have measurable results at the end of their creative process. These folks are usually called artistic professionals. Teachers cannot be artists. The public cannot afford them to be so. They can certainly be artistic professionals, but like other artistic professionals there will be some kind of loose structure around their work product and it will be judged at regular intervals. This measure will be centered solely around students, which is natural because the only reason any teacher would be paid would be because of a student. There might be a great argument about how the current tests are bad. But I think that's another discussion. As a side note, and I want to make sure I hit this the right way, I do not care about teachers as such. Yes, I care about the dignity and humanity and kindness we show to all people, but teachers do not get some special place above, say, policemen. Or doctors. They're just like the rest of us. If you try to split up each profession and say we should treat them all in special ways due to the nature of their jobs, it gets much too complex for me to manage. Instead, when I meet somebody we pay our taxes to help us out, I usually try to say "thank you" It doesn't matter to me whether we employ one teacher or ten million. If we could teach all of the students of the country with one teacher, that would be fine with me. The same goes for those other professions.

lemmsjid 15y ago

If programming worked the way the education system worked, it would be more like this: As a customer, you pay a company to build software. Due to federal law the programmers must fill out paperwork for (arbitrarily) each class they build in Java (the universally mandated language accepted as the best compromise by the government, after no small amount of backdoor lobbying by Oracle). The resulting code must pass mandated federal unit testing standards of "100% code coverage" due to the No Line of Code Left Behind Act. (Various measures of quality were discussed on the federal level, but things like overall code complexity, terseness, flexibility, etc. didn't resonate with the public as much as the slogan, "100% Coverage--you want it for your car insurance, why not your software?", which got several freshmen congressmen elected) In order to ensure compliance the programmers must run a special tool that verifies the coverage. Programmers who don't achieve 100% coverage have their pay docked and/or face dismissal. Shops that consistently fail to achieve 100% coverage face closure. As a customer you're looking for a one-off administrative web application. In several years it's delivered (along with a warehouse of paperwork) in all of its 100% unit tested glory. As required by Federal Web Compliance 625, it uses EJB and JSP against an Oracle database. One of the tables has data duplication, requiring Federal Normalization Exemption 835. It's so late because there was a brief controversy during development. One of the developers had increased his productivity by using a code generator to achieve 100% coverage. Suddenly he had a lot of time on his hands--he spent it refactoring the code, building prototypes, knocking out requirements, etc. Due to management oversight his transgression wasn't identified for several months, and the mysterious increase in productivity was hailed by local politicians as a shining example of how the system is working. Once identified, he was escorted out of the building, and a new developer had to rewrite his code from scratch. There was a brief splash in the local papers but that blew over, thank god... During the meeting in which the resulting steaming pile of crap is delivered to you, the customer, one of the programmers breaks down crying. You can't put your arm around him and comfort him because it's against the law.

Tloewald 15y ago
It's a nice story, but in reality (a) the code would never be delivered and (b) the criteria would be poorly designed so that something other than the desired result would pass the criteria, e.g. you're required to write unit tests and the program has to pass them but the unit tests don't actually need to exercise the features of the program in a meaningful way.

ylem 15y ago

Perhaps it might be worth examining one metric: IMPACT--championed by Rhee in Washington DC public schools and also used by the LA times to judge teachers publicly. Here's the basic idea--one should try to determine the "value added" of a teacher. It's not fair to look at a teacher teaching the gifted class and the teacher teaching a remedial class and look at their absolute results. However, one could look at the test scores of students before they come into a class and after they leave a class and compare. If the teacher adds value, then one would expect the scores to go up. Because an individual student may have a bad day, you average the incremental gains across multiple students. What are the problems with this? It seems rather fair right? A few problems: 1) The teacher in the previous year cheated on the test results (or the test changed), so that the students scored unnaturally high in the previous years 2) Gains may be unevenly distributed--is it as easy to move a student from 90% to 95% as it is to move a student from 20% to 40%? 3) Stability of the measure--if you look at this metric over say 5-10 years, is it consistent for a given teacher, or does it fluctuate? 4) If you run the metric backwards in time, is it still predictive? That is, if we look at this year's results, can we predict what the student would have achieved in the past? 5) What about background issues? This in some sense assumes that the background of student achievement is flat in the absence of a teacher--but middle class parents will spend a lot of time working with their kids, that poorer parents (on average) do not. This means that even if the teacher does nothing, the middle class students will likely still perform better. If these students are not equally distributed (which they won't be), this will also effect the metric. I could go on, but my point is that this is one of the more objective methods of valuing student performance and it fails--I think there are several problems with schools--I think that professionalizing the teaching profession would go a long way in predominately middle class districts--for example, requiring teachers to obtain a masters, offering higher salaries to recruit more talented teachers into the field, requiring subject matter competency (my family has a friend who's background is teaching middle school math who got drafted into teaching geography! With relatively little time given in advance to prep.), etc. I think setting curriculum standards and tests that reflect them and allowing the teachers autonomy in how they teach the required subjects would be useful. Generally, promoting more understanding and less drill and kill--but just saying, "throw the whole thing out" is not terribly useful. There's a fun read called "Surpassing Shanghai" which offers a brief review of education systems in other countries that is worth reading...

mattchew 15y ago

> it's one thing to say the structure of the process itself is so byzantine and brittle that schools suck, but it's quite another to say we should remove measurable goals as well. What good is it to have measurable goals if you are measuring the wrong thing, or your measurements are corrupt, or there is no effective means of correction if goals are not met? > To continue the programming analogy, as a customer you guys are delivering crap all the time. I pay ten times as much and the crap just gets worse. Now we can sit around and dicker all day long about the process you have for making crap, but one thing we are not going to do is remove the testing at the end that makes it obvious to everybody just how shitty the whole thing is. Keep it in then. Your programming team can learn to game the system, and your management team can quietly lobby to change the metrics so the numbers look better, and you can even bring in a pair of Bobs for a development cycle to shake things up, but none of that actually gets you to working software.