4 ms·
Unfortunately, all the excuses in the world don't change the fact that these tests hurt education and hurt children. They also help, by identifying bad teacher
by yummyfajitas 15y ago
Unfortunately, all the excuses in the world don't change the fact that these tests hurt education and hurt children.
They also help, by identifying bad teachers and bad schools.
Do you have any data suggesting they hurt more than they help?
- keithpeter 15y agoIn the UK, we have spent ages and a lot of effort in testing, league tables, improvement plans, improvement targets for schools. There is a slowly growing realisation that the result may not be good for the children. See the Wolf report https://www.education.gov.uk/publications/standard/publicationDetail/Page1/DFE-00031-2011 https://www.education.gov.uk/publications/standard/publicati... and also a short OFSTED report into Maths teaching, see the newspaper summary below. The full report may be of interest to you as it explains the pedagogy of mathematics well. http://www.telegraph.co.uk/education/2982483/Ofsted-testing-culture-undermining-maths.html http://www.telegraph.co.uk/education/2982483/Ofsted-testing-... Teaching is a highly dimensional task. Assuming you could devise a metric space adequate to the task, my teaching at any point would be represented by a set of coordinates. The norm of the coordinates of my 'point' in the space might be higher or lower than the norm of another teacher. How do you decide which one of us is less 'bad'?
- yummyfajitas 15y agoYour first link, near as I can tell, doesn't even attempt to address the question of whether standardized tests are beneficial or harmful. It seems to be about the merits of vocational vs higher education. Your second link provides no data on whether testing is good for children. It merely shows that actual teaching methods do not conform to what the author's believe are the best teaching methods. No data is provided on student outcomes. Teaching is a highly dimensional task...How do you decide which one of us is less 'bad'? Any goal-oriented system is designed to maximize some arbitrary objective function. With standardized tests, you are forced to write down your objective function and admit it's an arbitrary choice. What benefit do we receive from having an unspecified objective function and no uniform method of measurement?
- keithpeter 15y agoLink 1: Appendix VIII talks about maths assessment, but I accept that this reference may be too UK specific to be useful anywhere else. It fits into a discourse about syllabus content &c Link 2: Outcomes are improving year on year, but OFSTED found that understanding of Maths is decreasing. Implication is that outcome measures are not appropriate "What benefit do we receive from having an unspecified objective function and no uniform method of measurement?" Children who can think for themselves. I think we may have an example of paradigm incommensurability here. You are seeing some kind of Goals -> Measuring Instrument -> Optimisation system, I'm seeing a political system with a lot of stakeholders and children that need to learn. As a practitioner, I find it hard to abstract from the daily process of meeting the needs of a very diverse student group. We may be talking past each other.
- yummyfajitas 15y agoI read Appendix VIII, I really can't see where it shows math assessments have harmed outcomes. Nor can I figure out where your second link shows math understanding is decreasing. Could you maybe quote the text you feel shows this? I must be missing something here. Children who can think for themselves. How do you know you get this without uniform tests? And for that matter, what does it even mean? As for me, I'm well aware of political realities. I just don't see how avoiding careful measurement, clearly stated objects and transparency helps a political system give better results.
- keithpeter 15y agoThe second link summarises an OFSTED report into the teaching of mathematics. The report suggests that teaching to the test is a problem in some UK schools. The report says that teaching to the test damages understanding in maths. I accept that the short abstract I linked to may not provide evidence of those things. "How do you know you get this without uniform tests?" The health or otherwise of our small companies, and the creativeness of artists, musicians, mathematicians, and the vigorousness of our politics. "And for that matter, what does it even mean?" It means what it has always meant! It means young adults who can take responsibility for themselves and others and who can act in society. Seriously, there is a level where one has to simply point rather than define (Wittgensteinish argument). "I just don't see how avoiding careful measurement, clearly stated objects and transparency helps a political system give better results." Because there are different stakeholders, and each part of your process will be challenged, and interpreted differently by some, and others will 'collapse' the wider concept of education down to a narrow focus on measurable outcomes. Not sure if I'm making sense here because I'm in a different place I suspect.
- almost 15y agoBut that's the thing, they don't identify bad teachers. And they do penalize good teachers. Being taught to pass these tests is not the same as being educated. All the tests do is identify teachers who don't put the majority of their teaching effort into getting children to pass the tests, to the detriment of everything that matters.
- yummyfajitas 15y agoBeing taught to pass these tests is not the same as being educated. How do they differ? And once you have a definition of "being educated", why not simply measure this? Once you change the tests to a measurement of "being educated", "passing the test" and "being educated" would be identical.
- kenjackson 15y agoAt the extreme, I can be taught to identify metadata in the test and use that to answer the question. E.g., if the question begins with "Why ..." the answer is 95% (c). If the question is an inequality with an irrational number than answer is almost always (b). Princeton Review was particular good at teaching metadata identification in helping to "guess" when you didn't know the answer.
- Drbble 15y agoPrinceton Review teaches students to use logic and some knowledge to exclude wrong answers, to help the top 10ish percent raise their scores. Students who are capable of using these techniques are much more intelligent/education than students at risk for failing state tests.
- yummyfajitas 15y agoE.g., if the question begins with "Why ..." the answer is 95% (c) This sounds like a job for random.shuffle. Incidentally, independent (i.e., not funded by Kaplan) investigations of test prep centers show minimal effect on performance. http://online.wsj.com/article/SB124278685697537839.html http://online.wsj.com/article/SB124278685697537839.html
- DanBC 15y agoDo they identify bad schools? Or do they identify schools with children that are harder to teach? This is a genuine question. I'm also a bit disappointed about the lack of rigorous evidence base with education. (Well, with everything, really.) In the UK the Department for Families and Schools has a lot of power over education. In theory that should help with an evidence base because they should be flowing good research down through to schools. But there's such a political atmosphere about teaching that interference from government is often seen as unwelcome. And, sometimes, that attitude is correct be government is suggesting something that's stupid. There's also a problem that bad teachers are difficult to remove from teaching.
- yummyfajitas 15y agoDo they identify bad schools? Or do they identify schools with children that are harder to teach? Test results are a measurement of how well educated the student is at a point in time. That's all. You can measure the effect of a school by studying value added - how well a student performs after education compared to how well they performed before. Or similarly, how well a student performs compared to statistically similar students. You can determine that some subsets of students are harder to teach by correlating observable characters (parental income, race, gender, etc) with outcomes.
- DanBC 15y agoSo, carefully used, with a statistician doing analysis, they're useful. How often are they carefully used, with an honest statistician doing the analysis, and without politicians / reporters / etc mangling the numbers?
- yummyfajitas 15y agoTell you what - lets eliminate all government accountability, since we can't avoid politicians/reporters/etc mangling things. End the FOIA, CBO, GAO, etc? Shut down thomas.loc.gov, and stop reporting the federal budget? Or maybe we can accept that nothing is perfect, and stop letting perfection be the enemy of the good.
- kenjackson 15y agoThe data has proved to be virtually useless in assessing teachers. See http://garyrubinstein.teachforus.org/2012/02/26/analyzing-released-nyc-value-added-data-part-1/ http://garyrubinstein.teachforus.org/2012/02/26/analyzing-re... They're useful for showing students are underperforming. But you can't say anything about teacher quality. In fact I suggest if you took the teachers from the "worst schools" and put them in the "best schools" and vice-versa the results the following year wouldn't look noticeably different. Does that mean that teacher have no impact at all. I don't believe that. But I do think the large gaps in achievement are systemic problems more than the problem of teacher quality at certain schools.
- yummyfajitas 15y agoThe fact that some teacher can't see the correlations using an inappropriate plotting technique (he uses a scatterplot, he should use hexbin or other density plot) doesn't mean they aren't there. And the numbers he gathers and dismisses show the correlation actually does exist. A simple numerical example, where the correlation is guaranteed (i.e., I took y=x+noise): http://i.imgur.com/rrmUI.png http://i.imgur.com/rrmUI.png https://gist.github.com/2183927 https://gist.github.com/2183927 Most of the correlations he expects to find are present. They are noisy, but present. The fact that in one case, reality is "contrary to what every teacher in the world knows" just suggests maybe teachers don't have a great grip on reality. ...if you took the teachers from the "worst schools" and put them in the "best schools" and vice-versa the results the following year wouldn't look noticeably different...Does that mean that teacher have no impact at all? Not quite, but almost. It means that variation among existing teachers impact less than the size of measurement, and the current crop of teachers are basically interchangeable cogs.
- acdha 15y ago> Not quite, but almost. It means that variation among existing teachers impact less than the size of measurement, and the current crop of teachers are basically interchangeable cogs. Or that the students - and their parents - are a very important factor with significant variation and classes a relatively small sample size. Ask any teacher and they can tell stories about how different classes can be - and how much a school having a weak discipline policy will ensure that many students will have poorer classroom experiences because of one or two difficult students. It's not like we don't know how to do scientific measurements of complex systems but it's expensive and slow at a time when the political requirement is fast and cheap. It'd be awesome if school districts actually hired people with backgrounds doing serious statistical analysis or large-scale studies with human subjects but nobody is jumping to fund that.
- blahedo 15y ago> They also help, by identifying bad teachers and bad schools. Aside from the arguments posted in sibling comments about what's being measured, you also have a problem of unaligned incentives. In particular, your claim only has a chance of being true if the students are actually interested in scoring as high as they can on the exam. Even at the AP level (I teach at university level and interact with secondary teachers that teach smart, high-level students taking CS), there are students who have decided that they don't care about the subject, or maybe even like the subject but have no (perceived) benefit from a high score due to their chosen college not giving credit in that subject or whatever. Such students may leave their answer book blank, or doodle in it, or maybe just blast through for the easy points and finish early and not worry about thinking about it. This isn't even necessarily a particularly irrational choice on their part! But it's a strong argument why the exams shouldn't be used to evaluate the teacher or the school. In a lot of places the students aren't permitted to opt out of the exam, even if they don't care about it, but there's no penalty to the student for taking a dive on it (and any penalty you could try to assess would have false positives and false negatives and still not motivate many of the students with differently-aligned incentives). All of these problems are going to be a million times worse on a general-education primary- or secondary-level assessment than they are on AP exams.
- Retric 15y agoAnnual tests end up directly costing students a month of education time, I think you need to prove benefit and not the other way around. As to their value, collages still trust SAT's and GPA more than any of the state tests in large part because they are uniformly terrible. If you really want to test teachers then randomly assign 1-2 tests to each student, it's just as statistically valid and takes ~1/3th the time.
- yummyfajitas 15y agoWell, ultimately the point I'm making is that we can't prove anything about anything without standardized tests. Until we have a measurement procedure, all we are doing is groping around in the dark without any way of knowing if we are helping or harming things. It's similarly difficult to prove that clinical trials are beneficial in medicine. You might try to compare medicine developed with clinical trials to medicine developed without it, but how would you actually make such a comparison? Not with a clinical trial, obviously...
- Retric 15y agoThere are plenty of useful pieces of information that have nothing to do with standardized tests. Attendance, Graduation Rates, teen pregnancy, collage admittance, GPA etc. It's true that you could gain useful information from great standardized tests administered well. Unfortunately, we don't actually have any of them that are worth a damn. Each state has it's own tests which are developed alongside the curriculum. So, is Virginia doing better than Maryland, sorry the test's can't actually tell you that. Are graduates in 2010 better prepared than ones from 2000, sorry they can't tell you that. The idea of tests sounds good, but the implementation is so bad they are effectively worthless. Again, you can argue that great tests could mean something but we don't have them we just have crap. So, if you want to defend tests you need to defend that crap because that's the reality.
- yummyfajitas 15y agoAttendance, graduation rates and teen pregnancy are meaningless. They are measures of hours of butt-seat contact, low graduation standards and horniness, respectively. GPA is closer to a useful metric. GPA and college admittance are basically the same thing as standardized tests. Except that unlike standardized tests, you can't compare any set of grades to any other set of grades. How does that help? Incidentally, do you realize your biggest criticism of standardized tests is that they are not standardized enough? I.e., there is too much geographic and spatial variation in them?
- spamizbad 15y ago> They also help, by identifying bad teachers and bad schools. Not really. Here in Chicago, the Noble charter high schools will often teach directly to the ACT. This results in some dramatic boosting of ACT scores, with some schools taking kids from the dysfunctional CPS System and getting a school average of 23. Unfortunately, once they reach college, they see little academic success despite being straight-A students with 25+ ACT scores. This has a detrimental impact on the curriculum: 1) The English program structured heavily around basic reading comprehension, with little to no emphasis on writing composition. A students understanding of essay composition s roughly: "Organize the things you want to talk about into paragraphs... then write a conclusion." However, to their credit, they're really good at reading test questions. 2) Math is focused on teaching Pre-algebra fundamentals and then layering on test-specific Algebra, Geometric (with that goofy proof system), and basic Trig. It's a sad, narrow sample of our already sad & narrow HS math curriculum. It covers few "advanced" Algebra and Trig subjects. This means anyone who has to take "college math" will need courses in Trig and Precalc in college, with the possibility of an Algebra refresher course before proceeding. 3) Social Studies and Science? All rote-to-test. Students are drilled on step-by-step procedures on how to interpret graphs that'll score correctly on tests... without giving them actual knowledge on how to critically think about information - be it historical or scientific. NCLB schools do the same song-and-dance, except with a much less rigorous test. If you've actually seen the questions on most NCLB tests, you'd be disappointed. Unfortunately, the composition of such tests is so political and messy it's impossible to provide any measure of quality. You cannot assume testing will provide you accurate information or a better outcome for students. If you're going to implement a testing regiment, you need to be very mindful of the Observer Effect: You can very easily change the outcome by measuring it. This is not an easy problem to solve.
- yummyfajitas 15y agoYour issue is with the objectives of the school system, not testing. Tests measure whether a school meets it's objectives, they don't define the objectives. Similarly, if your manager writes a bunch of nonsensical unit tests, it's ridiculous to blame unit testing if you wind up building the wrong product.