5 ms·

> “The report concluded that there were observable, repeatable and verifiable ways of measuring teacher effectiveness,” wrote Gates in the letter. Anonymous stu

> “The report concluded that there were observable, repeatable and verifiable ways of measuring teacher effectiveness,” wrote Gates in the letter. Anonymous student surveys that asked such questions as “Does your teacher use class time well, get class organized quickly, help you when you are confused" – were proven to provide useful feedback as were reports from trained professionals observing teachers at work. Students are notoriusly bad at rating their professors. This was proved with almost ideal control groups at the Air Force Academy [1] and again with groups of trained professionals/graduate students who learned first hand about the 'Dr. Fox Effect' [2]. Even teachers don't seem to like the teacher evaluations done by students [3]. Anecdotally, I can say that most students in my college classes either didn't show up on the survey days, or they walked out the door as soon as the surveys were being handed out. There's also no incentive to provide useful feedback from the student's perspective. If you're taking a survey about the class, it means the class is over and you'll probably never see that professor again, so why bother? I made an effort only because I felt an obligation to help future students, but I'm not sure there are many kids in college who share that feeling. > Mary Ann Stavney, a high school “Master Teacher” profiled in the annual letter, spends 70% of her time observing other teachers, meeting with them and providing input. The problem, of course, is that this kind of measuring, particularly the hands-on observation in classrooms, is costly, adding about 2% onto payroll. So you can have cheap and unreliable measurements, or you can have accurate but costly ones. Who's going to pay for the latter? The rating agencies? The schools? The students? Imagine the costs to enact such a program across all colleges in the U.S. alone -- some of the larger state schools easily have > 1,000 teaching faculty across a myriad of disciplines, and they're teaching increasingly diverse student bodies. The thought of trying to implement a thorough, standardized program of that scale is mind boggling. And that's probably why we've been facing this dilemma of measuring teacher effectiveness since the day the first schools opened. Bill Gates is right that we have a serious problem, but it doesn't sound like he's any closer to a solution. [1] http://voices.washingtonpost.com/college-inc/2010/06/study_high-rated_professors_ar.html http://voices.washingtonpost.com/college-inc/2010/06/study_h... [2] http://en.wikipedia.org/wiki/Dr._Fox_effect http://en.wikipedia.org/wiki/Dr._Fox_effect [3] http://en.wikipedia.org/wiki/Course_evaluation#Criticism_of_course_evaluations_as_measures_of_teaching_effectiveness http://en.wikipedia.org/wiki/Course_evaluation#Criticism_of_...

tokenadult 14y ago
There was an earlier Hacker News submission about the Gates Foundation research on teacher effectiveness, http://news.ycombinator.com/item?id=4559682 http://news.ycombinator.com/item?id=4559682 linking to an article that reported details of the methodology. http://www.theatlantic.com/magazine/archive/2012/10/why-kids-should-grade-teachers/309088/ http://www.theatlantic.com/magazine/archive/2012/10/why-kids... I looked up other research on the matter for the reply I posted in that thread. From the article submitted then, this is one way this process has been validated: "The responses did indeed help predict which classes would have the most test-score improvement at the end of the year. In math, for example, the teachers rated most highly by students delivered the equivalent of about six more months of learning than teachers with the lowest ratings. (By comparison, teachers who get a master’s degree—one of the few ways to earn a pay raise in most schools —delivered about one more month of learning per year than teachers without one.) . . . . "The survey did not ask Do you like your teacher? Is your teacher nice? This wasn’t a popularity contest. The survey mostly asked questions about what students saw, day in and day out. "Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward: 1. Students in this class treat the teacher with respect. 2. My classmates behave the way my teacher wants them to. 3. Our class stays busy and doesn’t waste time. 4. In this class, we learn a lot almost every day. 5. In this class, we learn to correct our mistakes." Here is earlier reporting (10 December 2010) from the New York Times about the same issue: http://www.nytimes.com/2010/12/11/education/11education.html http://www.nytimes.com/2010/12/11/education/11education.html Here is the website of Ronald Ferguson's research project at Harvard: http://tripodproject.wpengine.com/about/our-team/ http://tripodproject.wpengine.com/about/our-team/ And here are some links about the project from the National Center for Teacher Effectiveness: http://www.gse.harvard.edu/ncte/news/NCTE_Conference_Using_Student_Surveys.php http://www.gse.harvard.edu/ncte/news/NCTE_Conference_Using_S... Simply put, don't assume that what the Gates Foundation was investigating was the same kind of student opinion survey that I have filled out as a postsecondary student. (But note that I'm not so sure that those surveys are as bad or as useless as college faculty often claim they are.) There is a research base for the primary school pupil and secondary school student ratings used in the Gates Foundation studies, and I have every reason to believe those ratings would help school effectiveness--so much so that I use the same questions to invite my clients of my mathematics program to evaluate my teaching from that point of view. Other comments in this thread are about the more general issue of college rankings as they currently exist. As a parent who has occasion to look at my children's college search process for four children, I really like the site College Results http://www.collegeresults.org/search1b.aspx?institutionid=110635 http://www.collegeresults.org/search1b.aspx?institutionid=11... which aggregates data that colleges are required by law to report to the federal government into user-friendly data look-ups that allow direct comparisons of similar colleges along many dimensions. For me as a parent, one of the most interesting data views is a view of "comparable colleges" for a college of interest, sorted under the Finance and Faculty tab for a ranking of colleges by instructional expenditures / FTE (full-time equivalent students). That comparison often reveals that even the "scholarships" (discounts from list price) that colleges offer still leave parents spending far more for their children's higher education than the college itself actually spends on educating students. That's a raw deal that more parents ought to know about. Colleges hire expensive consultants to learn how to confuse parents on the issue of value, http://www.maguireassoc.com/services-challenges/optimize-net-revenue/ http://www.maguireassoc.com/services-challenges/optimize-net... and parents have to defend themselves by looking up comparable data.

tokenadult 14y ago

There was an earlier Hacker News submission about the Gates Foundation research on teacher effectiveness, http://news.ycombinator.com/item?id=4559682 http://news.ycombinator.com/item?id=4559682 linking to an article that reported details of the methodology. http://www.theatlantic.com/magazine/archive/2012/10/why-kids-should-grade-teachers/309088/ http://www.theatlantic.com/magazine/archive/2012/10/why-kids... I looked up other research on the matter for the reply I posted in that thread. From the article submitted then, this is one way this process has been validated: "The responses did indeed help predict which classes would have the most test-score improvement at the end of the year. In math, for example, the teachers rated most highly by students delivered the equivalent of about six more months of learning than teachers with the lowest ratings. (By comparison, teachers who get a master’s degree—one of the few ways to earn a pay raise in most schools —delivered about one more month of learning per year than teachers without one.) . . . . "The survey did not ask Do you like your teacher? Is your teacher nice? This wasn’t a popularity contest. The survey mostly asked questions about what students saw, day in and day out. "Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward: 1. Students in this class treat the teacher with respect. 2. My classmates behave the way my teacher wants them to. 3. Our class stays busy and doesn’t waste time. 4. In this class, we learn a lot almost every day. 5. In this class, we learn to correct our mistakes." Here is earlier reporting (10 December 2010) from the New York Times about the same issue: http://www.nytimes.com/2010/12/11/education/11education.html http://www.nytimes.com/2010/12/11/education/11education.html Here is the website of Ronald Ferguson's research project at Harvard: http://tripodproject.wpengine.com/about/our-team/ http://tripodproject.wpengine.com/about/our-team/ And here are some links about the project from the National Center for Teacher Effectiveness: http://www.gse.harvard.edu/ncte/news/NCTE_Conference_Using_Student_Surveys.php http://www.gse.harvard.edu/ncte/news/NCTE_Conference_Using_S... Simply put, don't assume that what the Gates Foundation was investigating was the same kind of student opinion survey that I have filled out as a postsecondary student. (But note that I'm not so sure that those surveys are as bad or as useless as college faculty often claim they are.) There is a research base for the primary school pupil and secondary school student ratings used in the Gates Foundation studies, and I have every reason to believe those ratings would help school effectiveness--so much so that I use the same questions to invite my clients of my mathematics program to evaluate my teaching from that point of view. Other comments in this thread are about the more general issue of college rankings as they currently exist. As a parent who has occasion to look at my children's college search process for four children, I really like the site College Results http://www.collegeresults.org/search1b.aspx?institutionid=110635 http://www.collegeresults.org/search1b.aspx?institutionid=11... which aggregates data that colleges are required by law to report to the federal government into user-friendly data look-ups that allow direct comparisons of similar colleges along many dimensions. For me as a parent, one of the most interesting data views is a view of "comparable colleges" for a college of interest, sorted under the Finance and Faculty tab for a ranking of colleges by instructional expenditures / FTE (full-time equivalent students). That comparison often reveals that even the "scholarships" (discounts from list price) that colleges offer still leave parents spending far more for their children's higher education than the college itself actually spends on educating students. That's a raw deal that more parents ought to know about. Colleges hire expensive consultants to learn how to confuse parents on the issue of value, http://www.maguireassoc.com/services-challenges/optimize-net-revenue/ http://www.maguireassoc.com/services-challenges/optimize-net... and parents have to defend themselves by looking up comparable data.