2 ms·

The data has proved to be virtually useless in assessing teachers. See http://garyrubinstein.teachforus.org/2012/02/26/analyzing-released-nyc-value-added-data-

The data has proved to be virtually useless in assessing teachers. See http://garyrubinstein.teachforus.org/2012/02/26/analyzing-released-nyc-value-added-data-part-1/ http://garyrubinstein.teachforus.org/2012/02/26/analyzing-re... They're useful for showing students are underperforming. But you can't say anything about teacher quality. In fact I suggest if you took the teachers from the "worst schools" and put them in the "best schools" and vice-versa the results the following year wouldn't look noticeably different. Does that mean that teacher have no impact at all. I don't believe that. But I do think the large gaps in achievement are systemic problems more than the problem of teacher quality at certain schools.

yummyfajitas 15y ago
The fact that some teacher can't see the correlations using an inappropriate plotting technique (he uses a scatterplot, he should use hexbin or other density plot) doesn't mean they aren't there. And the numbers he gathers and dismisses show the correlation actually does exist. A simple numerical example, where the correlation is guaranteed (i.e., I took y=x+noise): http://i.imgur.com/rrmUI.png http://i.imgur.com/rrmUI.png https://gist.github.com/2183927 https://gist.github.com/2183927 Most of the correlations he expects to find are present. They are noisy, but present. The fact that in one case, reality is "contrary to what every teacher in the world knows" just suggests maybe teachers don't have a great grip on reality. ...if you took the teachers from the "worst schools" and put them in the "best schools" and vice-versa the results the following year wouldn't look noticeably different...Does that mean that teacher have no impact at all? Not quite, but almost. It means that variation among existing teachers impact less than the size of measurement, and the current crop of teachers are basically interchangeable cogs.
- acdha 15y ago
  > Not quite, but almost. It means that variation among existing teachers impact less than the size of measurement, and the current crop of teachers are basically interchangeable cogs. Or that the students - and their parents - are a very important factor with significant variation and classes a relatively small sample size. Ask any teacher and they can tell stories about how different classes can be - and how much a school having a weak discipline policy will ensure that many students will have poorer classroom experiences because of one or two difficult students. It's not like we don't know how to do scientific measurements of complex systems but it's expensive and slow at a time when the political requirement is fast and cheap. It'd be awesome if school districts actually hired people with backgrounds doing serious statistical analysis or large-scale studies with human subjects but nobody is jumping to fund that.

yummyfajitas 15y ago

The fact that some teacher can't see the correlations using an inappropriate plotting technique (he uses a scatterplot, he should use hexbin or other density plot) doesn't mean they aren't there. And the numbers he gathers and dismisses show the correlation actually does exist. A simple numerical example, where the correlation is guaranteed (i.e., I took y=x+noise): http://i.imgur.com/rrmUI.png http://i.imgur.com/rrmUI.png https://gist.github.com/2183927 https://gist.github.com/2183927 Most of the correlations he expects to find are present. They are noisy, but present. The fact that in one case, reality is "contrary to what every teacher in the world knows" just suggests maybe teachers don't have a great grip on reality. ...if you took the teachers from the "worst schools" and put them in the "best schools" and vice-versa the results the following year wouldn't look noticeably different...Does that mean that teacher have no impact at all? Not quite, but almost. It means that variation among existing teachers impact less than the size of measurement, and the current crop of teachers are basically interchangeable cogs.

acdha 15y ago
> Not quite, but almost. It means that variation among existing teachers impact less than the size of measurement, and the current crop of teachers are basically interchangeable cogs. Or that the students - and their parents - are a very important factor with significant variation and classes a relatively small sample size. Ask any teacher and they can tell stories about how different classes can be - and how much a school having a weak discipline policy will ensure that many students will have poorer classroom experiences because of one or two difficult students. It's not like we don't know how to do scientific measurements of complex systems but it's expensive and slow at a time when the political requirement is fast and cheap. It'd be awesome if school districts actually hired people with backgrounds doing serious statistical analysis or large-scale studies with human subjects but nobody is jumping to fund that.