4 ms·
Are there other possible confounding factors? They picked 6 schools that were known to have done well by other measures, and then analyzed a new statistic over
by btilly 14y ago
Are there other possible confounding factors?
They picked 6 schools that were known to have done well by other measures, and then analyzed a new statistic over the same time period. But the fact that they had done well by other measures could be explained if they had been lucky in the students they got in the lottery, which would also explain their doing well by a new measure.
The simplest way to solve this is to select schools by how well they did with one cohort, and analyze them by how well they do with another. For instance you can select them based on performance during people who entered in even years on one exam, then analyze them by people who entered in odd years on the other exam. Yes, this cuts your sample size in half. But it avoids this particular bias.
(This does not, unfortunately, avoid the possible bias from getting lucky with who they hired as teachers. Yes, the school may work. But was it by design or chance? The school will always think by design, but unless some charter school can start pumping out successful franchisees, it will be hard to find proof.)
- beat 14y agoCharter schools that have a specific educational approach will attract a certain kind of teacher. At the charter my kids attended, teachers would either come and be gone in a year (they didn't get it), or they would become totally devoted to the cause, to the point where it would be difficult for another school to lure them away even if they tried. Moreover, charters with a clearly-defined academic approach attract a certain kind of student, and a certain kind of parent. So you're not getting a flat, level field of either teachers or students (what I call a "perfectly spherical cow"). You're getting a selective subset of both teachers and students, which will bias the results right there.
- btilly 14y agoOnly looking at schools with lotteries can let you control for the student bias. But you have an excellent point that a certain charter may be an excellent fit for some subset of teachers/parents, but not be appropriate for the broader population.
- beat 14y agoThat gets into state-by-state law. In Minnesota, all charters have lotteries (and those that can't "sell out" and completely fill their classrooms will probably have financial problems). States that allow charters to be selective are on a bad path, imho. Like public schools, charters should be required to accept any students who apply and get through the lottery.
- yummyfajitas 14y agoWhat's wrong with creating schools that serve one group of students well, and don't serve other students? It seems like a win for some, and neutral for everyone else. Market segmentation is a good thing. Lenovo builds great laptops for me and other linux geeks. Apple builds great laptops for hipsters and their moms. And both Apple and Lenovo (together with their customers) can segment the market. Does that mean the computer industry is on a "bad path"?
- beat 14y agoAllow the students and parents to be selective, not the schools. This supports market segmentation (a good way to look at it), but doesn't introduce actual unfairness. What you're talking about is more like the case of Abercrombie and Fitch refusing to make clothes for people that they don't find attractive size-wise. That's fine for private business, but not appropriate for publicly-funded systems like schools.
- yummyfajitas 14y agoThe publicly-funded system doesn't segment the market. Individual pieces of it do. The public schools will continue to service all segments, but some segments will have other options. How is this not a pareto win (for students)?
- btilly 14y agoIf you're satisfied with a pareto win, why don't we go back to an absolute monarchy? After all no other system could be better for the monarch! Back in the real world, when you create one system for relatively well-off people and another for the rest, the tendency is to see the privileged system constantly improved and the fallback languish into a horrible state. This has certainly happened in education. If you create charter schools and allow them to choose who can attend, it is inevitable that successful ones will cherrypick children who start with intellectual abilities and the financial/educational family background to back that up. This creates the kind of two-tiered system that has lead to problems in the past. That result may be pareto optimal, but it is a social injustice of the kind that many don't on principle want supported by tax dollars. (Of course in practice, if you're personally benefitting, protests tend to be much more muted...)