3 ms·
Here is how I would inspect schools: * Collect together a bunch of metrics of each school. Eg. student test scores, parent satisfaction scores, number of pol
by londons_explore 3y ago
Here is how I would inspect schools:
* Collect together a bunch of metrics of each school. Eg. student test scores, parent satisfaction scores, number of police callouts to the school, number of leaks in the school roof. Also include metrics that aren't obviously good/bad: Number of acres of playgrounds, average tenure of staff, etc.
* Gather data of the success of past students, 30-50 years on. For example, employment rate, total earnings, percentage convicted, percentage in good health.
* Build a model to predict success metrics from the school metrics.
* To rate a school, go collect the school metrics, then run through the model to predict future success metrics. That is your rating.
Sure, such an approach has the correlation/causation problem. But this is self-correcting if schools try to optimize their scores as the models are rebuilt each year.
- charlieyu1 3y agoAll these metrics are easily manipulated. Test scores can be manipulated by giving easier tests. Measuring number of police call-outs just encourages not calling police on emergencies.
- oogali 3y agohttps://en.wikipedia.org/wiki/Goodhart%27s_law#Generalization https://en.wikipedia.org/wiki/Goodhart%27s_law#Generalizatio...
- londons_explore 3y agoSure, but assuming everyone puts similar amounts of efforts into manipulating the metrics, the model will come to reflect the fact that those metrics are no longer predictive, and there will no longer be a benefit to manipulating them. Thats the benefit of this scheme - it isn't statistically sound, but all the statistical shortcomings eliminate themselves as people try to exploit them.
- wharvle 3y agoYou’ve just made a map of rich and poor areas with extra steps. Which is effective. If you’re a parent trying to decide which schools you want your kids in, maps of where the money is and maps of school rankings are damn near interchangeable (mostly not for funding-related reasons, though). You could use either and come to similar conclusions.