4 ms·

>To be clear, there is a lot of bias in standardized tests Is there? I'd say that there is very, very minimal amount of bias in them. >When I teach I make my

• godelski 3y ago
  > Is there? I'd say that there is very, very minimal amount of bias in them. Yes, it is the exact thing we've been talking about since standardized testing is the status quo. There are also plenty of studies on this if you'd like to google for more. It's also important to remember that there are very clear test taking skills, and that these standardized tests intentionally try to trick students (you'll even see mentions of this in your test prep books). > Projects are better, but you cannot do them at scale and compare them. Unfortunately some things don't scale very well. Best you can do is scale by hierarchy. Using local groups. But before you say that this requires more trust I'd also note that we do require a lot of trust in our society and for the most part people are pretty trustworthy. It is when metric hacking is encouraged that trust is lost. A lot of standardized testing has a lot of trust built in as well. But there's another way to think about it. If at the local level you are evaluating someone a certain way then those are the skills that you are teaching them. When you are doing standardized testing you are mainly teaching students how to be good test takers. But we've seen that that tends to overfit as any standardized test is incapable of incorporating nuanced thinking due to requiring to allow for high variance in test takers. On the other hand, local evaluations via projects are directly teaching students to learn to do something much closer to what they'll do in the real world. Also tends to have larger benefits in keeping students excited/motivated as well as find their direction. But yes, this requires more hierarchical trust. But this kinda already exists. We already rank schools, we just tend to do this by standardized test scores. But you can use other metrics as well. The schools could even improve by weighting teachers differently. But my main point here is that the main benefit here isn't actually the evaluation ability at the abstracted level (the level bureaucrats want, the level we judge populations, the aggregate levels -- which I am actually arguing are extremely noisy predictors (read: unreliable). Aggregate metrics are often unreliable fwiw: see {Simpson's,Berkson's} Paradox). Means often get you into trouble), but rather that if in local cliques you enforce good metrics that this propagates upwards. Think like Conway's game (or even physics): if your local set of rules is good, you can generate arbitrary complexity at the large scale due to compounding effects. To be clear: the goals I'd want schooling to align with are: 1) maximizing education of people (both specific to their desired specialization as well as general knowledge and skills to learn to learn), 2) achieve threshold value for socialization level (ensuring people can interact in society. This is often implicit but I'm making explicit), 3) maximizing labor efficiency when student enters the workforce (minimize onboard training). This is an incomplete list and should be thought out more and far more carefully so we can describe the nonlinear program, but this is a good enough outline -- at least for HN. But I'll also add that there's a certain level of noise that is quite hard to get rid of. To truly evaluate good teaching methods with good accuracy you actually need to track students over very long periods of time. Which gets extremely difficult to model since those students take varying paths and interact with a lot of different teachers who you must all decouple. You're now essentially trying to solve a problem similar to neural network interpretation/explainability. This is true for no matter what evaluation method you use. TLDR: metrics are fucking hard, do not underestimate the difficulty of evaluation. It'll bite you in the ass