3 ms·
> There’s a lot of finger pointing at metrics and complacency about problems, but the solutions are super vague The solution is obvious, and not vague at all:
by inimino 7y ago
> There’s a lot of finger pointing at metrics and complacency about problems, but the solutions are super vague
The solution is obvious, and not vague at all: stop over-relying on metrics, and stop pretending that what matters can in most cases be measured.[1] However, I think you dislike this answer (for reasons given in my other reply in this thread) so you are looking for ways to replace bad metrics with better metrics. Which is worthwhile, but not the immediate answer.
> just listening to first-person accounts (which is probably a bad idea if you actually want to help).
I didn't see in the article where anyone suggested only listening to first-person accounts. However my strong belief is that if you don't listen to and seek out first-person accounts, you have almost no chance whatsoever of doing any good in basically any kind of complex social problem, and a high likelihood of doing harm while patting yourself on the back because some metric you settled on is going up.
[1]: Edit to add: This implies not using AI/ML in some areas where it currently is being used. The infamous example of AI grading essays, linked from the article, is one of the most egregious examples of the misuse of ML for something that, to me, is unimaginably, breathtakingly, forehead-slappingly idiotic to give to anyone but people, and not just any people, but the people closest to the students whose work is being graded.
- mlthoughts2018 7y ago> “The solution is obvious, and not vague at all: stop over-relying on metrics, and stop pretending that what matters can in most cases be measured.” That is vague. You, like the article, are not being specific or explaining how this can be systematically applied.
- inimino 7y ago"systematically applied"... I'm not giving you a system, I'm specifically telling you not to rely on systems and rules in cases where they do not work. I'm arguing that people need to take responsibility for the correspondence of their own actions and values. This is not a system, it's an attitude. If you're using ML to grade essays, then you need to stop doing that, because the things modern ML can measure are not the things that make an essay good or bad. If you're using metrics to drive a business process, and the output of that process that corresponds to your values cannot be captured by a metric, then you need to stop using metrics and instead design your process according to principles that reflect your values.
- wtracy 7y agoI think the fundamental message here is that there isn't a solution that can be systematically applied.