5 ms·

The problem with discussions like this is that they never provide systematic examples of how a portfolio of metrics or qualitative checking can be integrated in

by mlthoughts2018 7y ago

The problem with discussions like this is that they never provide systematic examples of how a portfolio of metrics or qualitative checking can be integrated into a modeling problem. There’s a lot of finger pointing at metrics and complacency about problems, but the solutions are super vague, like the sanctimonious passage in this article about hiring from under-indexed groups in tech companies and just listening to first-person accounts (which is probably a bad idea if you actually want to help). Ultimately I agree with the underlying idea, but I think to be helpful you have to present case studies of reproducing research but with metric optimization swapped out for a holistic variety of metrics plus qualitative checking. I recommend the books Bayesian Data Analysis by Gelman et al and Data Analysis Using Regression and Multilevel/Hierarchical Models by Gelman and Hill if you want to read good accounts of doing this in practice with real data sets. There’s definitely room for a book like this that focuses on more domain specific models in NLP, computer vision and deep neural networks.

marmadukester39 7y ago
I’m not so sure it’s a bad idea to listen to first person accounts from underindexed groups. For one, it may help surface new metrics to readjust with; for another, it may still help to get a sampling of “is this helping or hurting in your opinion” and watch how it trends over time.
- mlthoughts2018 7y ago
  I understand what you mean, but the problem is that first person accounts are so idiosyncratic and situation-specific. It takes a lot of discipline to distill only the most well-supported high level themes from a set of anecdotal qualitative data. The discussion also is at high risk of becoming politicized, where more credence will be given to salient first-person accounts that happen to fit well into some prevalent social narrative of the times. If you can prove that a lot of care was taken to avoid these types of bias in extracting summaries from a collection of qualitative data, then it can be valuable. But often this means you have to be so cautious and err on the side of rejecting all but the most consistent summaries from the data that you run afoul of critics who want the data to be used to support their preferred political narrative. This risk of subverting qualitative data like this into a disingenuous claim that something is data-driven is really high, in addition to all the usual cognitive biases that lead people to read what they want into narrative accounts.
  - inimino 7y ago
    > This risk of subverting qualitative data like this into a disingenuous claim that something is data-driven is really high, in addition to all the usual cognitive biases that lead people to read what they want into narrative accounts. We have engaged on related ideas before[1], and I think your position represents something that I find quite common and deeply unsettling about modern tech culture, but which I don't yet feel I can fully characterize. There is something about the need for everything to be "data-driven" and metrics-focused that I think represents an abrogation of moral responsibility to form, apply, and maintain our own value systems. It is easier and we would apparently rather assign this messy work to some external process or algorithm or metric, which can then be argued to be perfectly rational, correct, data-driven, unbiased[2]. To me this is a dangerous fantasy, closely related to scientism or the general failure to accept any reality outside of what can be measured, counted, or quantified. This is a characteristic failure (in my opinion) of people in their twenties and of STEM majors in our times, so it's not surprising that it would be a characteristic failure of tech, where both those categories are over-represented. It also seems correlated with a general ignorance of and arrogance towards the humanities among tech workers. [1]: https://news.ycombinator.com/item?id=21097061 https://news.ycombinator.com/item?id=21097061 [2]: A related symptom of this disease is in hiring, where companies come up with systems that look very logical and bias free on paper, for example having "objective" scoring criteria and technical challenges (which end up being almost random number generators, but with some bias that tends to reinforce existing imbalances in the industry, e.g. towards recent graduates), and equally "objective" ways of combining results from hiring committee members or interviewers that tend to create and enforce conformity and exclude divergent viewpoints, while giving the appearance of being bias-free, data-driven, and rational. In fact it's a great way that entire companies fool themselves and come up with terrible processes, and then they whine about how hard it is to maintain diversity (of gender, age, whatever) in their ranks. The people that design these systems aren't doing this as some kind of great conspiracy; as far as I can see, they're not evil, just blind.
    - perl4ever 7y ago
      "an abrogation of moral responsibility to form, apply, and maintain our own value systems." I think it's a big (and almost certainly invalid) assumption that in the absence of metrics, people "apply and maintain" a consistent value system. It seems to me inconsistent with a belief in the destructive effect of the wrong metrics. If you think that metrics lead to bad results, it's completely inconsistent to suggest that not having metrics is better. When you have metrics, the harm comes from people ignoring what really should be optimized in favor of what is being measured. If you don't have any metrics, then you're only decreasing the pressure to do whatever may be the right thing. It's like, no matter how bad the government is, the only solution is better government, not eliminating it. So, I think your general attitude is ironically capturing the essence of a common intellectual failing of the people you are stereotyping. You know, the glib libertarian tech worker.
inimino 7y ago
> There’s a lot of finger pointing at metrics and complacency about problems, but the solutions are super vague The solution is obvious, and not vague at all: stop over-relying on metrics, and stop pretending that what matters can in most cases be measured.[1] However, I think you dislike this answer (for reasons given in my other reply in this thread) so you are looking for ways to replace bad metrics with better metrics. Which is worthwhile, but not the immediate answer. > just listening to first-person accounts (which is probably a bad idea if you actually want to help). I didn't see in the article where anyone suggested only listening to first-person accounts. However my strong belief is that if you don't listen to and seek out first-person accounts, you have almost no chance whatsoever of doing any good in basically any kind of complex social problem, and a high likelihood of doing harm while patting yourself on the back because some metric you settled on is going up. [1]: Edit to add: This implies not using AI/ML in some areas where it currently is being used. The infamous example of AI grading essays, linked from the article, is one of the most egregious examples of the misuse of ML for something that, to me, is unimaginably, breathtakingly, forehead-slappingly idiotic to give to anyone but people, and not just any people, but the people closest to the students whose work is being graded.
- mlthoughts2018 7y ago
  > “The solution is obvious, and not vague at all: stop over-relying on metrics, and stop pretending that what matters can in most cases be measured.” That is vague. You, like the article, are not being specific or explaining how this can be systematically applied.
  - inimino 7y ago
    "systematically applied"... I'm not giving you a system, I'm specifically telling you not to rely on systems and rules in cases where they do not work. I'm arguing that people need to take responsibility for the correspondence of their own actions and values. This is not a system, it's an attitude. If you're using ML to grade essays, then you need to stop doing that, because the things modern ML can measure are not the things that make an essay good or bad. If you're using metrics to drive a business process, and the output of that process that corresponds to your values cannot be captured by a metric, then you need to stop using metrics and instead design your process according to principles that reflect your values.

marmadukester39 7y ago

I’m not so sure it’s a bad idea to listen to first person accounts from underindexed groups. For one, it may help surface new metrics to readjust with; for another, it may still help to get a sampling of “is this helping or hurting in your opinion” and watch how it trends over time.

mlthoughts2018 7y ago
I understand what you mean, but the problem is that first person accounts are so idiosyncratic and situation-specific. It takes a lot of discipline to distill only the most well-supported high level themes from a set of anecdotal qualitative data. The discussion also is at high risk of becoming politicized, where more credence will be given to salient first-person accounts that happen to fit well into some prevalent social narrative of the times. If you can prove that a lot of care was taken to avoid these types of bias in extracting summaries from a collection of qualitative data, then it can be valuable. But often this means you have to be so cautious and err on the side of rejecting all but the most consistent summaries from the data that you run afoul of critics who want the data to be used to support their preferred political narrative. This risk of subverting qualitative data like this into a disingenuous claim that something is data-driven is really high, in addition to all the usual cognitive biases that lead people to read what they want into narrative accounts.
- inimino 7y ago
  > This risk of subverting qualitative data like this into a disingenuous claim that something is data-driven is really high, in addition to all the usual cognitive biases that lead people to read what they want into narrative accounts. We have engaged on related ideas before[1], and I think your position represents something that I find quite common and deeply unsettling about modern tech culture, but which I don't yet feel I can fully characterize. There is something about the need for everything to be "data-driven" and metrics-focused that I think represents an abrogation of moral responsibility to form, apply, and maintain our own value systems. It is easier and we would apparently rather assign this messy work to some external process or algorithm or metric, which can then be argued to be perfectly rational, correct, data-driven, unbiased[2]. To me this is a dangerous fantasy, closely related to scientism or the general failure to accept any reality outside of what can be measured, counted, or quantified. This is a characteristic failure (in my opinion) of people in their twenties and of STEM majors in our times, so it's not surprising that it would be a characteristic failure of tech, where both those categories are over-represented. It also seems correlated with a general ignorance of and arrogance towards the humanities among tech workers. [1]: https://news.ycombinator.com/item?id=21097061 https://news.ycombinator.com/item?id=21097061 [2]: A related symptom of this disease is in hiring, where companies come up with systems that look very logical and bias free on paper, for example having "objective" scoring criteria and technical challenges (which end up being almost random number generators, but with some bias that tends to reinforce existing imbalances in the industry, e.g. towards recent graduates), and equally "objective" ways of combining results from hiring committee members or interviewers that tend to create and enforce conformity and exclude divergent viewpoints, while giving the appearance of being bias-free, data-driven, and rational. In fact it's a great way that entire companies fool themselves and come up with terrible processes, and then they whine about how hard it is to maintain diversity (of gender, age, whatever) in their ranks. The people that design these systems aren't doing this as some kind of great conspiracy; as far as I can see, they're not evil, just blind.
  - perl4ever 7y ago
    "an abrogation of moral responsibility to form, apply, and maintain our own value systems." I think it's a big (and almost certainly invalid) assumption that in the absence of metrics, people "apply and maintain" a consistent value system. It seems to me inconsistent with a belief in the destructive effect of the wrong metrics. If you think that metrics lead to bad results, it's completely inconsistent to suggest that not having metrics is better. When you have metrics, the harm comes from people ignoring what really should be optimized in favor of what is being measured. If you don't have any metrics, then you're only decreasing the pressure to do whatever may be the right thing. It's like, no matter how bad the government is, the only solution is better government, not eliminating it. So, I think your general attitude is ironically capturing the essence of a common intellectual failing of the people you are stereotyping. You know, the glib libertarian tech worker.

inimino 7y ago

> There’s a lot of finger pointing at metrics and complacency about problems, but the solutions are super vague The solution is obvious, and not vague at all: stop over-relying on metrics, and stop pretending that what matters can in most cases be measured.[1] However, I think you dislike this answer (for reasons given in my other reply in this thread) so you are looking for ways to replace bad metrics with better metrics. Which is worthwhile, but not the immediate answer. > just listening to first-person accounts (which is probably a bad idea if you actually want to help). I didn't see in the article where anyone suggested only listening to first-person accounts. However my strong belief is that if you don't listen to and seek out first-person accounts, you have almost no chance whatsoever of doing any good in basically any kind of complex social problem, and a high likelihood of doing harm while patting yourself on the back because some metric you settled on is going up. [1]: Edit to add: This implies not using AI/ML in some areas where it currently is being used. The infamous example of AI grading essays, linked from the article, is one of the most egregious examples of the misuse of ML for something that, to me, is unimaginably, breathtakingly, forehead-slappingly idiotic to give to anyone but people, and not just any people, but the people closest to the students whose work is being graded.

mlthoughts2018 7y ago
> “The solution is obvious, and not vague at all: stop over-relying on metrics, and stop pretending that what matters can in most cases be measured.” That is vague. You, like the article, are not being specific or explaining how this can be systematically applied.
- inimino 7y ago
  "systematically applied"... I'm not giving you a system, I'm specifically telling you not to rely on systems and rules in cases where they do not work. I'm arguing that people need to take responsibility for the correspondence of their own actions and values. This is not a system, it's an attitude. If you're using ML to grade essays, then you need to stop doing that, because the things modern ML can measure are not the things that make an essay good or bad. If you're using metrics to drive a business process, and the output of that process that corresponds to your values cannot be captured by a metric, then you need to stop using metrics and instead design your process according to principles that reflect your values.