2 ms·

> This risk of subverting qualitative data like this into a disingenuous claim that something is data-driven is really high, in addition to all the usual cognit

> This risk of subverting qualitative data like this into a disingenuous claim that something is data-driven is really high, in addition to all the usual cognitive biases that lead people to read what they want into narrative accounts. We have engaged on related ideas before[1], and I think your position represents something that I find quite common and deeply unsettling about modern tech culture, but which I don't yet feel I can fully characterize. There is something about the need for everything to be "data-driven" and metrics-focused that I think represents an abrogation of moral responsibility to form, apply, and maintain our own value systems. It is easier and we would apparently rather assign this messy work to some external process or algorithm or metric, which can then be argued to be perfectly rational, correct, data-driven, unbiased[2]. To me this is a dangerous fantasy, closely related to scientism or the general failure to accept any reality outside of what can be measured, counted, or quantified. This is a characteristic failure (in my opinion) of people in their twenties and of STEM majors in our times, so it's not surprising that it would be a characteristic failure of tech, where both those categories are over-represented. It also seems correlated with a general ignorance of and arrogance towards the humanities among tech workers. [1]: https://news.ycombinator.com/item?id=21097061 https://news.ycombinator.com/item?id=21097061 [2]: A related symptom of this disease is in hiring, where companies come up with systems that look very logical and bias free on paper, for example having "objective" scoring criteria and technical challenges (which end up being almost random number generators, but with some bias that tends to reinforce existing imbalances in the industry, e.g. towards recent graduates), and equally "objective" ways of combining results from hiring committee members or interviewers that tend to create and enforce conformity and exclude divergent viewpoints, while giving the appearance of being bias-free, data-driven, and rational. In fact it's a great way that entire companies fool themselves and come up with terrible processes, and then they whine about how hard it is to maintain diversity (of gender, age, whatever) in their ranks. The people that design these systems aren't doing this as some kind of great conspiracy; as far as I can see, they're not evil, just blind.

perl4ever 7y ago
"an abrogation of moral responsibility to form, apply, and maintain our own value systems." I think it's a big (and almost certainly invalid) assumption that in the absence of metrics, people "apply and maintain" a consistent value system. It seems to me inconsistent with a belief in the destructive effect of the wrong metrics. If you think that metrics lead to bad results, it's completely inconsistent to suggest that not having metrics is better. When you have metrics, the harm comes from people ignoring what really should be optimized in favor of what is being measured. If you don't have any metrics, then you're only decreasing the pressure to do whatever may be the right thing. It's like, no matter how bad the government is, the only solution is better government, not eliminating it. So, I think your general attitude is ironically capturing the essence of a common intellectual failing of the people you are stereotyping. You know, the glib libertarian tech worker.
- inimino 7y ago
  > in the absence of metrics, people "apply and maintain" a consistent value system What on earth makes you think I'm making this assumption? I never said anything of the sort. I said that people are abrogating their responsibility to do difficult work (figuring out if the ship is going in the right direction or not) by trying to offload this onto metrics, which are theoretically objective and unbiased. Nowhere did I ever say anything about what people not over-using metrics are doing. > If you don't have any metrics, then you're only decreasing the pressure to do whatever may be the right thing. That is not necessarily true at all. Having no metrics at all, just like having no government, means that you must fill the void with something. You must decide for yourself how to act, probably based on a subjective judgement of what the situation is combined with your personal values.
  - perl4ever 7y ago
    "Having no metrics at all, just like having no government, means that you must fill the void with something." How can one "fill the void" without measuring anything? Personal values may vary, but I can't see the concept of determining the consequences of your actions as being an optional one of them. It seems to me that if you are definitely rejecting that, then you have no personal values. Even if you say you do.
    - inimino 7y ago
      Please be careful with your terminology here. When we talk about metrics, especially in a business context, we are talking about numerical quantities that can be tracked and unambiguously measured. A metric that can't be measured, or that nobody agrees on the measurement of, is no good as a metric, in this context. Are we agreed on that part of the definition of a metric? > determining the consequences of your actions That is an entirely different thing. > measuring anything And that is yet another entirely separate concept. You can weigh a course of action without measuring anything. > It seems to me that if you are definitely rejecting that, then you have no personal values. In most situations in life, you must make a decision without any chance to know the results of your actions. Can you know in advance the result of picking a career, or taking a job, getting married, moving to another country, or voting for a particular candidate? Of course not. The number of decisions in life that are repeated, and where a numerical, measurable value is available to guide your decision making over time (which is what is meant by tracking metrics in a business context) are a vanishingly small minority of the decisions we must make. Unless you calculate a utility function before crossing the street, or having breakfast, I don't think you really mean to argue for what you appear to be arguing for.

perl4ever 7y ago

"an abrogation of moral responsibility to form, apply, and maintain our own value systems." I think it's a big (and almost certainly invalid) assumption that in the absence of metrics, people "apply and maintain" a consistent value system. It seems to me inconsistent with a belief in the destructive effect of the wrong metrics. If you think that metrics lead to bad results, it's completely inconsistent to suggest that not having metrics is better. When you have metrics, the harm comes from people ignoring what really should be optimized in favor of what is being measured. If you don't have any metrics, then you're only decreasing the pressure to do whatever may be the right thing. It's like, no matter how bad the government is, the only solution is better government, not eliminating it. So, I think your general attitude is ironically capturing the essence of a common intellectual failing of the people you are stereotyping. You know, the glib libertarian tech worker.

inimino 7y ago
> in the absence of metrics, people "apply and maintain" a consistent value system What on earth makes you think I'm making this assumption? I never said anything of the sort. I said that people are abrogating their responsibility to do difficult work (figuring out if the ship is going in the right direction or not) by trying to offload this onto metrics, which are theoretically objective and unbiased. Nowhere did I ever say anything about what people not over-using metrics are doing. > If you don't have any metrics, then you're only decreasing the pressure to do whatever may be the right thing. That is not necessarily true at all. Having no metrics at all, just like having no government, means that you must fill the void with something. You must decide for yourself how to act, probably based on a subjective judgement of what the situation is combined with your personal values.
- perl4ever 7y ago
  "Having no metrics at all, just like having no government, means that you must fill the void with something." How can one "fill the void" without measuring anything? Personal values may vary, but I can't see the concept of determining the consequences of your actions as being an optional one of them. It seems to me that if you are definitely rejecting that, then you have no personal values. Even if you say you do.
  - inimino 7y ago
    Please be careful with your terminology here. When we talk about metrics, especially in a business context, we are talking about numerical quantities that can be tracked and unambiguously measured. A metric that can't be measured, or that nobody agrees on the measurement of, is no good as a metric, in this context. Are we agreed on that part of the definition of a metric? > determining the consequences of your actions That is an entirely different thing. > measuring anything And that is yet another entirely separate concept. You can weigh a course of action without measuring anything. > It seems to me that if you are definitely rejecting that, then you have no personal values. In most situations in life, you must make a decision without any chance to know the results of your actions. Can you know in advance the result of picking a career, or taking a job, getting married, moving to another country, or voting for a particular candidate? Of course not. The number of decisions in life that are repeated, and where a numerical, measurable value is available to guide your decision making over time (which is what is meant by tracking metrics in a business context) are a vanishingly small minority of the decisions we must make. Unless you calculate a utility function before crossing the street, or having breakfast, I don't think you really mean to argue for what you appear to be arguing for.