3 ms·

This is a common misunderstanding. When models impact the world, what matters more are their outcomes and consequences.

by loopz 7y ago

manfredo 7y ago
Right. And when you force a model to avoid disparities when disparities really do exist, the outcome is inaccurate data and the consequence is that decisions are made based on inaccurate data.
- loopz 7y ago
  Sure, and there is also the assumption that good decisions are made on perfect/good data, which is also just an assumption. We can't even define good decisions well enough to be sure. Just because bad data may tend (in hindsight) to yield inaccurate decisions, or decisions based on faulty data, doesn't promote data as the main driver behind sound decisions. That would just be another belief system, and we don't have all possible data or omniscience to prove it!
  - manfredo 7y ago
    No, you can test and gauge the quality of that data. At this point, you're not really even making a point about whether or not we should introduce biases in our models to make them match our worldview - you're saying the entire endeavor of trying to build a model is worthless. If what you're saying is true, that we can't measure what decisions are successful or unsuccessful, then there's no point in trying to build a model because we no way of knowing whether the model is working or not.
    - loopz 7y ago
      It is the claim of vendors that their models works. It is irresponsible to just take their word for it, and such tests and critique need be ongoing.
- alexknvl 7y ago
  What people in this thread seem to be missing is that modeling and decision making do not have to be done simultaneously. You always want the most accurate model of reality as you can get, but what you do with that model is up to you. Perhaps you think there are certain feedback loops that need to be broken (your model is usually a static representation of the reality), or perhaps you prefer low rate of false negatives / false positives, or perhaps if a model is uncertain you would like to defer to a human. If a model predicts high probability of re-offence, you might decide to delay an action as an example, or re-examine the case in more detail. I get a feeling that machine learning practitioners are not properly trained to recognize such subtleties, everything is a binary softmax output these days :) EDIT: This seems to stem from the obsession with purely discriminative models that either directly model a binary function y = f(x) or model a function that returns some kind of score, which is later compared to a threshold y = f(x) > theta. Neither of these lend themselves nicely to this conceptual separation between modeling and decision making.

This is a common misunderstanding. When models impact the world, what matters more are their outcomes and consequences.

by loopz 7y ago

This is a common misunderstanding. When models impact the world, what matters more are their outcomes and consequences.

Right. And when you force a model to avoid disparities when disparities really do exist, the outcome is inaccurate data and the consequence is that decisions are made based on inaccurate data.

loopz 7y ago
Sure, and there is also the assumption that good decisions are made on perfect/good data, which is also just an assumption. We can't even define good decisions well enough to be sure. Just because bad data may tend (in hindsight) to yield inaccurate decisions, or decisions based on faulty data, doesn't promote data as the main driver behind sound decisions. That would just be another belief system, and we don't have all possible data or omniscience to prove it!
- manfredo 7y ago
  No, you can test and gauge the quality of that data. At this point, you're not really even making a point about whether or not we should introduce biases in our models to make them match our worldview - you're saying the entire endeavor of trying to build a model is worthless. If what you're saying is true, that we can't measure what decisions are successful or unsuccessful, then there's no point in trying to build a model because we no way of knowing whether the model is working or not.
  - loopz 7y ago
    It is the claim of vendors that their models works. It is irresponsible to just take their word for it, and such tests and critique need be ongoing.
alexknvl 7y ago
What people in this thread seem to be missing is that modeling and decision making do not have to be done simultaneously. You always want the most accurate model of reality as you can get, but what you do with that model is up to you. Perhaps you think there are certain feedback loops that need to be broken (your model is usually a static representation of the reality), or perhaps you prefer low rate of false negatives / false positives, or perhaps if a model is uncertain you would like to defer to a human. If a model predicts high probability of re-offence, you might decide to delay an action as an example, or re-examine the case in more detail. I get a feeling that machine learning practitioners are not properly trained to recognize such subtleties, everything is a binary softmax output these days :) EDIT: This seems to stem from the obsession with purely discriminative models that either directly model a binary function y = f(x) or model a function that returns some kind of score, which is later compared to a threshold y = f(x) > theta. Neither of these lend themselves nicely to this conceptual separation between modeling and decision making.