4 ms·

But how did their model compare against others? The article only mentions how their interpretable model compared against their own ML attempts

jph00 7y ago
Their model didn't win. IBM's model won, based on actual metrics around useful insights.
- Mathnerd314 7y ago
  The IBM team got $5,000 and the second place/honorable mention NYU got $2,000. So going by prize amounts, the Duke model was still pretty good. IBM turned the model/paper into a toolkit: https://www.ibm.com/blogs/research/2019/08/ai-explainability-360/ https://www.ibm.com/blogs/research/2019/08/ai-explainability... Their model seems to be a variant of decision trees that has a knob controlling how complicated the trees are. And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them.
  - A_No_Name_Mouse 7y ago
    > And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them. That's what you get if you use a black box for judging :-)
  - netcan 7y ago
    Reading "subjective" to mean "nonexistent" is a potentially big mistake. Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria is the definite goal. In this area, we don't exactly know how to define narrow, objective goals & subsequent criteria. We can definee goalposts, but not goals. These are guesses at useful markers of success, useful to the larger goal of useful/novel ai. Subjective goals have their own (massive problems), but since we can't objectively define the goals of ai research... we need to fall back on human subjectivity to define our subgoals.
    - dragonwriter 7y ago
      All objective criteria are chosen, directly or indirectly, based on subjective criteria.
      - netcan 7y ago
        Also true. Subjectivity is unavoidable as long as we are relevant.. or so it seems circa 2020

jph00 7y ago

Their model didn't win. IBM's model won, based on actual metrics around useful insights.

Mathnerd314 7y ago
The IBM team got $5,000 and the second place/honorable mention NYU got $2,000. So going by prize amounts, the Duke model was still pretty good. IBM turned the model/paper into a toolkit: https://www.ibm.com/blogs/research/2019/08/ai-explainability-360/ https://www.ibm.com/blogs/research/2019/08/ai-explainability... Their model seems to be a variant of decision trees that has a knob controlling how complicated the trees are. And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them.
- A_No_Name_Mouse 7y ago
  > And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them. That's what you get if you use a black box for judging :-)
- netcan 7y ago
  Reading "subjective" to mean "nonexistent" is a potentially big mistake. Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria is the definite goal. In this area, we don't exactly know how to define narrow, objective goals & subsequent criteria. We can definee goalposts, but not goals. These are guesses at useful markers of success, useful to the larger goal of useful/novel ai. Subjective goals have their own (massive problems), but since we can't objectively define the goals of ai research... we need to fall back on human subjectivity to define our subgoals.
  - dragonwriter 7y ago
    All objective criteria are chosen, directly or indirectly, based on subjective criteria.
    - netcan 7y ago
      Also true. Subjectivity is unavoidable as long as we are relevant.. or so it seems circa 2020