3 ms·

They never got to the results of the competition and how the interpretable model did! From https://www.fico.com/en/newsroom/fico-announces-winners-of-inaugural

by mkl 7y ago

They never got to the results of the competition and how the interpretable model did! From https://www.fico.com/en/newsroom/fico-announces-winners-of-inaugural-xml-challenge https://www.fico.com/en/newsroom/fico-announces-winners-of-i...: "The team representing Duke University, which included Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang and Tong Wang, received the FICO Recognition Award acknowledging their submission for going above and beyond expectations with a fully transparent global model and a user-friendly dashboard to allow users to explore the global model and its explanations. The Duke team took home $3,000." Cynthia Rudin is one of the article's authors.

RSchaeffer 7y ago
But how did their model compare against others? The article only mentions how their interpretable model compared against their own ML attempts
- jph00 7y ago
  Their model didn't win. IBM's model won, based on actual metrics around useful insights.
  - Mathnerd314 7y ago
    The IBM team got $5,000 and the second place/honorable mention NYU got $2,000. So going by prize amounts, the Duke model was still pretty good. IBM turned the model/paper into a toolkit: https://www.ibm.com/blogs/research/2019/08/ai-explainability-360/ https://www.ibm.com/blogs/research/2019/08/ai-explainability... Their model seems to be a variant of decision trees that has a knob controlling how complicated the trees are. And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them.
    - A_No_Name_Mouse 7y ago
      > And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them. That's what you get if you use a black box for judging :-)
    - netcan 7y ago
      Reading "subjective" to mean "nonexistent" is a potentially big mistake. Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria is the definite goal. In this area, we don't exactly know how to define narrow, objective goals & subsequent criteria. We can definee goalposts, but not goals. These are guesses at useful markers of success, useful to the larger goal of useful/novel ai. Subjective goals have their own (massive problems), but since we can't objectively define the goals of ai research... we need to fall back on human subjectivity to define our subgoals.
      - dragonwriter 7y ago
        All objective criteria are chosen, directly or indirectly, based on subjective criteria.
        netcan 7y ago
        Also true. Subjectivity is unavoidable as long as we are relevant.. or so it seems circa 2020

by mkl 7y ago

RSchaeffer 7y ago

But how did their model compare against others? The article only mentions how their interpretable model compared against their own ML attempts

jph00 7y ago
Their model didn't win. IBM's model won, based on actual metrics around useful insights.
- Mathnerd314 7y ago
  The IBM team got $5,000 and the second place/honorable mention NYU got $2,000. So going by prize amounts, the Duke model was still pretty good. IBM turned the model/paper into a toolkit: https://www.ibm.com/blogs/research/2019/08/ai-explainability-360/ https://www.ibm.com/blogs/research/2019/08/ai-explainability... Their model seems to be a variant of decision trees that has a knob controlling how complicated the trees are. And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them.
  - A_No_Name_Mouse 7y ago
    > And the evaluation was completely subjective, so there's not any meaning to the Duke people losing besides that the judges didn't like them. That's what you get if you use a black box for judging :-)
  - netcan 7y ago
    Reading "subjective" to mean "nonexistent" is a potentially big mistake. Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria is the definite goal. In this area, we don't exactly know how to define narrow, objective goals & subsequent criteria. We can definee goalposts, but not goals. These are guesses at useful markers of success, useful to the larger goal of useful/novel ai. Subjective goals have their own (massive problems), but since we can't objectively define the goals of ai research... we need to fall back on human subjectivity to define our subgoals.
    - dragonwriter 7y ago
      All objective criteria are chosen, directly or indirectly, based on subjective criteria.
      - netcan 7y ago
        Also true. Subjectivity is unavoidable as long as we are relevant.. or so it seems circa 2020