2 ms·

Shouldn't this fall into the "trust by verify" category of things? Whilst your model may not be able to explain to you today why it can operate on future data,

• psv1 7y ago
  > You can always verify why it made the choices it did on old data. This makes no sense. Your model typically isn't asked to make predictions on "old data" because it was trained on it. And understanding why it made certain prediction isn't any different on "new" vs old data. You can look into bias-variance tradeoff, train-test splitting and cross validation to get a better picture of this.
  • shakna 7y ago
    > You can look into bias-variance tradeoff, train-test splitting and cross validation to get a better picture of this. Or, like in the article, you can make use of introspective models that can tell you what dimensions were fitted. This can more easily show you where overfitting is occurring. You can see the weights of the model and follow the decision network, which allows you to explain why something has occurred in a much clearer fashion. An introspective model gives you a much clearer picture of why certain choices were made, rather than the fuzzy picture you get running statistical analysis against a black-box'd algorithm.