3 ms·

In general, if you are a machine learning person and you are using a regression system that "explains" itself somehow, I highly recommend frequently looking at

by jerf 4y ago

In general, if you are a machine learning person and you are using a regression system that "explains" itself somehow, I highly recommend frequently looking at those explanations. Once you're excited that you've built a 90%+ model, crack open the hood of that model before you deploy it and do a sanity check on it. While ML algorithms can occasionally pluck some interesting insights out of the data, in my admittedly-limited experience I've also seen some models put together out of what is clearly just garbage, meaningless weights placed on attributes that can't possibly be that relevant carving out a hyperplane through hyperspace that can't possibly represent reality, despite all the efforts of ML models to hold out validation data and rotate their validation data and everything else to avoid overfitting.

mjhay 4y ago
Once these systems get complicated enough, it gets really hard to do that. Unless you are doing linear models or tree ensembles, it can be really difficult to get a clear picture of variable importance/effects. It's one thing to extrapolate locally (fine for linear models), but that has little to do with anything relevant when there's lots of higher-order interactions.
- RodgerTheGreat 4y ago
  What is a safer assumption? - My model is accurate, for reasons I can't understand, verify, or externally justify - My model is not accurate edit: to be more clear on my stance, if a modeling technique is too difficult to justify or explain, it is not a good technique and should not be used, even if the results seem appealing.
  - mjhay 4y ago
    The second one. It's important to have a clear picture of biases and how your model could fail to generalize. In the case of social data from a deeply biased society, these issues can be seen a mile away.
- jerf 4y ago
  "Once these systems get complicated enough, it gets really hard to do that." That's why I qualified it. Obviously if you're working in neural nets, you're not going to be cracking open that hood. But one of the dirty secrets of ML is you don't always need those techniques. There's the space of problems where linear regression and similar techniques are all you need, there's the class of problems where it doesn't matter what you throw at it you're not getting anything, and the "in between" where deep learning and such helps certainly has some interesting problems in the space, but it isn't necessarily the majority of problems you encounter in the wild. The scary thing is that not being able to crack open the hood doesn't mean the model isn't complete nonsense. It just means you can't examine it.

by jerf 4y ago

mjhay 4y ago

Once these systems get complicated enough, it gets really hard to do that. Unless you are doing linear models or tree ensembles, it can be really difficult to get a clear picture of variable importance/effects. It's one thing to extrapolate locally (fine for linear models), but that has little to do with anything relevant when there's lots of higher-order interactions.

RodgerTheGreat 4y ago
What is a safer assumption? - My model is accurate, for reasons I can't understand, verify, or externally justify - My model is not accurate edit: to be more clear on my stance, if a modeling technique is too difficult to justify or explain, it is not a good technique and should not be used, even if the results seem appealing.
- mjhay 4y ago
  The second one. It's important to have a clear picture of biases and how your model could fail to generalize. In the case of social data from a deeply biased society, these issues can be seen a mile away.
jerf 4y ago
"Once these systems get complicated enough, it gets really hard to do that." That's why I qualified it. Obviously if you're working in neural nets, you're not going to be cracking open that hood. But one of the dirty secrets of ML is you don't always need those techniques. There's the space of problems where linear regression and similar techniques are all you need, there's the class of problems where it doesn't matter what you throw at it you're not getting anything, and the "in between" where deep learning and such helps certainly has some interesting problems in the space, but it isn't necessarily the majority of problems you encounter in the wild. The scary thing is that not being able to crack open the hood doesn't mean the model isn't complete nonsense. It just means you can't examine it.