5 ms·

> "but I rarely see good results, especially with structured data." what algorithms are you using? I dont know if I am a ds neanderthal, but DNN are the last

> "but I rarely see good results, especially with structured data." what algorithms are you using? I dont know if I am a ds neanderthal, but DNN are the last thing I try. xgboost and some ensembling with catboost usually gets very good results with little effort

halflings 8y ago
Wouldn't call the ensembling of xgboost models + catboost "little effort". One thing that is true however is that xgboost will usually give better performance without tuning, vs DNNs without any tuning / hyperparameter search.
- autokad 8y ago
  psudocode here: yhat_xgboost = xg.predict(x_test) y_hat_cat = cat.predict(x_test) y_hat = .7 * y_hat_xgboost + .3 * y_hat_cat how is that not little effort?
  - namuol 8y ago
    I really think the FastAI course needs to be taken with a heaping dose of salt in the form of "here are all the non-DL solutions to common problems" -- there's a ton of material out there already, but based on anecdotal evidence, there are a lot of folks very new to ML & data science in general that are taking the course (i.e. people like myself).
    - harveynick 8y ago
      Have you looked at fast.ai's ML course? It might also be worth a try. http://forums.fast.ai/t/another-treat-early-access-to-intro-to-machine-learning-videos/6826 http://forums.fast.ai/t/another-treat-early-access-to-intro-... I'd also recommend the Andrew Ng Coursera course, which I talked about here: https://harveynick.com/2018/04/25/some-notes-on-the-andrew-ng-coursera-machine-learning-course/ https://harveynick.com/2018/04/25/some-notes-on-the-andrew-n...
namuol 8y ago
There are lectures that specifically use deep learning (in the form of fastAI's high-level `ColumnarData` API) to get state of the art results on the Rossmann Kaggle competition, but they _do_ borrow a good amount of feature engineering from the third-place winners from that competition. It might very well be that the DNN sledgehammer is simply the wrong tool for the job in most structured-data cases; if that's so, even an article outlining _why not_ to use FastAI for these scenarios (with actual performance comparisons) would be nice. That's all I'm looking for -- I should have mentioned that I am almost 100% new to ML/DL/data science & friends.

halflings 8y ago

Wouldn't call the ensembling of xgboost models + catboost "little effort". One thing that is true however is that xgboost will usually give better performance without tuning, vs DNNs without any tuning / hyperparameter search.

autokad 8y ago
psudocode here: yhat_xgboost = xg.predict(x_test) y_hat_cat = cat.predict(x_test) y_hat = .7 * y_hat_xgboost + .3 * y_hat_cat how is that not little effort?
- namuol 8y ago
  I really think the FastAI course needs to be taken with a heaping dose of salt in the form of "here are all the non-DL solutions to common problems" -- there's a ton of material out there already, but based on anecdotal evidence, there are a lot of folks very new to ML & data science in general that are taking the course (i.e. people like myself).
  - harveynick 8y ago
    Have you looked at fast.ai's ML course? It might also be worth a try. http://forums.fast.ai/t/another-treat-early-access-to-intro-to-machine-learning-videos/6826 http://forums.fast.ai/t/another-treat-early-access-to-intro-... I'd also recommend the Andrew Ng Coursera course, which I talked about here: https://harveynick.com/2018/04/25/some-notes-on-the-andrew-ng-coursera-machine-learning-course/ https://harveynick.com/2018/04/25/some-notes-on-the-andrew-n...

namuol 8y ago

There are lectures that specifically use deep learning (in the form of fastAI's high-level `ColumnarData` API) to get state of the art results on the Rossmann Kaggle competition, but they _do_ borrow a good amount of feature engineering from the third-place winners from that competition. It might very well be that the DNN sledgehammer is simply the wrong tool for the job in most structured-data cases; if that's so, even an article outlining _why not_ to use FastAI for these scenarios (with actual performance comparisons) would be nice. That's all I'm looking for -- I should have mentioned that I am almost 100% new to ML/DL/data science & friends.