3 ms·

I'd love to see more articles from fast.ai students covering topics other than image (multi) classification. I've been gradually going through the course, and a

by namuol 8y ago

I'd love to see more articles from fast.ai students covering topics other than image (multi) classification. I've been gradually going through the course, and attempting to apply what I'm learning as I go, but I rarely see good results, especially with structured data.

autokad 8y ago
> "but I rarely see good results, especially with structured data." what algorithms are you using? I dont know if I am a ds neanderthal, but DNN are the last thing I try. xgboost and some ensembling with catboost usually gets very good results with little effort
- halflings 8y ago
  Wouldn't call the ensembling of xgboost models + catboost "little effort". One thing that is true however is that xgboost will usually give better performance without tuning, vs DNNs without any tuning / hyperparameter search.
  - autokad 8y ago
    psudocode here: yhat_xgboost = xg.predict(x_test) y_hat_cat = cat.predict(x_test) y_hat = .7 * y_hat_xgboost + .3 * y_hat_cat how is that not little effort?
    - namuol 8y ago
      I really think the FastAI course needs to be taken with a heaping dose of salt in the form of "here are all the non-DL solutions to common problems" -- there's a ton of material out there already, but based on anecdotal evidence, there are a lot of folks very new to ML & data science in general that are taking the course (i.e. people like myself).
      - harveynick 8y ago
        Have you looked at fast.ai's ML course? It might also be worth a try. http://forums.fast.ai/t/another-treat-early-access-to-intro-to-machine-learning-videos/6826 http://forums.fast.ai/t/another-treat-early-access-to-intro-... I'd also recommend the Andrew Ng Coursera course, which I talked about here: https://harveynick.com/2018/04/25/some-notes-on-the-andrew-ng-coursera-machine-learning-course/ https://harveynick.com/2018/04/25/some-notes-on-the-andrew-n...
- namuol 8y ago
  There are lectures that specifically use deep learning (in the form of fastAI's high-level `ColumnarData` API) to get state of the art results on the Rossmann Kaggle competition, but they _do_ borrow a good amount of feature engineering from the third-place winners from that competition. It might very well be that the DNN sledgehammer is simply the wrong tool for the job in most structured-data cases; if that's so, even an article outlining _why not_ to use FastAI for these scenarios (with actual performance comparisons) would be nice. That's all I'm looking for -- I should have mentioned that I am almost 100% new to ML/DL/data science & friends.
ericd 8y ago
It can be a bit tricky, there are a lot of gotchas, and it's generally hard to debug if you're new to it. I've found it helps to walk through and dig into everything the notebooks are doing, how the data loaders work, etc. The Fast.ai library does a lot of automagical self-configuration, which can save time if you're experienced with it and know what to expect, but can burn a lot of time when it does something you don't expect, and don't notice it doing that.
slow_donkey 8y ago
Dl for structured data is something I'm interested in as well. There's a lecture on fastai going over categorical embeddings which is interesting. There's also an airbnb article on the same topic that shouldn't be hard to find.

by namuol 8y ago

autokad 8y ago

> "but I rarely see good results, especially with structured data." what algorithms are you using? I dont know if I am a ds neanderthal, but DNN are the last thing I try. xgboost and some ensembling with catboost usually gets very good results with little effort

halflings 8y ago
Wouldn't call the ensembling of xgboost models + catboost "little effort". One thing that is true however is that xgboost will usually give better performance without tuning, vs DNNs without any tuning / hyperparameter search.
- autokad 8y ago
  psudocode here: yhat_xgboost = xg.predict(x_test) y_hat_cat = cat.predict(x_test) y_hat = .7 * y_hat_xgboost + .3 * y_hat_cat how is that not little effort?
  - namuol 8y ago
    I really think the FastAI course needs to be taken with a heaping dose of salt in the form of "here are all the non-DL solutions to common problems" -- there's a ton of material out there already, but based on anecdotal evidence, there are a lot of folks very new to ML & data science in general that are taking the course (i.e. people like myself).
    - harveynick 8y ago
      Have you looked at fast.ai's ML course? It might also be worth a try. http://forums.fast.ai/t/another-treat-early-access-to-intro-to-machine-learning-videos/6826 http://forums.fast.ai/t/another-treat-early-access-to-intro-... I'd also recommend the Andrew Ng Coursera course, which I talked about here: https://harveynick.com/2018/04/25/some-notes-on-the-andrew-ng-coursera-machine-learning-course/ https://harveynick.com/2018/04/25/some-notes-on-the-andrew-n...
namuol 8y ago
There are lectures that specifically use deep learning (in the form of fastAI's high-level `ColumnarData` API) to get state of the art results on the Rossmann Kaggle competition, but they _do_ borrow a good amount of feature engineering from the third-place winners from that competition. It might very well be that the DNN sledgehammer is simply the wrong tool for the job in most structured-data cases; if that's so, even an article outlining _why not_ to use FastAI for these scenarios (with actual performance comparisons) would be nice. That's all I'm looking for -- I should have mentioned that I am almost 100% new to ML/DL/data science & friends.

ericd 8y ago

It can be a bit tricky, there are a lot of gotchas, and it's generally hard to debug if you're new to it. I've found it helps to walk through and dig into everything the notebooks are doing, how the data loaders work, etc. The Fast.ai library does a lot of automagical self-configuration, which can save time if you're experienced with it and know what to expect, but can burn a lot of time when it does something you don't expect, and don't notice it doing that.

slow_donkey 8y ago

Dl for structured data is something I'm interested in as well. There's a lecture on fastai going over categorical embeddings which is interesting. There's also an airbnb article on the same topic that shouldn't be hard to find.