4 ms·
Awesome post, thanks for writing it up! I haven't been very involved in using random forest at work (yet, I hope to), but I've done various mathematical progra
by geebee 10y ago
Awesome post, thanks for writing it up!
I haven't been very involved in using random forest at work (yet, I hope to), but I've done various mathematical programming work in the past to generate business insights (mainly through regressions and linear programming/optimization).
One thing that you make very clear through this blog post is how much value comes from the "non-technical" aspects of mathematical decision analysis. You have to see the application, find the data, clean the data, figure out what to actually put into the model, and get results in a way that can lead to an actual outcome with value.
Here's the thing, the reason I put "non-technical" in quotations is that it's actually a mix of technical and non-technical. You need to be aware of how these algorithms work and how they are implemented in order to have that insight. There's the old statement that everything looks like a nail to someone with a hammer, but knowing what tools are and what they can do can help frame issues in a way that you can approach them from new angles. This is why I do think it's worth learning various ML and other algorithms (like LP, NLP, etc) through contrived examples - once you understand them, you'll start to see the opportunity to apply them.
One last thing - kaggle. Kaggle is super fun, and I highly recommend it for people looking for an opportunity to try this out and learn it. However, good real world data science probably has less to do with making exceptional refinements to models. You know that data set you get when you are doing a kaggle competition? That's a huge amount of the actual work, right there.
You can do so much with basic RF and KNN (and with LP for that matter). This post is a pretty good illustration of this.
Anyway, pretty cool, thanks for sharing.
- selectron 10y agoAs another plug for Kaggle - it lets you know what is state of the art. For instance, from my Kaggle experience I know that gradient boosted decision tress (specifically xgboost) are virtually always superior to random forests. They are also basically just as easy to use, in contrast to neural nets which are not as user friendly. The machine learning step was probably the easiest step in this process, but it is also easy to leave simple gains on the table. Gradient boosted decision trees don't get nearly enough hype.
- bllguo 10y agoYeah on Kaggle I found myself getting routinely outperformed by people with scripts that simply ran a gradient boosted decision tree and nothing else. And yet the topic was never mentioned in my modeling and stats courses!
- cshenton 10y agoThat's because it's practically new. Tianqi Chen authored the R package for it (original release Aug 2015) and actually posts about it on the Kaggle forums quite frequently.
- cshenton 10y agoI would even argue it's easier to use that random forests. The standalone R package is super slick, with built in cross validation. And if you want to tune the hyperparameters you can just do it in caret. Really accessible stuff.