4 ms·
an active kaggler here. I dont use NN because they simply don't have great accuracy, and most importantly they have a huge amount of variance. this is mostly
by autokad 8y ago
an active kaggler here.
I dont use NN because they simply don't have great accuracy, and most importantly they have a huge amount of variance. this is mostly because the data on kaggle is not very large. the gbm trifecta (xgboost, catboost, lgbm) also does really really well.
> "Kaggle prioritizes chasing a metric, but real-world data science has more considerations."
this is counter to your point. most real-world considerations need things like model explanability.
I notice that things dont make hacker news if they are doing anything other than NN. model with xgboost gets X% accuracy - crickets. model with X-Y% accuracy with DNN - headline news.
I also notice that teams in industry tend to throw a DNN at a problem and never try something more simpler like xgboost. I saw a team with an LSTM for text lament they had 80% accuracy on training/evaluation, but when pushed to prod dropped down to 50%. I saw the errors they were getting, I said: maybe its too complicated and not generalizing well, have you tried xgboost?
they retorted LSTM's w/ word2vec is very robust. I thought, obviously its not given your results. I tried to offer the idea that word2vec was trained on an entirely different kind of corpus (also a problem when trying to use word2vec in kaggle)
- soraki_soladead 8y agoI agree good models like xgboost get buried and inexperienced practitioners jump to deep models too quickly. Often without understanding how to properly architect and tune them. Always start with a simple baseline (EDIT: and good process). However, what's lacking in the ML practitioner community is nuance. Some applications need deep models some problems need xgboost. There isn't a "best" model in text classification because it depends on your data and problem.
- asah 8y agoThis chart is great for newbies, perhaps give it an update? https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html https://scikit-learn.org/stable/tutorial/machine_learning_ma...
- autokad 8y agoI'm not a huge fan of the chart as it looks pretty 2014 dated. SVMs are rarely used anymore, so perhaps if you replaced all of them (including in the regression part) with gbm. Didn't notice any mention of logistic regression, which I still heavily use as well. dimentionality reduction is part of any step, which depends on number of observations, features, and model. my favorite is l1 or pca, but I am not afraid to use stepwise regression or some tree method.