4 ms·
I'm glad you're being open minded about responses. While I completely agree with you on the massive backlog of problems that could be solved with very simple d
by mendicantB 13y ago
I'm glad you're being open minded about responses.
While I completely agree with you on the massive backlog of problems that could be solved with very simple data analysis (this is the reason I am in the field), your thoughts about data modeling techniques are very much off the mark and sound like something read from a dated statistics textbook.
First, on logistic regression:
-I can tell you with certainty that logistic regression is not vastly underused, and is consistently among the most popular machine learning algorithms [1]. From personal experience I think it's likely the most powerful model there is.
-People are excited about Bayesian models and NN because they work. Actually, most logistic regression in practice is Bayesian by nature (regularization much?). More so, logistic regression is actually a piece of what powers NNs, and it's incorrect to directly compare the two. To put it more blunt, these are drastic improvements to logistic regression that allow the technique to solve greater problems.
-You are trivializing the amount of training needed to properly apply a classifier by calling it simple.
Second, there are plenty of basic questions in the data field to answer [2]:
-Doing anything with large amounts of data is still complicated and hard
-Unifying multiple data sets
-General model comparison
-Most modeling assumptions and as a result conclusions reached from them are wrong
-Communicating results to people. This is the most underestimated. One of the reasons we have such a backlog of problems that haven't been touched by simple data analysis techniques is the difficulty involved in explaining results. P-values are fucking meaningless and massively misinterpreted and misused. I won't start on frequentist statistics.
1) http://www.quora.com/What-are-the-top-10-data-mining-or-machine-learning-algorithms http://www.quora.com/What-are-the-top-10-data-mining-or-mach...
2)http://normaldeviate.wordpress.com/2012/06/21/90/ http://normaldeviate.wordpress.com/2012/06/21/90/
- erikpukinskis 13y agoWhen I say logistic regression is underused, I don't mean underused among data scientists. We have an incredible shortfall of "data scientists". What I mean is there is data sitting on computers somewhere, which is a logistic regression away from giving someone some information that they could use to make better decisions. But there's no one at Debbie's Diner who is capable of doing the regression, so it goes undone. And the computer scientist who eats Debbie's waffles every Saturday spends his weekends reading about SpaceX instead of pulling her data into R and giving her some insights that would give her a much safer retirement.