3 ms·

I work for a company that does a lot of Data Science Consulting. And I'm still surprised by how many companies I see that try to use complex methods to solve p

by apohn 11y ago

I work for a company that does a lot of Data Science Consulting. And I'm still surprised by how many companies I see that try to use complex methods to solve problems that can be solved by high school statistics and a little lateral thinking. There's just so much hype around "Data Science" right now. I see this a lot when I walk into companies where I'm introduced to employees with "Joe has a Master's in Applied Mathematics. Nitya has a PhD in Computer Science with a focus on AI." Some managers love the idea that they are going to spearhead some initiative that will solve some problem in the most complex way possible, thus getting some visibility from people in the C-Suite. A few years ago we worked with a customer who was trying to reduce issues with their industrial processes. Most of the battle was just processing the raw sensor data quickly enough. Using Stats 101 methods to predict issues, they ended up saving a lot(!!) of money every year and were extremely happy. They were so happy they agreed without hesitation to present the work at some major conferences. During one conference presentation one audience member basically said "You're not using Machine Learning so you're not doing predictive Analytics or Data Science" and basically acted like we were a bunch of fakers and idiots. The hype is strong with Data Science. Absolutely agree with the following from the post. "Talk to customers. Watch what products sell and which ones don’t. Think about the economics that drive the business and how you can help it succeed more. Knowing what matters is the real key to being an effective data scientist." All that being said, I do see plenty of the opposite as well. Lots of people only know about sums and means, so that's all they want to use. There are plenty of cases where advanced methods make more sense. Knowing the difference is why you hire a good Data Scientist.

slv77 11y ago
Asking the stupid question... Was the stats 101 basic control charts and statistical sampling? Do you have a link to the paper?
- apohn 11y ago
  Stats 101 is basically what you would learn in the first applied statistics course taken by an undergraduate (or advanced secondary schooling). Mean, Median, Standard Deviations, Z scores, T-tests, etc. For control charting, just look at the "Control Charts" page on Wikipedia, which provides different methods used to figure out when data point are outside of limits. I do have a published paper, but I try to avoid linking my internet accounts to my real life. So I'm reluctant to link to it as that would give away my identity.

by apohn 11y ago

slv77 11y ago

Asking the stupid question... Was the stats 101 basic control charts and statistical sampling? Do you have a link to the paper?

apohn 11y ago
Stats 101 is basically what you would learn in the first applied statistics course taken by an undergraduate (or advanced secondary schooling). Mean, Median, Standard Deviations, Z scores, T-tests, etc. For control charting, just look at the "Control Charts" page on Wikipedia, which provides different methods used to figure out when data point are outside of limits. I do have a published paper, but I try to avoid linking my internet accounts to my real life. So I'm reluctant to link to it as that would give away my identity.