3 ms·

I think people are missing the point of this article. The author isn't saying that people without math degrees should go out and try to be statisticians. What

by physcab 17y ago

I think people are missing the point of this article. The author isn't saying that people without math degrees should go out and try to be statisticians. What he's really saying is that anyone with the right mindset can do statistical analysis. I believe this to be 100% true. If you are in the type of position that looks at a lot of data, you will naturally develop an eye for trends and outliers. Then if you are curious enough, you might draft up some conclusions which you then bring to attention to others. They will say, so how does this compare to the population as a whole? So you go back and look through the numbers. Even if you don't have a background in math, you're lucky that you've got Google. You type in "how to handle outliers" or "how to analyze trends in data". You learn about the basics (ie normalization, standard deviation, means, percentages etc). Then you go back and look at your data again. It's a continual process of discovery and one that you can learn through application. For the record, I really like FlowingData.com. There have been some awesome posts, particularly the one about creating a U.S County thematic map.

joe_the_user 17y ago
If you are in the type of position that looks at a lot of data, you will naturally develop an eye for trends and outliers. So what do you base your "eye" on? A lot of good statistics involves debunking the trends that people with an "eye" read into the data without them really being there. The human is a fantastic instrument for seeing patterns - it just has the small problem that it can easily see patterns in entirely random data - especially patterns with complex, non-normal distributions (lot of fractal random distributions are great for seeming like they were produced intentionally but without maths, how would you know that?). It's true that sometimes when you see a trend, it really is there - sometimes. What differentiates a good statistician from a crank or snake-oil salesman is that they use a rigorous method to sift the trends they see. Just consider that the "Eliot Wave Theory" is huge, utterly bogus theoretical method based on "seeing patterns" of stock market movement that can't be found by rigorous methods. We don't actually need more of that kind of thing.
- physcab 17y ago
  > So what do you base your "eye" on? This article refers to everyday people who might look at "ordinary" data. We're not talking about government climatologists or financial analysts here. We're talking about people in Sales and Marketing who might want to know the demographics of their customers, perhaps by location for example. So in this case, I mean quite literally, use what you see in front of you to identify anomalies. Maybe you notice that the age bracket 18-24 year olds seems significantly higher than the rest of the population. I don't want to discredit the need for statisticians. I'm doing my PhD research on machine learning techniques. But in most practical scenarios, you don't need to resort to such a rigorous analysis. You can get quite far just using run-of-the-mill techniques (mean, standard deviation, normalization, what have you)

by physcab 17y ago

joe_the_user 17y ago

If you are in the type of position that looks at a lot of data, you will naturally develop an eye for trends and outliers. So what do you base your "eye" on? A lot of good statistics involves debunking the trends that people with an "eye" read into the data without them really being there. The human is a fantastic instrument for seeing patterns - it just has the small problem that it can easily see patterns in entirely random data - especially patterns with complex, non-normal distributions (lot of fractal random distributions are great for seeming like they were produced intentionally but without maths, how would you know that?). It's true that sometimes when you see a trend, it really is there - sometimes. What differentiates a good statistician from a crank or snake-oil salesman is that they use a rigorous method to sift the trends they see. Just consider that the "Eliot Wave Theory" is huge, utterly bogus theoretical method based on "seeing patterns" of stock market movement that can't be found by rigorous methods. We don't actually need more of that kind of thing.

physcab 17y ago
> So what do you base your "eye" on? This article refers to everyday people who might look at "ordinary" data. We're not talking about government climatologists or financial analysts here. We're talking about people in Sales and Marketing who might want to know the demographics of their customers, perhaps by location for example. So in this case, I mean quite literally, use what you see in front of you to identify anomalies. Maybe you notice that the age bracket 18-24 year olds seems significantly higher than the rest of the population. I don't want to discredit the need for statisticians. I'm doing my PhD research on machine learning techniques. But in most practical scenarios, you don't need to resort to such a rigorous analysis. You can get quite far just using run-of-the-mill techniques (mean, standard deviation, normalization, what have you)