6 ms·

Statistics is not trusted because it shouldn't be. It is incredibly easy to lie with statistics. There's even a tutorial: https://www.amazon.com/How-Lie-Statis

by mjfl 10y ago

Statistics is not trusted because it shouldn't be. It is incredibly easy to lie with statistics. There's even a tutorial: https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/0393310728 https://www.amazon.com/How-Lie-Statistics-Darrell-Huff/dp/03... Statistics is ultimately counting, and therefore is incredibly vulnerable to discretion in choosing "what counts". Take the unemployment rate as one example. When people realize that its not equivalent to "people who do not have a job", how can you complain that they trust statistics less? Expert authority is in decline because it should be, as there is an increasing body of evidence that experts, from politics to medicine, have almost no advantage in forecasting power than the average person. https://www.amazon.com/Expert-Political-Judgment-Good-Know/dp/0691128715/ref=sr_1_1?s=books&ie=UTF8&qid=1485036840&sr=1-1&keywords=expert+political+judgment https://www.amazon.com/Expert-Political-Judgment-Good-Know/d... Why should "experts" (often just pundits) have any authority when they have consistently demonstrated they deserve very little? Finally, the political slant of this article, going along with the decried "fake news", blaming the election results on these declines in authority, is pathetic. It's basically an extension of "the other side is filled with stupids" and has no credibility, no matter how you dress it in professional journalistic veneer.

amalcon 10y ago
Most people who have recently been to a pro baseball game know that there are times to mistrust statistics. The home team likes to give attending fans something to look at during the long periods of relative downtime. For each hitter who comes up, they like to come up with a statistic that favors the home team to display on the screens. There are lots of options, so they can take their pick, even if they sometimes come up with things like "At-bats on a Tuesday against teams out-of-division". However, everyone who's trusted any technology that accounts for quantum effects (e.g. modern computer processors) has trusted statistics. Everyone who's ever received a modern medical treatment (of any kind) has trusted statistics. The hard part is knowing when you should, and when you should not, trust statistics. Frankly I don't think there's anyone who can do that 100% of the time -- but I think most people could probably learn to do well enough to benefit. The learning part is tough, though.
psyklic 10y ago
Implying that most experts have agendas that supersede the truth is outrageous and frankly dangerous. Saying we should just give up on using data to make predictions is ludicrous -- even if it hasn't worked before in some situations. We should hopefully be in agreement that using data to make decisions is preferred to winging it. We should also hopefully be in agreement that we should trust people who devote their life to studying an issue over someone new who merely has gut feelings about it -- and hence is even more biased in many additional ways. The thrust of your argument is valid -- summarizing data inherently leaves things out. However, the solution to this is not to give up and throw statistics out the window. And the solution is definitely not to unfairly discredit people who devote their lives to these issues. Rather, if you see statistics, wonder how they were generated and why some data was included and others thrown out. By being open and candid about the decisions made, the statistics are meaningful. Just giving numbers without providing details of how they were achieved is not valid statistical work. From my experience, most scientists (and myself in my work) care about the truth over everything else. I use statistics to better understand my data. My objective is always to choose the most sensible path in analyzing it that allows me to most accurately answer the questions I pose.
- engi_nerd 10y ago
  > Implying that most experts have agendas that supersede the truth is outrageous and frankly dangerous. I think that "experts with agendas that supersede the truth" claim is true if you consider the pool of pundits that regularly appear in mainstream media. Unfortunately they are not representative of scientists like yourself.
  - psyklic 10y ago
    This attitude of not trusting experts and not trusting statistics unfortunately permeates from the political arena into science. The belief that all statistics are not to be trusted is a clear danger to our society. Not trusting climate scientists. Not even trusting simple statistics on vaccinations. As another poster noted, the only alternative is ignorance.
    - 10y ago
diego_moita 10y ago
Everything you wrote would make sense if we had a good alternative to statistics and expertise. The only alternative is ignorance. > there is an increasing body of evidence that experts Is this "body of evidence" non-statistical?
- lliamander 10y ago
  > The only alternative is ignorance. Sometimes ignorance isn't even an option: it's mandatory. The only choice we are left with is whether to admit to ourselves that we are ignorant. The choice isn't between governing peoples lives using statistics vs. governing peoples lives based on gut-instinct (or whatever). It's between using weak statistics as a pretext for governing peoples lives vs. just admitting there's a lot we don't know and choosing to give people space to figure out things for themselves. Jim Manzi, who literally built his business off of effective use of statistics, spent 10 years figuring out where to put the snickers bar at the checkout stand[0] and he admitted it was a really hard problem. Yet it's a joke compared to some issues that experts claim they can solve. I have to depend upon statistical reasoning, both directly and indirectly, for work. I understand and appreciate the validity of the methods of statistics. That doesn't mean I'm going to trust any statistics that are used to back a policy proposal. [0]http://www.weeklystandard.com/jim-manzi-talks-science-knowledge-and-freedom/article/666107 http://www.weeklystandard.com/jim-manzi-talks-science-knowle...
- zigzigzag 10y ago
  There are plenty of alternatives to statistics and expertise, as the sibling comment points out, letting people get on with things themselves (i.e. the default action of do nothing) is essentially a delegation to the wisdom of the crowds. Gut feeling and intuition are also alternatives people frequently rely on, and aren't necessarily nonsense: the brain is very good at synthesising conclusions at the level of the subconscious that surface consciously as hunches or gut feelings. The "body of evidence" that self-proclaimed experts are frequently wrong is rarely presented statistically, but it could be so. However this would be circular, it'd be an expert using statistics to claim that experts using statistics can't be trusted. That's a time when 'common sense' knowledge is sufficient: if you've seen endless stories in the press about how experts predit X, and none about experts predicting not X, then when X doesn't happen you can safely lower your trust in experts without needing an Excel spreadsheet to prove it to yourself.
mikeash 10y ago
BLS is quite up-front about what the "unemployment rate" measures, and provides six different numbers using different definitions in order to better capture the big picture. I don't see why I can't complain if people's reaction to this information is to trust statistics less, rather than spend thirty seconds reading about where the numbers come from.
- nkurz 10y ago
  One argument I've seen is that although BLS provides many metrics, they consciously avoid several useful metrics that paint a realistic but negative picture of the economy. If I understand it, one big complaint is that "long term discouraged workers" are dropped from U6, and BLS does not publish a metric showing total labor force participation. I don't know enough about the details to assess know if this is true, but I thought this piece seemed like an objective criticism: http://www.shadowstats.com/article/c810x.pdf http://www.shadowstats.com/article/c810x.pdf
  - mikeash 10y ago
    That seems like a fair criticism. But at the same time, as long as they don't pretend that the numbers say more than they do, I don't see a big problem with them.

by mjfl 10y ago

amalcon 10y ago

Most people who have recently been to a pro baseball game know that there are times to mistrust statistics. The home team likes to give attending fans something to look at during the long periods of relative downtime. For each hitter who comes up, they like to come up with a statistic that favors the home team to display on the screens. There are lots of options, so they can take their pick, even if they sometimes come up with things like "At-bats on a Tuesday against teams out-of-division". However, everyone who's trusted any technology that accounts for quantum effects (e.g. modern computer processors) has trusted statistics. Everyone who's ever received a modern medical treatment (of any kind) has trusted statistics. The hard part is knowing when you should, and when you should not, trust statistics. Frankly I don't think there's anyone who can do that 100% of the time -- but I think most people could probably learn to do well enough to benefit. The learning part is tough, though.

psyklic 10y ago

Implying that most experts have agendas that supersede the truth is outrageous and frankly dangerous. Saying we should just give up on using data to make predictions is ludicrous -- even if it hasn't worked before in some situations. We should hopefully be in agreement that using data to make decisions is preferred to winging it. We should also hopefully be in agreement that we should trust people who devote their life to studying an issue over someone new who merely has gut feelings about it -- and hence is even more biased in many additional ways. The thrust of your argument is valid -- summarizing data inherently leaves things out. However, the solution to this is not to give up and throw statistics out the window. And the solution is definitely not to unfairly discredit people who devote their lives to these issues. Rather, if you see statistics, wonder how they were generated and why some data was included and others thrown out. By being open and candid about the decisions made, the statistics are meaningful. Just giving numbers without providing details of how they were achieved is not valid statistical work. From my experience, most scientists (and myself in my work) care about the truth over everything else. I use statistics to better understand my data. My objective is always to choose the most sensible path in analyzing it that allows me to most accurately answer the questions I pose.

engi_nerd 10y ago
> Implying that most experts have agendas that supersede the truth is outrageous and frankly dangerous. I think that "experts with agendas that supersede the truth" claim is true if you consider the pool of pundits that regularly appear in mainstream media. Unfortunately they are not representative of scientists like yourself.
- psyklic 10y ago
  This attitude of not trusting experts and not trusting statistics unfortunately permeates from the political arena into science. The belief that all statistics are not to be trusted is a clear danger to our society. Not trusting climate scientists. Not even trusting simple statistics on vaccinations. As another poster noted, the only alternative is ignorance.
  - 10y ago

diego_moita 10y ago

Everything you wrote would make sense if we had a good alternative to statistics and expertise. The only alternative is ignorance. > there is an increasing body of evidence that experts Is this "body of evidence" non-statistical?

lliamander 10y ago
> The only alternative is ignorance. Sometimes ignorance isn't even an option: it's mandatory. The only choice we are left with is whether to admit to ourselves that we are ignorant. The choice isn't between governing peoples lives using statistics vs. governing peoples lives based on gut-instinct (or whatever). It's between using weak statistics as a pretext for governing peoples lives vs. just admitting there's a lot we don't know and choosing to give people space to figure out things for themselves. Jim Manzi, who literally built his business off of effective use of statistics, spent 10 years figuring out where to put the snickers bar at the checkout stand[0] and he admitted it was a really hard problem. Yet it's a joke compared to some issues that experts claim they can solve. I have to depend upon statistical reasoning, both directly and indirectly, for work. I understand and appreciate the validity of the methods of statistics. That doesn't mean I'm going to trust any statistics that are used to back a policy proposal. [0]http://www.weeklystandard.com/jim-manzi-talks-science-knowledge-and-freedom/article/666107 http://www.weeklystandard.com/jim-manzi-talks-science-knowle...
zigzigzag 10y ago
There are plenty of alternatives to statistics and expertise, as the sibling comment points out, letting people get on with things themselves (i.e. the default action of do nothing) is essentially a delegation to the wisdom of the crowds. Gut feeling and intuition are also alternatives people frequently rely on, and aren't necessarily nonsense: the brain is very good at synthesising conclusions at the level of the subconscious that surface consciously as hunches or gut feelings. The "body of evidence" that self-proclaimed experts are frequently wrong is rarely presented statistically, but it could be so. However this would be circular, it'd be an expert using statistics to claim that experts using statistics can't be trusted. That's a time when 'common sense' knowledge is sufficient: if you've seen endless stories in the press about how experts predit X, and none about experts predicting not X, then when X doesn't happen you can safely lower your trust in experts without needing an Excel spreadsheet to prove it to yourself.

mikeash 10y ago

BLS is quite up-front about what the "unemployment rate" measures, and provides six different numbers using different definitions in order to better capture the big picture. I don't see why I can't complain if people's reaction to this information is to trust statistics less, rather than spend thirty seconds reading about where the numbers come from.

nkurz 10y ago
One argument I've seen is that although BLS provides many metrics, they consciously avoid several useful metrics that paint a realistic but negative picture of the economy. If I understand it, one big complaint is that "long term discouraged workers" are dropped from U6, and BLS does not publish a metric showing total labor force participation. I don't know enough about the details to assess know if this is true, but I thought this piece seemed like an objective criticism: http://www.shadowstats.com/article/c810x.pdf http://www.shadowstats.com/article/c810x.pdf
- mikeash 10y ago
  That seems like a fair criticism. But at the same time, as long as they don't pretend that the numbers say more than they do, I don't see a big problem with them.