18 ms·
This reminds me of the sort of light poking of common "correlation is not the same as causation" and "beware of confounding factors" statistical failures behind
by xoa 5y ago
This reminds me of the sort of light poking of common "correlation is not the same as causation" and "beware of confounding factors" statistical failures behind the Church of the Flying Spaghetti Monster "clearly reduction in pirates has caused global warming!" [0]. But it's a major in modern public discourse, and one for once that I'm quite willing to lay heavily at the feet of the public education system. Easily one of the most valuable classes I took in my entire time K-12 was AP Probability & Stats as a sophomore, but that was an entirely optional class with restricted openings anyway (a single teacher in a school of 1200+) which the vast majority never took even where it was offered. Yet interpreting the deluge of data in the modern world requires some level of being able to reason about things probabilisticly and have a sense of what actually goes into a measure of "significance", null hypothesis and the danger of result-driven analysis finding links that don't exist and/or cannot possibly be causitive, how sampling a population works and what the error bars look like at different sizes, random vs biased distributions, the underlying distributions, confounding factors etc. I really wish in general kids got started on some light probability thinking as early as possible, in elementary school even, without any real math (let alone calc and such) yet, but just some initial stuff to start to illustrate the mindset. Lots of very fun games and hands-on exercises with dice and so on use or could be made to use important aspects of probability and its misuse.
At any rate, I'm certainly not an expert. But there seems to be some missing BS filter where people can recognize something as silly if the example is silly enough but not in the exact same logic fail for something that seems "more reasonable somehow".
----
0: https://www.spaghettimonster.org/wp-content/uploads/2007/10/hq-graphcopy2_800.jpg https://www.spaghettimonster.org/wp-content/uploads/2007/10/...
- Hnrobert42 5y ago> … there seems to be some missing BS filter where people can recognize something as silly if the example is silly enough but not in the exact same logic fail for something that seems "more reasonable somehow". Maybe combined with motivated reasoning.
- giantg2 5y agoAnd our justice system is largely built off of what is reasonable, yet it's undefined.
- throwawayboise 5y agoExactly. What it is that one wants to be true has a huge effect on how one interprets something.
- LongTimeAnon 5y agoSounds like 13% of all silly correlations cause 50% of the fallacy.
- kwertyoowiyop 5y agoStatistics is probably more useful than Calculus for just about everyone.
- AzzieElbab 5y agoSure. “An average human has one breast and one testicle” - N.N. Taleb
- sokoloff 5y agoCorollary: If a human has two legs, they have an above-average number of legs.
- fer 5y agoAlso, most humans have more fingers than the average.
- sokoloff 5y agoI agree that people could have a more effective BS filter, but I think there’s also some obligation to not publish BS in the first place at the risk of your trustworthiness reputation. People intentionally or repeatedly inadvertently publishing BS should be called out and their opinions down-weighted heavily. People having a good BS filter is one step in the chain towards this down-weighting, but isn’t the whole chain. If anything, people slinging BS today are likely to gain additional reach and opinion-weight, rather than to lose it.
- indigochill 5y ago> should be called out and their opinions down-weighted heavily But thinking about this from a loosely signal-process-y angle, once you have a BS signal in one channel that routes to a a particular set of minds, how does one route the corrective signal to that same set of minds (as opposed to, say, just your local friends who already agree with you anyway)? And even supposing you do, those minds are already programmed to process these signals in particular ways that probably means they're predisposed to accepting one or the other before either signal even arrives. The answer to this has always been to address the root of the problem, the preprogramming, through (relatively) uniform mass education, because that's the one and only place where you can (relatively) uniformly cram ideas into everyone's brain before they scatter to the winds and begin fancying themselves "free thinkers".
- sokoloff 5y agoIf you integrate over time, an erroneous signal (a stray bit of noise) can still result in a few minds here and there having the wrong information. That’s not ideal, but it’s unavoidable and not catastrophic so long as the source of the bad information systemically has reduced amplitude next time. Right now, sources of bad information get systemically louder, not quieter, over time. That’s way worse.
- geofft 5y agoThe post does exactly this, calling out Alex Berenson by name - but despite its best efforts, it will have very little impact on the people paying him to receive BS directly from his Substack. I don't think your opinion is unpopular because it's wrong. It's unpopular because it's empirically impractical. People don't suffer a reputational hit for publishing BS, and saying "But they should" doesn't get us anywhere (as you yourself point out). How do you propose that we actually cause reputational hits for such people? One way is to teach additional statistical literacy to the general public. There are probably other proposals worth trying and I'd be interested in hearing your takes on what they are. (One that might be effective - though certainly controversial - is for the government to say that Alex Berenson is actively putting lives in danger with lies and that, if the untruth of his statements can be proven in court, he should be subject to criminal penalties, just like Elizabeth Holmes is on trial for lying about her blood tests. But that seems a lot less good for society as a whole than teaching statistical literacy.)
- DarylZero 5y agoI took high school statistics. I don't think it really had the kind of effect on students that you're going for. Statistics like any math class was just another pointless and imposed game of symbol manipulation, for most. Not something that affects how they see the world or how they process disinformation (which disables rational thinking by appeals to emotion, so rational capabilities aren't necessarily even the issue). Humans are naturally using statistical type reasoning all the time and are very good at it. But when it comes to things like in-group out-group consensus-forming conformity mechanisms, the whole point is they _overcome rationality_ for social cohesion. To follow rationality instead of the group, you have to leave the group, which is unthinkable, so your mind prevents you from thinking it long enough to change your mind, by emotional terminations.
- newsbinator 5y agoThe best (and only) thing I learned in high school statistics is that no matter how many times you flip a coin, each flip has an equal chance of landing on heads or tails.
- aj7 5y agoThat’s huge though.
- GuB-42 5y agoA fair coin. It is important because if you flip a coin 20 times and get 20 tails, there is probably something fishy with that coin and you probably should bet tails. If it is 100 out of 100 (or even 95 out of 100), there is effectively zero chance that the coin is fair. This can be modeled using Bayesian statistics. You start by assuming with a reasonably high probability that the coin is fair, then you revise your assumptions as you get more and more data. The general idea is: if a coin flip lands on tails for too many times, you should probably bet on tails. You shouldn't try that in casinos: games are seriously checked for fairness (of course, the rules make it so that the house has an edge). But in an informal setting, it can get you a small advantage.
- 5y ago
- jkhdigital 5y agoStatistics are meaningless without a rigorously examined causal model of the phenomenon under investigation. In my experience of statistics education, the art of crafting causal theories was scarcely addressed.
- jmugan 5y agoCan you point to a good source on that? The stats books that I've seen seem to treat it as a collection of tools, and causality in computer science (AI) seems like a separate subfield with the do operator and all that.
- jkhdigital 5y agoI guess you may be familiar with Judea Pearl’s work already, but he did write a popular treatment of the subject in The Book of Why. I’m not trying to put computer scientists on a pedestal, but there is something about the uncompromising rigor that comes with putting abstractions inside brainless machines.
- eliasmacpherson 5y agoThanks for the tip, I'd read some material of his from the 1980s that helped me to start understanding bayesian networks, it inspired me to revisit my university statistics course which was fruitful. I am a sucker for pop sci!
- lolinder 5y agoThe average citizen doesn't need to craft casual theories, they need to be able to look at them critically. Yes, an introductory statistics course doesn't cover everything, but it's absolutely the 20% effort that gets 80% of the results. Most of the worst misinformation I've seen lately surrounding covid, vaccines, etc, would be solved if everyone had a basic understanding of introductory stats as currently taught. If we can get there, then we can talk about what we can do to improve things from there.
- 5y ago
- chiefalchemist 5y ago> But it's a major in modern public discourse, and one for once that I'm quite willing to lay heavily at the feet of the public education system. Should the general population be better at such things? Yes, of course. However, in nearly every case such things reach a broader audience via mainstream media. And how many times have we seen those entries confuse correlation with causation? We've seen it so many times that it's safe to assume it's intentional. Surely, after each incident of such negligence a teacher or professor or math savvy citizen reaches out to correct them. Yet? Never a correction or retraction?? Never a spark of "we need to educate our journalists"? Repeat something often enough and it becomes truth in the minds of the receivers. Toss in confirmation bias and echo chambers and even if your better educated the masses, the media and those "journalists" would mitigate public's understanding.
- photochemsyn 5y agoIn addition, the entire advertising-based consumer economy is based on getting the potential consumer to buy the product by whatever means available. A public education system that creates a population conditioned to believe whatever any 'authority figure' says is also a system designed to create a population ripe for exploitation by advertisers. Similarly, the ideal authoritarian state desires a population that is generally ignorant and obedient, and that's what's been created in much of the United States. A general population that has the tools and skills needed to independently analyze the claims of government authority figures and cable TV and Internet advertisers, that's not what an elitist-authoritarian system desires. It's very sad to see people who completely lack these tools and skills attempting to do their own well-intentioned analysis, they're so easily manipulated by dishonest actors. They know enough to distrust 'authority figures', but not enough to conduct independent evaluations of claims. Such people have been sabotaged by the educational system.
- tgv 5y agoAnd even then, classical null hypothesis testing is fraught with problems. Best stick to comparing models that explain the effect (instead of some default linear model where basically everything is correlation) and use Bayesian statistics.
- SubiculumCode 5y agoI also encountered recently a chart indicating that all causes mortality is higher in the vaccinated than the unvaccinated. It was clear to me, but not to the casual theater, that the median vaccinated individual is much older than the unvaccinated individual, and older adults tend to die more frequently than younger adults.
- lordnacho 5y agoI would go as far as sacrificing calculus and vectors for stats classes if it came to a showdown. There's an absolute jungle of information we're presented with each day, and clearly many people would just straight up buy the vaccines-cause-death statistics uncritically. There's no critical thinking without critical stats. How are the numbers made, what do they mean? Just about every field requires you to understand this, especially the social science fields where we're talking about some quite substantial issues like replication crisis. Things like economics as well, they're things everyone wants to understand but we don't give people the tools. Somehow we have also missed out causality, when we've had the tools for a while. I reckon it's actually quite teachable though it currently feels like an advanced subject due to historical quirks.
- mycologos 5y agoIs there any society of, say, 1mil+ people that 1) doesn't have a strong government that strictly controls information, and 2) has a population that doesn't have this "major [problem] in public discourse"? Has anybody ever pulled this off?
- pstuart 5y agoI think we teach math wrong -- it's taught like everybody is going to go the distance with it, whereas that's a small minority. We should teach math literacy -- how to use math to understand the world we live in, e.g. some stats as you mentioned, basic finance (the glories of compound interest), sizing stuff, etc.
- path411 5y agoThe BS filter is only obvious because we all agree that scientifically, watching a movie can't change your medical risk. Try swapping the graph labels with: Took vitamin D pills, Excercised 30mins a week, Got the vaccine. Most would agree that all three points probably have some level of health benefit. Suddenly the cause vs correlation can be much more difficult for the average person to determine. Do all three reduce the death risk? Do some of them just also trend with more health conscious individuals? Are any of the three just completely bogus?