6 ms·
I've become increasingly convinced that the idea of averaging is one of the biggest obstacles to understanding things... it contains the insidious trap of feeli
by _yb2s 2y ago
I've become increasingly convinced that the idea of averaging is one of the biggest obstacles to understanding things... it contains the insidious trap of feeling/sounding "rigorous" and "quantitative" while making huge assumptions that are extremely inappropriate for most real world situations.
Once I started noticing this I can't stop seeing this almost everywhere- almost every news article, scientific paper, etc. will make clearly inappropriate inferences about a phenomenon based on the exact same mistake of confusing the average for a complete description of a distribution, or a more nuanced context.
Just a simple common example, is the popular myth that ancient people died of old age in their 30s, based on an "average life span of ~33 years" or such. In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality.
The above example is a case where thinking in terms of averages causes you to grossly misunderstand simple things, in a way that would be impossible even with basic common sense in a person that had never encountered the idea of math... yet it is a mistake you can reliably expect people in modern times to make.
- KK7NIL 2y agoI think you'd enjoy this video on different types of means and their applications: https://youtube.com/watch?v=V1_4nNm8a6w https://youtube.com/watch?v=V1_4nNm8a6w
- mcmoor 2y agoIt's just trying to assume normal distribution when it's not normal. Modern science rely so much on that distribution that i actually whether they have overestimated its ubiquity just because it's so damn convenient to use.
- Jweb_Guru 2y agoWhile it is true that not all distributions are normal, many distributions are approximately normal (or at least, normal in some sensible space that maps onto the actual collected data). IMO the amount of ink spilled on the idea that science is fundamentally flawed because distributions aren't always normal is probably too high (especially among non-statisticians), and frankly that's not where statistical analyses usually go wrong. A much larger problem (that has nothing to do with the ultimate shape of the distribution) is stuff like postselecting from a set of plausible models until you find one that finds significant results, and claiming that was what you intended to measure all along (this is why it's important to consider stuff like hyperpriors, much moreso than lack of normality).
- _yb2s 2y agoYes, this is the actual issue- assuming a normal distribution in cases that are not normally distributed. People also even leave off the standard deviation, and seem to be mentally thinking of the average as a point rather than considering even the “width” of the normal distribution- so the overemphasis of the average even causes people to misunderstand things that really are normally distributed.
- hinkley 2y agoAverages are very bad in bimodal distributions. And that includes issues of public policy, where going left sort of works, and going right sort of works, and going in the middle sucks for absolutely everyone.
- JumpCrisscross 2y ago> Averages are very bad in bimodal distributions They're bad with multimodal distributions, generally, as well as any random process governed by a distribution with no mean, e.g. Cauchy [1]. (Neuronal firing appears to be non-Gaussian, possibly lognormal [2], which does have a mean [3], but it isn't equal to the simple average.) [1] https://en.wikipedia.org/wiki/Cauchy_distribution#Properties https://en.wikipedia.org/wiki/Cauchy_distribution#Properties [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6633220/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6633220/ [3] https://en.wikipedia.org/wiki/Log-normal_distribution https://en.wikipedia.org/wiki/Log-normal_distribution
- bosma 2y ago> In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality. This isn't true: https://ourworldindata.org/its-not-just-about-child-mortality-life-expectancy-improved-at-all-ages https://ourworldindata.org/its-not-just-about-child-mortalit...
- Spooky23 2y agoIt mostly is. The biggest gains are in childhood. Aldo consider that you’re looking at figures for England and Wales, which isn’t necessarily representative. The largest contribution to improving life expectancy is measured by reductions in child mortality. The factors that drove those improvements (infection control, improved hygienic practices, food quality, regulation of food and drug purity, medicine) benefited everyone, but had the biggest impact on the old and young. In the 1850s, 8,000 infants died annually from adulterated milk alone. I think of my own journey. In 1824, 200 year ago Spooky23 would have died 20 years ago, a half blind cripple. 2024 Spooky23 is healthy with no back issues and god willing a few more decades.
- JumpCrisscross 2y ago> It mostly is. The biggest gains are in childhood. The life expectancy of a 60-year old going from 74.4 to 84, or a 70-year old from 79.1 to 85.9, is significant and meaningful. Not as much as a newborn's LE going from 41.6 to 81.1. But far from "pretty stable in the 70s-80s range for most of human history." Also, recent life-expectancy increases have come from adult morality reductions [1]. [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3000019/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3000019/
- _yb2s 2y agoHow are "74.4 to 84" and "79.1 to 85.9" "pretty far from" the "70s-80s range"? > adult morality reductions LOL
- llm_trw 2y ago
- llm_trw 2y ago>In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality. This is even wronger than what you critique. For every period in history that we have good data for people had a half-life - a period in which you'd expect half of all people to die: https://batintheattic.blogspot.com/2011/07/medieval-population-pyramid.html https://batintheattic.blogspot.com/2011/07/medieval-populati... For example in medieval Germany it looked something like: | Age | Half-life | | 0-10 | 10 | | 10-20 | 40 | | 20-40 | 20 | | 40-80 | 10 | It's called a population pyramid not a population column for a reason. The exact age varies by location, but even if we ignore everyone under 10, half of all people left would still die before they are 40.
- SkyBelow 2y agoIs there an issue with how the data is grouped? At 19.9, you have a 50% chance of living to 59.9. But less than a season later at 20.1, you now only have a 50% chance o living to 40.1. How can the former be right if the latter is right? The growth in life expectancy from surviving early childhood makes sense, but the decline in life expectancy for crossing 20 feels too sharp.
- JumpCrisscross 2y ago> if we ignore everyone under 10, half of all people left would still die before they are 40 Wouldn't it be 50 since the half-life is an interval?
- llm_trw 2y agoNo, the half life for age 10-20 is much higher than that for 20-40 and the exponential function is non-linear.
- _yb2s 2y agoWhat you said in no way conflicts with what I said. For example, if people have dangerous lives in a way that is unconnected to age they will tend to not live long, yet the modal life expectancy due to the additional mortality of actual old age can still be quite old. What I was saying is in reference to this study I read long ago, which suggests a modal life span of about 72 years of age in the paleolithic: https://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2007.00171.x https://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2007...
- aaron695 2y ago[dead]
- Mistletoe 2y agoI was at the Denver museum’s mummy exhibit and disturbed to see that they said the lady died at 30, a “normal age for death in those times”. You would think a museum should know better. https://aeon.co/ideas/think-everyone-died-young-in-ancient-societies-think-again https://aeon.co/ideas/think-everyone-died-young-in-ancient-s...
- _yb2s 2y agoGreat article that pretty well explains the situation with modern vs ancient life expectancy.
- orwin 2y agoHalf the people died before 40 in the middle ages, discounting pre-5 mortality. I would assume it is lower for female, as first pregnancies had a 10% mortality rate (this rate increased after the middle age, until germ theory), and following ones carried a 2-5% mortality rates.
- ddfs123 2y ago> In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- I am pretty sure this is wrong. East Asian cultures celebrate 60th birthday as becoming very elderly, and if you live to the 70s it's almost as if you achieved Buddhahood.
- JumpCrisscross 2y agoYour beef appears to be with simple averages, not averaging per se. The average for life expectancy is the mean of the Gompertz distribution [1]. Specifically, one that is "left skewed and with a flattened slope for ages under 50 years for men and 60 years for women," which proceeds to become "more right skewed and leptokurtic" as age increases [2]. So a simple average in the <55 domain would underestimate the mean while in the >55 domain it would overestimate it. Which is almost comically problematic when comparing ancient societies that had a median age below that level to modern ones above it. > the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history Not quite. 63 in 1900 to 83 in 2000 (in Sweden). Bigger differences when you go further back. [1] https://en.wikipedia.org/wiki/Gompertz_distribution https://en.wikipedia.org/wiki/Gompertz_distribution [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2652977/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2652977/ [3] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3000019/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3000019/ Figure 1
- _yb2s 2y agoWhat I was saying is in reference to this study, which suggests a modal life span of about 72 years of age in the paleolithic: https://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2007.00171.x https://onlinelibrary.wiley.com/doi/10.1111/j.1728-4457.2007... Life spans were abnormally short in medieval Europe, even compared to ancient humans.
- dalmo3 2y agoI came to the same realisation about a decade ago, after being a "science" enthusiast growing up. As you said once you see it you can't unsee it. Most of science is just a scam. The exceptions are those fields backed up by real world engineering. All of social and most of biological sciences are worse than useless, they are outright dangerous.
- hifromwork 2y ago>Most of science is just a scam Most of the scientists are not scammers. So if you believe that all science disciplines other than engineering are wrong[1], you should use another word that doesn't suggest researcher malice. [1]Which is a very strong statement, because you claim to be an expert in all science disciplines at once.
- zmgsabst 2y agoI think there’s clear evidence that a sizable portion of “science” is indeed a scam for prestige and funding. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
- tourmalinetaco 2y agoAt minimum the social and even a fair bit of medical science are scams, as they rarely have enough evidence to definitively draw claims due to a mixture of flawed methodology, small sample sizes, and lack of additional studies to reinforce their claims. See for instance the claim every man makes at some point, that “going braless makes breasts perkier”. Small sample size that isn’t indicative of all breast sizes/shapes, few if any studies supporting their findings. At least with engineering experimentation is far easier, can be extrapolated, and can even be simulated reliably. And no, one does not claim to be “an expert in all disciplines at once” by pointing out the objective fact that we lack concrete data in a lot of fields.
- _yb2s 2y agoI think you are overreaching with that. Science at its core is curious and open minded problem solving combined with an anti-authoritarian skepticism- you accept things once you can understand and confirm them for yourself. Ideally, it is also Bayesian- e.g. considering all of the evidence, even weak evidence like personal anecdotes, while correctly keeping track of how strong or weak each bit of evidence really is. You have the courage to trust what you understand deeply even if other powerful authorities disagree- and take the responsibility to make sure you do actually understand deeply (the conspiracy theory crowd is missing the second bit). As a practicing academic scientist, I feel like the vast majority of my colleagues with research focused academic careers also see it this way. Nearly everything called "science" in popular culture (including school science classes, pop science books, etc.) is actually a sort of dogmatic religion that idolizes science. One of the weirdest things for me, is coming on here and when I share the type of creative thinking that actually leads scientists to new hypotheses, I am insulted and accused of being an "idiot" or "anti-science" by mostly career programmers for whom science is a religion, and not a creative process. When they talk to a scientist, they see someone that doesn't align with their fixed dogmatic views, and label it as basically the opposite of what they are actually encountering- real science looks like "pseudoscience" to them. These people would even attack someone like Isaac Newton as an idiot for daring to discuss something new that seems "weird" and idolize, e.g. a medical doctor repeating some old official stance from an institution that has been thoroughly disproven by new research.
- ants_everywhere 2y agoYeah it's a common mistake, but this is like intro to stats stuff. It's not some big secret that if you summarize a distribution with a single number then you've lost information. > I've become increasingly convinced that the idea of averaging is one of the biggest obstacles to understanding things. I'd counter that it's easily one of the biggest assistants in understanding things. The Central Limit Theorem in particular has been enormously influential. Without averaging statistical mechanics and thermodynamics would have been impossible and with them would go the industrial revolution. What you're noticing is one kind of mistake caused by lack of literacy in science. There are many many more similar mistakes. The solution isn't less literacy.
- ithkuil 2y ago> The solution isn't less literacy. This. It's interesting how a common instinct to seeing problems is way too often to "destroy and rebuild" instead of "correct and improve". "Institutions have failed us?" A) "Let's burn them down" vs B) "Let's have better institutions, keep what's good, fix what's bad" I get it that sometimes, rarely, it's impossible to improve something because it's soo rotten and entrenched that you literally have to destroy and rebuild from scratch, but it's almost never the case.
- _yb2s 2y ago> this is like intro to stats stuff Amazing how the replies to my initial comment range from angrily calling me an idiot for saying something so obviously and egregiously wrong, to claiming it is so trivial and obvious that I should not have needed to mention it. > The solution isn't less literacy. Indeed, I can see how what I wrote could be taken that way, I also don't think so. The problem is combining an inadequate amount of literacy, with an almost religious belief in its absolute correctness, e.g. scientism.
- ants_everywhere 2y agoTo clarify, I'm not saying that it's trivial or obvious, just that it's fundamental. Just like derivatives aren't trivial or obvious but everyone who has taken Intro to Calculus knows about them. > "e.g. scientism" Isn't this just an anti-science term adopted by religious people and postmodernists?
- domofutu 2y agoAverages can definitely oversimplify things, especially in neuroscience where outliers often tell the real story. Taleb touches on this in Antifragile—focusing too much on the average can make us miss what’s happening at the edges, where the most interesting things are. Instead of leaning on averages, we might get more insight by paying attention to the extremes, where the real nuances are hiding.
- _yb2s 2y agoYes, I like Taleb's suggestion of considering a "fat tailed" Cauchy distribution- as to not assume extreme outliers can't exist
- _yb2s 2y ago@ flagged aaron695: > The brutal reality is you don't have the IQ to understand averages or statistics like most people. > Most of our ancestors lucky enough to made it to 45 years old in human history did not make it to 70. > Using mode is misleading with "age", and you quickly showed you didn't understand 'the con' when you accidently tried to apply it. > This is a political, the "past was wonderful fantasy" which is anti-science. It's used by the Woke for instance." In this case I am using an equally "wrong" model on purpose while being well aware of its limitations, just to make a specific point. It highlights a point where people are doing exactly what you are accusing me of- romanticizing the present, and not understanding the reality of how it actually differs from the past. E.g. what are the actual reasons people had shorter life spans in the past, and what their lives were actually like. One should not forget the limitations of such a simple model, which was basically my point in the first place. I am fascinated by "evolutionary medicine" and using such ideas as a hypothesis generator to figure out ways to treat "modern diseases of civilization." I am in no way romanticizing the past, but trying to understand the specifics, to better figure out how to develop more effective modern day medical treatments, not to return to the past. In truth I despise political thinking altogether, and like to look at mechanisms and biological details. Your post smacks of "scientism" which is incompatible with actual practice of science. The very idea that a certain line of thinking or theorizing is "anti science" or should be taboo for political reasons is itself incompatible with creative open minded problem solving. I can see it was a mistake to use this specific example for discussing the problem with averages- ironically because it is so accurate. Since so many people on here hold the exact misunderstanding I was criticizing, they are getting angry and insulting me instead of my intent, which was to explain a phenomenon and have this click as a simple example of it. A less charged example, where people don't have strong opinions already would have been better.
- throw_pm23 2y agoI've heard this argument a million times, but I am very skeptical: where would the reliable data on infant mortality in ancient times come from? (so that it would allow us to compute precise values of average lifespan). All we have from those times are a few bone samples and a few anecdotes preserved in fragments. I'm not saying anything for or against the ~33 years claim, just that I doubt that it comes from a precise estimate of expected lifespan at birth.
- achierius 2y ago> All we have from those times are a few bone samples and a few anecdotes preserved in fragments. This is the bit that you're missing: antiquity is generally considered to extend all the way into the 500-600s AD, with the Roman empire dominating its center and 'late antiquity' covering the whole early period of the Byzantine empire. This means that even just in Europe we have extensive documents (e.g. in the Christian era, baptism and burial records), as well as a wealth of burial evidence from which to extrapolate lifespans. If you look into the sources for Rome you'll find Egypt in particular come up a lot, which isn't an accident: the dry climate and extensive use of papyrus means that the region preserved far more records (and bodies) than any other part of the empire.
- deleted 2y ago[deleted]
- nosianu 2y agoIn a Gresham College video lecture about microbial disease (if only I remembered the title...) the lecturing professor briefly mentions that in Victorian Britain life expectancy was actually "higher than today" when you exclude children and women (who had a great risk of death during pregnancy and birth). https://www.gresham.ac.uk/watch-now/return-microbes-how-infections-are-once-more-taking-over https://www.gresham.ac.uk/watch-now/return-microbes-how-infe... The video has a text with the words. Look for > So, your life expectancy at five, in England, as a male, in 1870 was slightly longer than it is now, which is an extraordinary statistic, slightly shorter then if you were a female. 1870s Britain is not "ancient", but as far as medicine is concerned much closer to that than to modern times. What medicine there was wasn't even available to most people. They were pretty good with "physical" things like broken bones by then, but anything concerning microbes probably wasn't much better than what was done a thousand years earlier. Obviously, as you point out, we don't have statistics from ancient times, only some hints, but that is the same for people making the claim about the short average life spans (without the little detail about infant and women mortality). There also is the question of how meaningful it is to average over vastly different sections of the population (what you did and how you lived and what you ate, access to good water, hygiene), or times of famine or war. Some related discussion, reddit, but the heavily moderated AskHistorians subreddit: https://www.reddit.com/r/AskHistorians/comments/12o4py/what_was_the_average_life_expectancy_of_a_native/ https://www.reddit.com/r/AskHistorians/comments/12o4py/what_... There we see the lower life expectancy of early agricultural societies vs. hunter-gatherers (ignoring their high number of deaths due to violence). I think all of this also serves to illustrate OPs point about the problem of averages :-) There's just too much important information that is lost.
- deleted 2y ago[deleted]
- bell-cot 2y ago> In reality the modal life expectancy of adults (most common age of death other than 0) has been pretty stable in the 70s-80s range for most of human history- the low average was almost entirely due to infant mortality. Until the last few-ish generations, pregnancy and childbirth have been leading causes of death for women in those in-between decades of their lives. (And obviously War and Famine, too, for both genders.)