7 ms·
> 30,000 times more initial energy than any recorded events As a statistician: this is why heavy-tailed modeling matters. Schools teach the finite-variance ver
by fritzo 4y ago
> 30,000 times more initial energy than any recorded events
As a statistician: this is why heavy-tailed modeling matters. Schools teach the finite-variance version of the Central Limit Theorem and students believe they live in a normal world. The real Central Limit Theorem [1] predicts large rare events following Lévy α-stable distributions.
[1] https://en.wikipedia.org/wiki/Stable_distribution#A_generalized_central_limit_theorem https://en.wikipedia.org/wiki/Stable_distribution#A_generali...
- oneoff786 4y agoThe central limit theorem applies to sample means, not individual events. I’m not sure it has any relevance to predicting the probability of a single large outlier tsunami
- nextos 4y agoThis is so true. Heavy-tailed distributions and extreme value theory [1,2] should be taught more widely. The normal distribution is just a very particular case of the generalized CLT. For example, some drug effects are heavy-tailed, but safety trials are often ignoring this. [1] https://adamwierman.com/wp-content/uploads/2020/09/2013-SIGMETRICS-heavytails.pdf https://adamwierman.com/wp-content/uploads/2020/09/2013-SIGM... [2] https://adamwierman.com/wp-content/uploads/2021/05/book-05-11.pdf https://adamwierman.com/wp-content/uploads/2021/05/book-05-1...
- buryat 4y ago> some drug effects are heavy-tailed where can i read about this? curious
- cosmojg 4y agoGot any sources which discuss the heavy-tailed effects of drugs and how they're ignored by safety trials? I'd love to read more!
- nextos 4y agoI don't have references with me right now, but there are quite a few reputable studies that have found effects to be log-normal. Log-normal is a quite nasty heavy-tailed distribution as it's easy to take it for normal if you have small sample sizes. This has tricked the safety calculations of a few trials. I don't think it's too worrying or generalized, just some area to improve.
- jacquesm 4y agoIt would save quite a few ocean going ships if this was more appreciated. Those once-in-a-thousand-year-rogue-waves are much more frequent than once in a thousand years. https://www.theguardian.com/science/2006/jul/26/uknews https://www.theguardian.com/science/2006/jul/26/uknews But we're still designing ships as though they only happen once every thousand years because the cargo and the ship are insured and the crews are considered expendable.
- echelon 4y ago> Research at Imperial College, London, shows that far from being rare events, rogue waves can emerge frequently, and may be responsible for some of the 200 supertankers and container ships longer than 200m that have sunk in poor weather conditions in the past two decades. The loss of ten ships a year seems incredible to me. Surely "weather", if it were to blame, could be avoided. This amount of loss is steep. Is it being investigated? We should have ocean swell data beamed to satellite from every ship that capsizes.
- Someone 4y agohttps://www.statista.com/statistics/1326758/number-of-oil-tankers-worldwide/ https://www.statista.com/statistics/1326758/number-of-oil-ta...: There were 8,883 oil tankers worldwide in 2021. If those are all ocean-going, ten a year is about 1.1%. Depending on how expensive it would be to prevent this, that may not be bad for ship owners, but it is very bad for those seeking a career in working on those ships.
- calderknight 4y ago1.1% of what?
- rhn_mk1 4y agoOf ocean-going oil tankers.
- Idiot_in_Vain 4y agoHeavy-tailed modeling doesn't reallly matter. There are very few enterprices where preparing for extremely rare events is worth the effort - nuclear power plants, weapons of mass destruction, bio labs working with dangerous organisms, big chemical plants, maybe AI research and big particle accelerators.
- baq 4y agothe most important of all - your 401k
- Idiot_in_Vain 4y agoWhat does a 401k have to do with extremely rare events? Preparing for stuff like a nuclear war would mean buying land in South America, Africa or Australia, building an underground bunker, stacking on water, food, medicine and weapons, etc.
- red-iron-pine 4y agoso you buy an index fund or ETF. capture some of the positives if/when they happen, and help mitigate the negatives
- baq 4y agosome would argue performance of the 60/40 portfolio last year was an event from a tail of a very non-normal distribution
- vba616 4y agoIf you tuned out of current events, financial and otherwise, five years ago - in early 2018 - and your 401k was entirely on autopilot in an S&P 500 fund, it looks like your average annual return would've been about 11.5%, annual US inflation was 3.7%, and return after inflation 7.5%. Per year. You can't reasonably expect better. And practically speaking, the tax avoidance means that's considerably better than 11% in a non-retirement account, but it was too much effort to do the math. Despite all the pestilence, war, earthquakes, and stuff like that, doing nothing was the smart move again, with $1 or $1 million. I don't think it makes sense to prepare for a global collapse of civilization because in the event whatever you will reap won't make hardly any difference in how much things suck. You can't eat money, much less use it as a shelter from a global rain of molten rock. So on principle, regardless of the probability of Armageddon, I don't think one should give up much to try to hedge the end of the world.
- Buttons840 4y agoI'm no expert on stats, this discussion is a bit over my head, but it reminds me that not all distributions have a mean. A Cauchy distribution has no mean; as you average successive samples, it never settles down to a consistent average. I was surprised when I first encountered Cauchy distributions and up until then had considered that all distributions would have a mean. I suppose it is one of those "heavy-tailed" distributions you're talking about?
- SnowHill9902 4y agoIt’s an example but long tailed distributions can have a mean, and short tailed distributions can have no mean. It’s not indicative.
- hanche 4y ago> short tailed distributions can have no mean That’s a surprise to me. Now I’m curious: With what definition of “short tailed” can that be true?
- oneoff786 4y agoThere’s no need to get mathy tbh. Distributions are not real, and the ones we create only describe parts of what actually happens. This is obvious and always has been. How much water does your faucet leak per day? That might be a good case for a normal distribution. But today I smack it with a hammer and it can’t stop flowing. We now have a ton of water far more than should ever appear on that original normal distribution. Is that because the model is wrong? No. It’s because the model is not designed for these circumstances. We can’t build a model that estimates flow distribution including the possibility that I smack it with a hammer. That doesn’t mean we don’t believe it won’t ever happen or even that we think it is unlikely to happen. It’s just a weird edge case. A lot of people (taleb) imply statisticians are idiots because they insist the distributions are comprehensive. But statisticians don’t insist this. Black swan, fat tails, and all that are just terms for things outside modeled conditions. And yeah. They happen.
- SnowHill9902 4y agoIf you work in finance and your competitors attended those universities you have an edge.