13 ms·

Nassim Taleb: We should retire the notion of standard deviation

248 points by pyduan 13y ago

spikels 13y ago
You gotta love the acronyms: STD versus MAD! Taleb is definitely mad but his use of the MAD acronym (mean absolute deviation) is actually correct. However the STD acronym (all caps) refers to "sexually transmitted disease" and not generally used for "standard deviation". Most people use SD, Stdev, StDev or sigma. Once again his ability to coin new terminology outstrips his ability to form coherent ideas that are anything more than trivial (eg. we have known about fat tails in stock returns for 50+ years). Like George Soros[1], Taleb's success says more about the state of the world of finance than their contributions to our knowledge. [1]-See his book "The Alchemy of Finance"
- Fomite 13y ago
  STD is also an obsolete term. Epidemiology, Medicine etc. most often now refer to them as STIs, or Sexually Transmitted Infections. The reasoning for this is that many sexually transmitted infections can be acquired, passed on to others, etc. without causing any clinical symptoms. See: HPV, among others.
truthteller 13y ago
he's really lost the plot. :(
zeidrich 13y ago
It's not that we should retire the notion of standard deviation. It's more that we should understand the tools that we are using and use the appropriate tool for the job.
- deleted 13y ago
  [deleted]
dschiptsov 13y ago
Why, it is pretty good in describing probability distributions. What we should retire are idiots, who assume that it predicts an outcome of the next event.
- regularfry 13y ago
  Everyone is an idiot some of the time.
valtron 13y ago
He makes a good point about infinite MAD vs. STD.
scythe 13y ago
While the mean deviation as presented is slightly nicer than sigma for intuitive purposes, it isn't as appropriate (iirc) for statistical tests on normal distributions and t-distributions. More importantly, it doesn't fix the real problem, which is that the mean and standard deviation don't tell you everything you need to know about a data set, but often people like to pretend they do. It's not rare to read a paper in the soft sciences which might have been improved if the authors had reported the skewness, kurtosis, or similar data which could shed light on the phenomenon they're investigating. These latter statistics can reveal, for instance, a bimodal distribution, which could indicate a heterogeneous population of responders and non-responders to a drug, and that's just one example. I'm not a statistician, so some of this might be a bit off.
- drblast 13y ago
  Whenever you try to describe a large data set with a single number, you lose a lot of information, like you said. Having more measurements helps, but I think the larger point is that we don't have to do this anymore, we could publish the entire data set instead. Without computers, this would be a waste of paper, but transmitting the data electronically is cheap. So why argue over the measurements? Publish the data and my software can give me any measurement I'm interested in.
cwyers 13y ago
"In fact, whenever people make decisions after being supplied with the standard deviation number, they act as if it were the expected mean deviation." Boy, is that statement useless without any kind of context, example or citation.
- icebraining 13y ago
  The next two paragraphs have examples with some context.
  - cwyers 13y ago
    Like this? "But it is not just journalists who fall for the mistake: I recall seeing official documents from the department of commerce and the Federal Reserve partaking of the conflation, even regulators in statements on market volatility. What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life." It doesn't tell us what happened, it just asserts that it did in certain contexts. It doesn't cite the paper he presumably wrote with Goldstein (which Goldstein?) about it. I feel like I'm getting a summary of an abstract with all the citations missing.
    - lstamour 13y ago
      > which Goldstein? From ClementM above, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480
      - cwyers 13y ago
        Thank you. From that paper: "We first posed this question to 97 portfolio managers, assistant portfolio managers, and analysts employed by investment management companies who were taking part in a professional seminar. The second group of participants comprised 13 Ivy League graduate students preparing for a career in financial engineering. The third group consisted of 16 investment professionals working for a major bank." From the article: "What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life." I get that "data scientist" is a really broad term at this point, but I don't think it's a very good description of the people quizzed in this paper, if this is the paper he was indeed referring to.
deleted 13y ago
[deleted]
ClementM 13y ago
This article is based on paper Taleb published in 2007. If you want to test yourself, submit yourself to experiment in page 3: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480
- drblast 13y ago
  Thanks! Always nice to read the full research.
- CamperBob2 13y ago
  A stock (or a fund) has an average return of 0%. It moves on average 1% a day in absolute value; the average up move is 1% and the average down move is 1%. How does that yield an average return of 0%?
  - MaysonL 13y ago
    It goes up a little more often than it goes down?
  - baking 13y ago
    The usual way to do this is to take the natural log, so an up 1% day followed by a down 1% day (or vice versa) will always net out to a 0% change.
- baking 13y ago
  Yes, start by writing a confusing question. One that starts talking about average up moves and average down moves and then switches to asking about the standard deviation of moves. Then publish a paper showing that people were confused by your question. Now you have "research" to back up your claim that everyone is confused by "mean deviation" and "standard deviation".
n00b101 13y ago
Taleb has a good point about people mistakenly interpreting standard deviation (sigma) as Mean Absolute Deviation (MAD). I like that he gives some conversions (sigma ~= 1.25 * MAD, for Normal distribution). I think it's rather silly to talk about "retiring" standard deviation, but we can't blame Taleb - the publication itself posed the question "2014: What Scientific Idea is Ready for Retirement?" to various scientific personalities. What Taleb failed to mention is that, once properly understood, standard deviation has distribution interpretations that can be much more useful than MAD. For example, if the data is approximately normally distributed, then there is approximately a 99.99% probability that the next data observation will be <= 4 * sigma. Not everything is approximately normally distributed, but a lot of phenomena ARE normally distributed. It's a well known fact that the phenomena which Taleb is most interested in (namely, financial return time-series) are not normally distributed. But I would like to know how Taleb proposes to "retire" volatility (sigma) from financial theory and replace it with MAD? Standard deviation is so central in finance that even the prices of some financial instruments (options) are quoted in terms of standard deviation (e.g. "That put option is currently selling at 30% vol"). How do we rewrite Black-Scholes option pricing theory and Markowitz portfolio theory in terms of MAD and remove all the sigmas everywhere? Surely Taleb has already written that paper for us so that we can retire standard deviation?
- regularfry 13y ago
  I think his point is that Black-Scholes et al are holed beneath the waterline precisely because they involve standard deviations. In his world, you're better off being unable to price an option than you would be with Black-Scholes. Your example of "That put option is currently selling at 30% vol" is actually an example of why the system is so completely broken: if volatility as standard deviation was valid, all options against the same underlying instrument would have the same implied volatility. The volatility smile shouldn't exist. This wouldn't matter if the down-side wasn't so crippling. I don't think Taleb has to be the one to propose a replacement for portfolio theory, and I think criticism of him for not doing so is pointless. You don't need to have a spare tire handy to point out that your neighbour's car has a flat, and you don't have to run an airline to tell people not to get on a plane with the engines visibly on fire.
bluecalm 13y ago
So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you to measure the "average daily variations" for the temperature of your town (or for the stock price of a company, or the blood pressure of your uncle) over the past five days. The five changes are: (-23, 7, -3, 20, -1). How do you do it? Ok... if I am to calculate the average I am calculating the average if I need to know standard deviation I calculate standard deviation... >> It corresponds to "real life" much better than the first—and to reality. What the flying fuck. What "real life" ? Standard deviation tells you how volatile measurements are not what mean deviation is. Those are both very real life things just not the same thing. >>It is all due to a historical accident: in 1893, the great Karl Pearson introduced the term "standard deviation" for what had been known as "root mean square error". The confusion started then: people thought it meant mean deviation. I don't know how one can read it and not think: "is this guy high or just stupid?". >>. The confusion started then: people thought it meant mean deviation. I am yet to see anybody who thinks that standard deviation is mean deviation. It's Taleb though. Baseless assertions insulting groups of people are his craft. >>What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life. One example please ? I can give hundreds when std dev is useful and mean deviation isn't. Anything when you decide what % of yoru bankroll to bet on perceived edge for example. Ok so he asserted that people should just use mean deviation instead of mean of squares. Guess what though, taking the squares have a purpose: it penalizes big deviations so two situations which have the same mean deviation but one is more stable have different standard deviations. THis information is useful for many things: risk estimation or calculating sample size needed for required confidence (if you need more experiments, how careful should you be with conclusions and predictions etc). He didn't mention how are we going to achieve those with his proposal. Meanwhile he managed to throw insults towards various groups without giving one single example of misuse he describes. This is not the first time he writes something this way. His whole recent book is like that. It's anti-intellectual bullshit with many words and zero points. He doesn't give any arguments, he throws a lot of insults, he misues words and makes up redundant terms which he then struggles to define. The guy is a vile idiot of the worst kind: ignorant and aggressive. Him gaining so much following by spewing nonsense like this article is for sure fascinating but there is no place for him in any serious debate.
RivieraKid 13y ago
I was just wondering about a very related problem. I do 5 measurenments of some random variable (let's say execution time) and average them. How should I report the variability of that average? State the sample size and standard deviation?
- darkhorn 13y ago
  90% of the data fall between this point and this point. 50% of the data fall between this point and this point. etc. MAD is more accurate than Standard Deviation to me, if you ask me.
- perlgeek 13y ago
  If it were 10 or more measurements, I'd use average, standard deviation and sample size if we can expect that variable to be normally distributed. I don't know how to make meaningful statistics with fewer data points.
- bayesianhorse 13y ago
  Citing 10 measurements in a table or plot is probably easier than explaining how and why you are taking mean and standard deviation (or anything else for that matter...)
deleted 13y ago
[deleted]
JASchilz 13y ago
The central limit theorem shows us that unimodal data with lots of independent sources of error tends towards a normal distribution. That description is a good first-pass, descriptive model for lots and lots of contexts, and standard deviation speaks well to normally distributed data. Squaring error isn't just a convenient way to remove sign, it's driven by a lot of data-sets' conformance to the central limit theorem.
- dj-wonk 13y ago
  Thank you. I don't think it is intellectually honest for Taleb to omit this fact.
lambdasquirrel 13y ago
I think we'd be better off if we recognized that there are statistical distributions in the world besides the plain old Gaussian. For example, wealth does not follow a Gaussian, so why the heck do we throw around ideas like "above average wealth"? Is MAD any better? Definitely. But I'd like to see a visual demonstration of how well it models exponential-based distributions. How well does it describe their "shape", the skew of the tail?
- chilldream 13y ago
  In that specific case, I'd submit that the word "average" doesn't belong in a conversation attempting any level of rigor. It only encourages the confusion between median and mean.
  - vorg 13y ago
    Medians and means are interrelated. Averages are best thought of as being on a scale from meanlike to medianlike: a median requires a mean of the middle 2 values when there's an even number of them, and a mean can have outliers on each side to be eliminated before calculation. The word average abstracts away this detail. (I guess there'd be another scale from meanlike to medianlike absolute deviation, but that's another story.)
programminggeek 13y ago
I think because it's called "standard deviation" that it sounds like the thing to use or look for. It sounds more correct because of the word standard. I feel like it is the same kind of failing due to human perception of language that programmers have with the idea of exceptions and errors, especially the phrase "exceptions should only be used for exceptional behaviors". That's a cool phrase, but people latch on to it because of the word exception sounding like something extremely rare and out of the ordinary whereas we see errors as common, but they are in fact the same thing. Broke is broke, it doesn't matter what you call it, but thousands of programmers think differently because of the name we gave it. We are human and language absolutely plays a role in our perception of things.
- klodolph 13y ago
  > I think because it's called "standard deviation" that it sounds like the thing to use or look for. Yes! Because it's an awesome trick and lets you do good estimates on napkins. The other day I was buying lunch at a food cart and thought about how much change the food carts had to carry, as a function of how many customers they have, under the assumption that they want to be able to provide correct change to 99% of their customers. Let's say that the average amount of change a customer needs is $5, and a 99-th percentile customer needs $15 in change. If we pretend that the distribution is approximately Gaussian we can calculate that 1,000 food carts with 1 customer each would need $15,000 in change, but 1 food cart with 1,000 customers would need $5 x 1,000 + ($15 - $5) * sqrt(1,000) ≈ $5,320. That's math you can do in your head without a calculator (being a programmer, 1,000 ≈ 2^10 so sqrt(1,000) ≈ 2^5). The standard deviation and assumptions of normality are so useful because of the central limit theorem. That is, if you have many iid variables which have finite standard deviation the sum will converge to a Gaussian distribution as the number of variables increases. Then you say "Well, the standard deviation weighs the tail too heavily" and the response is "well use higher order moments then, that's what they're they're for".
  - skybrian 13y ago
    It's a neat math trick, but it seems more accurate to say this lets you calculate bad estimates on the back of a napkin. Unless you really think food carts carry $5000 in change. The quantitative work I do has to do with measuring latency, where the minimum, median, 90%, and 99% values are more meaningful than the mean or standard deviation. Programs typically have a best-case scenario (everything cached) and a long one-sided tail.
deleted 13y ago
[deleted]
roywei 13y ago
four-day returns of stock x: (-.3, .3, -.3, .3) -> MAD = 0; four-day returns of stock y: (-.5, .5, -.5, .5) -> MAD = 0.
- andrewcooke 13y ago
  nope. look at what the "A" in MAD stands for.
- michaelhoffman 13y ago
  No, it's the median absolute deviation. Absolute deviation is the absolute difference between an average and a data point. So (-.3, .3, -.3, .3) -> MAD = 0.3; (-.5, .5, -.5, .5) -> MAD = 0.5.
- jameshart 13y ago
  The A in MAD stands for 'absolute', so no, the MAD for those two stocks is .3 and .5 respectively - versus standard deviations of 0.35 and 0.58.
- spikels 13y ago
  You have calculated the mean. This is only the first step in calculating the MAD (mean absolute deviation). You then need to take the absolute value of the difference (deviation) of each datapoint from this mean - for your x you get (.3, .3, .3, .3) and for your y (.5, .5, .5, .5). Finally you take the mean of these absolute deviations to get MAD(x)=0.3 and MAD(x)=0.5.
cheald 13y ago
I really tried to get through "The Black Swan" and Taleb's writing struck me as so pretentious and self-involved that it made it impossible for me to finish. He strikes me as someone who is so desperate to be important and recognized that an assertion like this doesn't really surprise me.
- andrewcooke 13y ago
  fascinating.
- Alex_MJ 13y ago
  His facebook doesn't inspire a lot of confidence. It's full of vague availability-heuristic-targeting generalities. "Virtue is when the income you wish to show the tax agency equals what you wish to show your neighbor" "The problem is that academics really think that nonacademics find them more intelligent than themselves", etc I've read black swan as well, and there were parts I didn't quite grok at the time (You cannot predict a black swan! etc) My take is that he's a Malcolm Gladwell with numbers and therefore less easily takedownable but has the same model of "hold prestigious academic position, appear wise, publish book with simple principle, be easily referenceable by people who want to sound well-read" except he has lots of math and charts to point at lest anyone call him out on it.
- calroc 13y ago
  I started reading that book and I thought he was a genius, but then, right abut the point where he starts bagging on the Uncertainty Principle, I realized that he's actually kind of an idiot. The book does make a good point though.
  - cheald 13y ago
    Maybe I'm the idiot, but I found the book's point to be pretty trivial. "Sometimes unexpected things happen, and people are bad at expecting the unexpected" seems pretty damn trivial to me. Maybe it's because I think like a programmer rather than a finance guy, but a large portion of what I do on a daily basis is about mitigating risk from the unknown. A programmer who only guards against known risks is going to get his ass handed to him sooner rather than later (these are the sorts of people who use blacklists and regexes to sanitize SQL). A finance guy who only guards against known risks is just playing the averages. My experience with traders is that they tend to be pretty abstracted from reality and like to impose rules and patterns where there are none. I can see why those sorts of people would find Taleb's work groundbreaking. Taleb's undoubtedly intelligent, but I feel like he woke up one day, decided that he wanted to be a philosopher who brings wisdom to the masses, and built his temple on a mind-searingly obvious principle, which he now proclaims to be his great gift to humanity, for which he should be praised, hallowed be his name. The impression I got off of him was that he considers himself a prophet, imparting a word to the masses that the rest of us are just too stupid to recognize. Gag me.
Homunculiheaded 13y ago
I sometimes think that progress in the 21st century will be summed up as: "The realization that the normal distribution is not the only way to model data". Taleb's favorite topic is the "black swan event" which is something that the normal distribution, and the idea of standard deviation, don't model that well. In a normal distribution very extreme events should only happen once in the lifetime of several universes. Of course assuming variation inline with a Gaussian process is at the heart of how the Black-Sholes model calculates risk/volatility/etc. Benoit Mandelbrot argued that financial markets follow a distribution much more similar to the Cauchy distribution (specifically the Levy distribution) rather than a Gaussian. The problem of course is that the Cauchy distribution is pathological in that it doesn't have a mean or variance, you can calculate similar properties for it (location and scale), but it doesn't obey the central limit theorem so in practice it can be very strange to work with. The normal distribution is fantastic in that it does appear frequently in nature, is very well behaved, and has been extensively studied. However a great amount of future progress is going to come from wrestling with more challenging distributions, and paying more attention to when assumptions of normality need to be questioned. Of course one of the challenges of this is that the normal distribution is baked into a very large number of our existing statistical tools.
- jerf 13y ago
  This is actually what I expected to read: "The standard deviation is useful because with the average and the standard deviation, one can fully characterize a normal distribution. However, the standard deviation is less useful a statistical summary the farther away from 'normal' you get, and in reality, there is no such thing as a normal distribution, as a true normal distribution is defined on the entire real number line from negative infinity to positive infinity. Reality always provides some bound, and it's often quite distorted from Guassian. For instance, a 'normal' distribution averaging 2 with a standard deviation of 1.4, bounded by 0, is quite non-Gaussian in many important ways! (Not least of which is that you're going to have to do something to replace the missing probability...) "People rarely check how closely their data conform to the standard distribution; indeed, many people blindly apply the standard deviation to their data regardless of its distribution! The resulting number is often more obfuscatory than helpful, to the extent that it crowded out more useful summaries. "It's a useful metric when treated carefully, but it is rare to encounter it treated carefully. Science courses would be well-served to stop teaching it in favor of a stronger emphasis on multiple distributions. (Multiple distributions are usually touched upon, but implicitly our curricula overfavor the Gaussian distribution and end up accidentally implicitly convincing students its the only one.)" But that's just me.
Glyptodon 13y ago
All I know is this reminds me a lot of high school where we had to always compute std dev in problems, homework, and sometimes labs, but nobody really ever explained how to interpret it. It was always like "This is std dev. This is how you compute it. Make sure you put it your tables and report." Eventually someone (or something) did explain it, but once I understood it, it became clear that it wasn't always a sensible thing to be asked to calculate but was instead just an instinctive requirement.
afterburner 13y ago
I've found MAD a potentially useful measure for monitoring whether something gets out of whack; when using STD I needed to modify it to give less weighting to outliers.
randomsample2 13y ago
Standard deviation and mean absolute deviation are both useful, but I think it's silly to suggest that we all adopt exactly one measure of variability to summarize data sets. When in doubt, make a fucking histogram.
- michaelhoffman 13y ago
  Histograms can be misleading too, especially when the breaks are not set well.
  - randomsample2 13y ago
    Anything can be misleading, but in most cases a histogram conveys more information than a single number ever could.
    - michaelhoffman 13y ago
      Absolutely. They just aren't a panacea.
etanazir 13y ago
The minimum uncertainty wave equation is ~ e^(-x^2) ergo the standard measure is in terms of x^2. QED.
vzhang 13y ago
I'm seriously questioning some people's reading comprehension - he NEVER said STD is not useful! He's only saying the name "Standard Deviation" is badly chosen.
- bluecalm 13y ago
  Read the blog post carefully: > it is time to retire it from common use and replace it with the more effective one of mean deviation > Standard deviation, STD, should be left to mathematicians, physicists and mathematical statisticians deriving limit theorems >There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good He is saying it's not useful for real world things and people are just confused. What you wrote is reasonable view. What Taleb writes isn't.
puranjay 13y ago
NNT is my intellectual superhero but the amount of hate he gets is tremendous. Please understand that NNT's biggest issues are not so much with the way statistical models are applied to economics and finance, but how social scientists sometimes feel compelled to apply them to social fields as well, which is plain unscientific, dumb, and mostly disastrous. So when you bear down on his arguments, please keep this context in mind.
- triplesec 13y ago
  You're making too many gross generalisations here, as does Mr Taleb. He's right that in all fields there are statistically illiterate people making huge errors, and academic theorists have been making such - and more complete - criticisms for a while now. So, there's a real issue. However, there are a good number of competent social scientists too, beyond the mediocre morass, who understand just what variable construction and limitations to statistical operations are.
- Fomite 13y ago
  Among other things, I've seen way more appalling applications of statistics in finance and economics than I have in social science. Also, the assertion in your post that the misapplication of statistical models in social science is "disastrous" but somehow giving finance a pass? You've got to be kidding me.
  - jessaustin 13y ago
    NNT, at least, doesn't give finance a pass. He excoriates them at every opportunity.
    - Fomite 13y ago
      That's why I was so puzzled - NNT rather eagerly takes finance to task.
dxbydt 13y ago
The notion of area has confused hordes of scientists; it is time to retire it from common use and replace it with the more effective one of circumference. Area should be left to mathematicians, topologists and developers selling real estate. There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good. Say someone just asked you to measure the area of a circle with radius pi. The area is exactly 31. But how do you do it? scala> math.round(math.Pi * math.Pi * math.Pi).toInt res1: Int = 31 Do you pack the circle with n people, count them up and verify n == 31 ? Or do you pour a red liquid into the circle and fill it up, then drain it and measure the amount of red ? For there are serious differences between the two methods. If instead, you were asked to measure the circumference of a circle with radius pi. scala> math.round(2 * math.Pi * math.Pi).toInt res2: Int = 20 You just ask an able-bodied man, perhaps an unemployed migrant, to walk around this circle while another man, an upstanding Stanford sophomore, starts walking from Stanford to meet his maker, I mean VC, well its the same thing... So by the time the migrant finishes walking around the circle, our upstanding Stanford entrepreneur is greeting the VC on the tarmac of the San Francisco International Airport. This leads one to rightfully believe that the circumference of the circle of radius pi is exactly the distance from Stanford to the SF Airport ie. 20 miles. It corresponds to "real life" much better than the first—and to reality. In fact, whenever people make decisions after being supplied with the area, they act as if it were the distance from their university to the airport. It is all due to a historical accident: in 250BC, the Greek mathematician Archimedes introduced Prop 2, the Prevention of Farm Cruelty Act ( http://en.wikipedia.org/wiki/California_Proposition_2_(2008) http://en.wikipedia.org/wiki/California_Proposition_2_(2008) ). No I believe this was a different Prop 2. This Prop 2 states that the area of a circle is to the square on its diameter as 11 to 14 (http://en.wikipedia.org/wiki/Measurement_of_a_Circle http://en.wikipedia.org/wiki/Measurement_of_a_Circle ) .The confusion started then: people thought it meant areas had to do with being cruel to farm animals. But it is not just journalists who fall for the mistake: I recall seeing official documents from the department of data scientists, which found that a high number of data scientists (many with PhDs) also get confused in real life. It all comes from bad terminology for something non-intuitive. Despite this confusion, Archimedes persisted in the folly by drawing circles in the sand, an infantile persuasion, surely. When the Romans waged war, Archimedes was still computing the area of the circle. The Roman soldier asked him to step outside, but Archimedes exclaimed "Do not disturb my circles!" (http://en.wikipedia.org/wiki/Noli_turbare_circulos_meos http://en.wikipedia.org/wiki/Noli_turbare_circulos_meos) He was rightfully executed by the soldier for this grievous offense. It is sad that such a minor mathematician can lead to so much confusion: our scientific tools are way too far ahead of our casual intuitions, which starts to be a problem with a mad Greek. So I close with a statement by famed rapper Sir Joey Bada$$, extolling the virtues of the circumference: "So I keep my circumference of deep fried friends like dumplings, But fuck that nigga we munching, we hungry." (http://rapgenius.com/1931938/Joey-bada-hilary-swank/So-i-keep-my-circumference-of-deep-fried-friends-like-dumplings http://rapgenius.com/1931938/Joey-bada-hilary-swank/So-i-kee...)
al2o3cr 13y ago
Shorter social scientists: "Gaussian distribution sez wut?"
notastartup 13y ago
I've been a long time fan of Dr. Nassim Taleb. First book I've read was the one about his time as a day trader and how on the Black Friday market crash, he made a killing and cleared his desk and never had to work again. There are those that dislike his ideas because it is threatening to their existing assumptions about probability and statistics. He argues that experts and majority of people do not account for the unpredictable but significant impact a single event can have which often shatters the commonly held belief. For example, swans were white until the discovery of black swans in Oceania, too big to fail multi-national corporations going bankrupt like Lehman's brothers and etc. He's not anti-academic, but he is against teachings in the common academia that is based on naive assumptions that is specifically tailored to serve those that thrives most off the limited quantitative measures, such as market callers, hedge funds selling complicated quantitative algorithm trades, academics seeking fame and fortune by writing the most logical and quantitative paper without questioning any of the tools they are using, it is this hypocrisy and laziness that is apparent and those that try to deny to the point of making ad hominem remarks against a man, who simply observes these things and decides to write it in an entertaining manner (otherwise nobody would give a shit because the topic would be dry without lay man's linguo). Keep an open mind, a lot of what he says I do find interesting ideas and it has influenced my thinking process quite a bit, however it's no way in anyway, grounds for cracking jokes or ridicule, in fact when I read some of the comments here, it's a bit shameful. We should be embracing new ideas in order to explore them, regardless of who the explosive nature of the claim, because the black swan event is very real and is not captured or understood completely by our current set of statistical tools and methodology based on questionable assumptions about how the real world operates. For example, 1/2500 chance is not what we really think it means in the real world because black swan events are more common than we think, a percentage probability do not fully reflect it's frequency and the magnitude of it's event. Note the fall of crime rates in the United States following a decision to legalize abortion, economists and experts would come on television and bring up all sorts of random theories and ideas but little did they realize it was a chain effect from a court ruling passed decades ago until two economists came out with a paper that was ridiculed because it suggested that 'killing babies from poor neighbourhoods = lower crime rate' where most poor neighbourhoods is occupied by African Americans. Because such idea was earthshakingly controversial and still denied even to this day. Because Galileo claimed the earth was round instead of flat, he was executed. This is simply the nature of our world, almost all part of life, there exists a hierarchy that people simply do not ask questions either due to blind trust or the fear of reprisal.
MaysonL 13y ago
How often do "six sigma" events occur in financial markets? A hell of a lot more often then the 0.0000001973% that they would in a normally distributed system.
- tsax 13y ago
  Exactly. I'm astounded that this basic fact - that financial modelers have been using the wrong distribution for decades IS NOT BEING DENOUNCED. For example, the Capital Asset Pricing Model (CAPM) which assumes that returns fall on a normal distribution.
justin66 13y ago
Taleb has a textbook draft up which is more technical than his popular writings: http://www.fooledbyrandomness.com/FatTails.html http://www.fooledbyrandomness.com/FatTails.html There might be something there for the more rabid critics. At least it will keep them off the internet for a few days...
snake_plissken 13y ago
I've always thought his writings were more allegorical than scientific; you can't rely on the standard deviation to never go against you at the worst possible time. But like anything else, it can and it (probably) will. Also, yes, his writing style is grating and he takes opportunistic character swipes at pretty much everyone.
tn13 13y ago
There is nothing wrong with STD or MAD. The real problem is a lot of people apply them without realizing the nature of their data and what kind of analysis they want to do. In this case what matters in the end is the kind of impact deviation from mean has on the real world variable you have. I agree that in most Gaussian experiments MAD might be more useful than STD. STD is more useful when the real world impact of the deviation increases exponentially with the magnitude of deviation and hence it is a good idea of magnify the (x-n) by squaring it. In many cases the impact is linear where MAD clearly works better. For example in cricket where n runs are n times better than 1 run. But in case of shooting. Hitting 9 targets out of 10 might be 100 times better than 1 out of 10 so there MAD will be misleading.
Beliavsky 13y ago
If data is drawn from a Laplace distribution of the form p(x) = exp(-|x|), the mean absolute deviation is more informative than the standard deviation, but if its form is close to the normal, p(x) = exp(-x^2), the standard deviation is more important. So whether to use the mean absolute or standard deviation depends on the distribution of the data. There is a field called robust statistics that looks at this question.
ChristianMarks 13y ago
Climate scientists--among others--have made similar recommendations to use the absolute mean error in place of the standard deviation, depending on the application. Taleb might have cited the extensive methodological literature--for example: Cort J. Willmott, Kenji Matsuuraa, Scott M. Robeson. Ambiguities inherent in sums-of-squares-based error statistics. Atmospheric Environment 43 (2009) 749–752. URL: http://climate.geog.udel.edu/~climate/publication_html/Pdf/WMR_Atmos_Env_09.pdf http://climate.geog.udel.edu/~climate/publication_html/Pdf/W...
tehwalrus 13y ago
at least he's leaving us physicists alone with it...
thetwiceler 13y ago
It is sad that Taleb does not see the value in the standard deviation; standard deviation is far more natural, and more useful, than MAD. For example, if X has a standard deviation of s, and Y has a standard deviation of t, then the standard deviation of X + Y is sqrt(s^2 + t^2). There is a geometry of statistics, and the standard deviation is the fundamental measure of length. To retire the standard deviation is to ignore the wonderful geometry inherent in statistics. Covariance is one of the most important concepts in statistics, and it is a shame to hide it from those who use statistics. Additionally, I will mention that we do not need normal distributions to make special the idea of standard deviations. In fact, it is the geometry of probability - the fact that independent random variables have standard deviations which "point" in orthogonal directions - which causes the normal distribution to be the resulting distribution of the central limit theorem.
bayesianhorse 13y ago
Nassim Taleb somehow likes to beat up on normals... We Bayesians have similar notions, but we usually try not to overly bully frequentist methods, the poor things. Also, being familiar with Bayesian methods, a lot of what Taleb is saying sounds vaguely familiar...
- kylebrown 13y ago
  Well, frequentist methods are taking the blame for the most recent financial crisis, such as assuming a normal distribution when the empirical one is fat-tailed. Perhaps the Bayesian methods will take the blame in the next financial crisis. Such as the error in estimating a non-stationary distribution and quantifying the uncertainty.
  - bayesianhorse 13y ago
    Statistical economists or analysts always said that the normal distribution is a simplification and that this simplification has its own problems. It's just that traders and bankers would fire statisticians who were too vocal about it as wasting their time with unnecessary explanations... Bayesian methods in general can only take the blame if you can prove some other method being more reliable. Also they have another advantage: Bayesian methods are so mind blowing and beautiful, that it is hard to blame them for anything!
deleted 13y ago
[deleted]
beloch 13y ago
I'm a physicist, so I'm one of the people this guy says standard deviation is still good for. However, despite some "oddities" (pointed out by others here) in his article, I'm more than willing to admit a simpler, easier to understand term would be helpful for explaining many things to the general public. Hell, it would be helpful for explaining things to journalists, who we then trust to explain things to the public! Look at an reputable news site or paper. Odds are they post articles based on polls several times a day. How many report confidence intervals or anything of the sort? These are crucial for interpreting polls, but are left out more often than not. Worse yet, many stories make a big deal about a "huge" shift in support for some political policy, party or figure, when the previous month's figure is actually well within the confidence interval of the current month's poll! Standard deviation, confidence intervals, etc. are all ways of expressing uncertainty, and it's become abundantly clear that the average journalist, to say nothing of the average person, has no clue about what the concept means. If the goal is to communicate with the public, then we really need to take a step back and appreciate the stupendously colossal wall of ignorance we're about to butt our heads against. When we talk about the general public, we should keep in mind that rather a lot of people know so little about the scientific method that they interpret the impossibility of proving theories as justification for giving religious fables equal footing in schools. This kind of ignorance isn't a nasty undercurrent lurking in the shadows. It's running the show, as evidenced by many state laws in the U.S.! There is absolutely no hope of explaining uncertainty to most of these people. There is hope of explaining basic statistics to journalists, if only because they are relatively few in number and it's a fundamental part of their job to understand what they are reporting. Yes, I just said that every journalist who has reported a poll result, scientific figure, etc. without the associated uncertainty has failed to adequately perform their job. We need to make journalists understand why they are failing. If simplifying the way we report uncertainties will assist with this, then I'm all for it. Bad journalism is a root cause of a great deal of ignorance, but it's not an insurmountable task to fix it. If you are a scientist who speaks to journalists about your work, make sure they include uncertainties. If you are an editor, slap your peons silly if they write a sensationalistic poll piece when the uncertainties say it's all a bunch of hot air. If you are a reader, please mercilessly mock bad articles and write numerous scornful letters to the editor until those editors pull out their beat-sticks and get slap-happy. We should not tolerate this kind of crap from people who are paid to get it right.
TTPrograms 13y ago
There is some argument that MAD is actually better than RMS for a lot of applications. Apparently it predated RMS, but one of the reasons it was switched to was because RMS minimizing linear regression is much, much simpler to calculate. Also consider comparing the robustness of RMS based regression with MAD based regression. See: http://matlabdatamining.blogspot.com/2007/10/l-1-linear-regression.html http://matlabdatamining.blogspot.com/2007/10/l-1-linear-regr...
yetanotherphd 13y ago
I had hoped this would be about the revolution occurring in statistics/econometrics where confidence intervals based on strong parametric assumptions (e.g. the confidence intervals you would obtain using the standard deviation) are being replaced by confidence intervals obtained using the bootstrap (and other non-parametric methods) that don't rely on such strong assumptions. But no, it is just advocating using Mean absolute distance instead of the standard deviation. Which I guess is to be expected from someone whose work focuses mostly on long-tailed distributions. Still, I think that non-parametric methods are much more valuable as a solution to dealing with non-normal data than what Taleb is proposing.
aredington 13y ago
The way I read it he's proposing two things: 1) Refer to the analysis of Root Mean Square Error always by that name. (RMS is already often used in certain jargon instead of stddev). 2) Stop treating RMS as a default measure of variance. Treat Mean Absolute Deviation as the default measure of variance, because the figure it provides is more consistent with people's psychological interpretation. It's not really retiring RMS, just retiring the idea that it is a good default statistical analysis.

248 points by pyduan 13y ago

You gotta love the acronyms: STD versus MAD! Taleb is definitely mad but his use of the MAD acronym (mean absolute deviation) is actually correct. However the STD acronym (all caps) refers to "sexually transmitted disease" and not generally used for "standard deviation". Most people use SD, Stdev, StDev or sigma. Once again his ability to coin new terminology outstrips his ability to form coherent ideas that are anything more than trivial (eg. we have known about fat tails in stock returns for 50+ years). Like George Soros[1], Taleb's success says more about the state of the world of finance than their contributions to our knowledge. [1]-See his book "The Alchemy of Finance"

Fomite 13y ago
STD is also an obsolete term. Epidemiology, Medicine etc. most often now refer to them as STIs, or Sexually Transmitted Infections. The reasoning for this is that many sexually transmitted infections can be acquired, passed on to others, etc. without causing any clinical symptoms. See: HPV, among others.

truthteller 13y ago

he's really lost the plot. :(

zeidrich 13y ago

It's not that we should retire the notion of standard deviation. It's more that we should understand the tools that we are using and use the appropriate tool for the job.

deleted 13y ago
[deleted]

dschiptsov 13y ago

Why, it is pretty good in describing probability distributions. What we should retire are idiots, who assume that it predicts an outcome of the next event.

regularfry 13y ago
Everyone is an idiot some of the time.

valtron 13y ago

He makes a good point about infinite MAD vs. STD.

scythe 13y ago

While the mean deviation as presented is slightly nicer than sigma for intuitive purposes, it isn't as appropriate (iirc) for statistical tests on normal distributions and t-distributions. More importantly, it doesn't fix the real problem, which is that the mean and standard deviation don't tell you everything you need to know about a data set, but often people like to pretend they do. It's not rare to read a paper in the soft sciences which might have been improved if the authors had reported the skewness, kurtosis, or similar data which could shed light on the phenomenon they're investigating. These latter statistics can reveal, for instance, a bimodal distribution, which could indicate a heterogeneous population of responders and non-responders to a drug, and that's just one example. I'm not a statistician, so some of this might be a bit off.

drblast 13y ago
Whenever you try to describe a large data set with a single number, you lose a lot of information, like you said. Having more measurements helps, but I think the larger point is that we don't have to do this anymore, we could publish the entire data set instead. Without computers, this would be a waste of paper, but transmitting the data electronically is cheap. So why argue over the measurements? Publish the data and my software can give me any measurement I'm interested in.

cwyers 13y ago

"In fact, whenever people make decisions after being supplied with the standard deviation number, they act as if it were the expected mean deviation." Boy, is that statement useless without any kind of context, example or citation.

icebraining 13y ago
The next two paragraphs have examples with some context.
- cwyers 13y ago
  Like this? "But it is not just journalists who fall for the mistake: I recall seeing official documents from the department of commerce and the Federal Reserve partaking of the conflation, even regulators in statements on market volatility. What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life." It doesn't tell us what happened, it just asserts that it did in certain contexts. It doesn't cite the paper he presumably wrote with Goldstein (which Goldstein?) about it. I feel like I'm getting a summary of an abstract with all the citations missing.
  - lstamour 13y ago
    > which Goldstein? From ClementM above, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480
    - cwyers 13y ago
      Thank you. From that paper: "We first posed this question to 97 portfolio managers, assistant portfolio managers, and analysts employed by investment management companies who were taking part in a professional seminar. The second group of participants comprised 13 Ivy League graduate students preparing for a career in financial engineering. The third group consisted of 16 investment professionals working for a major bank." From the article: "What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life." I get that "data scientist" is a really broad term at this point, but I don't think it's a very good description of the people quizzed in this paper, if this is the paper he was indeed referring to.

deleted 13y ago

[deleted]

ClementM 13y ago

This article is based on paper Taleb published in 2007. If you want to test yourself, submit yourself to experiment in page 3: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480 http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480

drblast 13y ago
Thanks! Always nice to read the full research.
CamperBob2 13y ago
A stock (or a fund) has an average return of 0%. It moves on average 1% a day in absolute value; the average up move is 1% and the average down move is 1%. How does that yield an average return of 0%?
- MaysonL 13y ago
  It goes up a little more often than it goes down?
- baking 13y ago
  The usual way to do this is to take the natural log, so an up 1% day followed by a down 1% day (or vice versa) will always net out to a 0% change.
baking 13y ago
Yes, start by writing a confusing question. One that starts talking about average up moves and average down moves and then switches to asking about the standard deviation of moves. Then publish a paper showing that people were confused by your question. Now you have "research" to back up your claim that everyone is confused by "mean deviation" and "standard deviation".

n00b101 13y ago

Taleb has a good point about people mistakenly interpreting standard deviation (sigma) as Mean Absolute Deviation (MAD). I like that he gives some conversions (sigma ~= 1.25 * MAD, for Normal distribution). I think it's rather silly to talk about "retiring" standard deviation, but we can't blame Taleb - the publication itself posed the question "2014: What Scientific Idea is Ready for Retirement?" to various scientific personalities. What Taleb failed to mention is that, once properly understood, standard deviation has distribution interpretations that can be much more useful than MAD. For example, if the data is approximately normally distributed, then there is approximately a 99.99% probability that the next data observation will be <= 4 * sigma. Not everything is approximately normally distributed, but a lot of phenomena ARE normally distributed. It's a well known fact that the phenomena which Taleb is most interested in (namely, financial return time-series) are not normally distributed. But I would like to know how Taleb proposes to "retire" volatility (sigma) from financial theory and replace it with MAD? Standard deviation is so central in finance that even the prices of some financial instruments (options) are quoted in terms of standard deviation (e.g. "That put option is currently selling at 30% vol"). How do we rewrite Black-Scholes option pricing theory and Markowitz portfolio theory in terms of MAD and remove all the sigmas everywhere? Surely Taleb has already written that paper for us so that we can retire standard deviation?

regularfry 13y ago
I think his point is that Black-Scholes et al are holed beneath the waterline precisely because they involve standard deviations. In his world, you're better off being unable to price an option than you would be with Black-Scholes. Your example of "That put option is currently selling at 30% vol" is actually an example of why the system is so completely broken: if volatility as standard deviation was valid, all options against the same underlying instrument would have the same implied volatility. The volatility smile shouldn't exist. This wouldn't matter if the down-side wasn't so crippling. I don't think Taleb has to be the one to propose a replacement for portfolio theory, and I think criticism of him for not doing so is pointless. You don't need to have a spare tire handy to point out that your neighbour's car has a flat, and you don't have to run an airline to tell people not to get on a plane with the engines visibly on fire.

bluecalm 13y ago

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you to measure the "average daily variations" for the temperature of your town (or for the stock price of a company, or the blood pressure of your uncle) over the past five days. The five changes are: (-23, 7, -3, 20, -1). How do you do it? Ok... if I am to calculate the average I am calculating the average if I need to know standard deviation I calculate standard deviation... >> It corresponds to "real life" much better than the first—and to reality. What the flying fuck. What "real life" ? Standard deviation tells you how volatile measurements are not what mean deviation is. Those are both very real life things just not the same thing. >>It is all due to a historical accident: in 1893, the great Karl Pearson introduced the term "standard deviation" for what had been known as "root mean square error". The confusion started then: people thought it meant mean deviation. I don't know how one can read it and not think: "is this guy high or just stupid?". >>. The confusion started then: people thought it meant mean deviation. I am yet to see anybody who thinks that standard deviation is mean deviation. It's Taleb though. Baseless assertions insulting groups of people are his craft. >>What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life. One example please ? I can give hundreds when std dev is useful and mean deviation isn't. Anything when you decide what % of yoru bankroll to bet on perceived edge for example. Ok so he asserted that people should just use mean deviation instead of mean of squares. Guess what though, taking the squares have a purpose: it penalizes big deviations so two situations which have the same mean deviation but one is more stable have different standard deviations. THis information is useful for many things: risk estimation or calculating sample size needed for required confidence (if you need more experiments, how careful should you be with conclusions and predictions etc). He didn't mention how are we going to achieve those with his proposal. Meanwhile he managed to throw insults towards various groups without giving one single example of misuse he describes. This is not the first time he writes something this way. His whole recent book is like that. It's anti-intellectual bullshit with many words and zero points. He doesn't give any arguments, he throws a lot of insults, he misues words and makes up redundant terms which he then struggles to define. The guy is a vile idiot of the worst kind: ignorant and aggressive. Him gaining so much following by spewing nonsense like this article is for sure fascinating but there is no place for him in any serious debate.

RivieraKid 13y ago

I was just wondering about a very related problem. I do 5 measurenments of some random variable (let's say execution time) and average them. How should I report the variability of that average? State the sample size and standard deviation?

darkhorn 13y ago
90% of the data fall between this point and this point. 50% of the data fall between this point and this point. etc. MAD is more accurate than Standard Deviation to me, if you ask me.
perlgeek 13y ago
If it were 10 or more measurements, I'd use average, standard deviation and sample size if we can expect that variable to be normally distributed. I don't know how to make meaningful statistics with fewer data points.
bayesianhorse 13y ago
Citing 10 measurements in a table or plot is probably easier than explaining how and why you are taking mean and standard deviation (or anything else for that matter...)

deleted 13y ago

[deleted]

JASchilz 13y ago

The central limit theorem shows us that unimodal data with lots of independent sources of error tends towards a normal distribution. That description is a good first-pass, descriptive model for lots and lots of contexts, and standard deviation speaks well to normally distributed data. Squaring error isn't just a convenient way to remove sign, it's driven by a lot of data-sets' conformance to the central limit theorem.

dj-wonk 13y ago
Thank you. I don't think it is intellectually honest for Taleb to omit this fact.

lambdasquirrel 13y ago

I think we'd be better off if we recognized that there are statistical distributions in the world besides the plain old Gaussian. For example, wealth does not follow a Gaussian, so why the heck do we throw around ideas like "above average wealth"? Is MAD any better? Definitely. But I'd like to see a visual demonstration of how well it models exponential-based distributions. How well does it describe their "shape", the skew of the tail?

chilldream 13y ago
In that specific case, I'd submit that the word "average" doesn't belong in a conversation attempting any level of rigor. It only encourages the confusion between median and mean.
- vorg 13y ago
  Medians and means are interrelated. Averages are best thought of as being on a scale from meanlike to medianlike: a median requires a mean of the middle 2 values when there's an even number of them, and a mean can have outliers on each side to be eliminated before calculation. The word average abstracts away this detail. (I guess there'd be another scale from meanlike to medianlike absolute deviation, but that's another story.)

programminggeek 13y ago

I think because it's called "standard deviation" that it sounds like the thing to use or look for. It sounds more correct because of the word standard. I feel like it is the same kind of failing due to human perception of language that programmers have with the idea of exceptions and errors, especially the phrase "exceptions should only be used for exceptional behaviors". That's a cool phrase, but people latch on to it because of the word exception sounding like something extremely rare and out of the ordinary whereas we see errors as common, but they are in fact the same thing. Broke is broke, it doesn't matter what you call it, but thousands of programmers think differently because of the name we gave it. We are human and language absolutely plays a role in our perception of things.

klodolph 13y ago
> I think because it's called "standard deviation" that it sounds like the thing to use or look for. Yes! Because it's an awesome trick and lets you do good estimates on napkins. The other day I was buying lunch at a food cart and thought about how much change the food carts had to carry, as a function of how many customers they have, under the assumption that they want to be able to provide correct change to 99% of their customers. Let's say that the average amount of change a customer needs is $5, and a 99-th percentile customer needs $15 in change. If we pretend that the distribution is approximately Gaussian we can calculate that 1,000 food carts with 1 customer each would need $15,000 in change, but 1 food cart with 1,000 customers would need $5 x 1,000 + ($15 - $5) * sqrt(1,000) ≈ $5,320. That's math you can do in your head without a calculator (being a programmer, 1,000 ≈ 2^10 so sqrt(1,000) ≈ 2^5). The standard deviation and assumptions of normality are so useful because of the central limit theorem. That is, if you have many iid variables which have finite standard deviation the sum will converge to a Gaussian distribution as the number of variables increases. Then you say "Well, the standard deviation weighs the tail too heavily" and the response is "well use higher order moments then, that's what they're they're for".
- skybrian 13y ago
  It's a neat math trick, but it seems more accurate to say this lets you calculate bad estimates on the back of a napkin. Unless you really think food carts carry $5000 in change. The quantitative work I do has to do with measuring latency, where the minimum, median, 90%, and 99% values are more meaningful than the mean or standard deviation. Programs typically have a best-case scenario (everything cached) and a long one-sided tail.

deleted 13y ago

[deleted]

roywei 13y ago

four-day returns of stock x: (-.3, .3, -.3, .3) -> MAD = 0; four-day returns of stock y: (-.5, .5, -.5, .5) -> MAD = 0.

andrewcooke 13y ago
nope. look at what the "A" in MAD stands for.
michaelhoffman 13y ago
No, it's the median absolute deviation. Absolute deviation is the absolute difference between an average and a data point. So (-.3, .3, -.3, .3) -> MAD = 0.3; (-.5, .5, -.5, .5) -> MAD = 0.5.
jameshart 13y ago
The A in MAD stands for 'absolute', so no, the MAD for those two stocks is .3 and .5 respectively - versus standard deviations of 0.35 and 0.58.
spikels 13y ago
You have calculated the mean. This is only the first step in calculating the MAD (mean absolute deviation). You then need to take the absolute value of the difference (deviation) of each datapoint from this mean - for your x you get (.3, .3, .3, .3) and for your y (.5, .5, .5, .5). Finally you take the mean of these absolute deviations to get MAD(x)=0.3 and MAD(x)=0.5.

cheald 13y ago

I really tried to get through "The Black Swan" and Taleb's writing struck me as so pretentious and self-involved that it made it impossible for me to finish. He strikes me as someone who is so desperate to be important and recognized that an assertion like this doesn't really surprise me.

andrewcooke 13y ago
fascinating.
Alex_MJ 13y ago
His facebook doesn't inspire a lot of confidence. It's full of vague availability-heuristic-targeting generalities. "Virtue is when the income you wish to show the tax agency equals what you wish to show your neighbor" "The problem is that academics really think that nonacademics find them more intelligent than themselves", etc I've read black swan as well, and there were parts I didn't quite grok at the time (You cannot predict a black swan! etc) My take is that he's a Malcolm Gladwell with numbers and therefore less easily takedownable but has the same model of "hold prestigious academic position, appear wise, publish book with simple principle, be easily referenceable by people who want to sound well-read" except he has lots of math and charts to point at lest anyone call him out on it.
calroc 13y ago
I started reading that book and I thought he was a genius, but then, right abut the point where he starts bagging on the Uncertainty Principle, I realized that he's actually kind of an idiot. The book does make a good point though.
- cheald 13y ago
  Maybe I'm the idiot, but I found the book's point to be pretty trivial. "Sometimes unexpected things happen, and people are bad at expecting the unexpected" seems pretty damn trivial to me. Maybe it's because I think like a programmer rather than a finance guy, but a large portion of what I do on a daily basis is about mitigating risk from the unknown. A programmer who only guards against known risks is going to get his ass handed to him sooner rather than later (these are the sorts of people who use blacklists and regexes to sanitize SQL). A finance guy who only guards against known risks is just playing the averages. My experience with traders is that they tend to be pretty abstracted from reality and like to impose rules and patterns where there are none. I can see why those sorts of people would find Taleb's work groundbreaking. Taleb's undoubtedly intelligent, but I feel like he woke up one day, decided that he wanted to be a philosopher who brings wisdom to the masses, and built his temple on a mind-searingly obvious principle, which he now proclaims to be his great gift to humanity, for which he should be praised, hallowed be his name. The impression I got off of him was that he considers himself a prophet, imparting a word to the masses that the rest of us are just too stupid to recognize. Gag me.

Homunculiheaded 13y ago

I sometimes think that progress in the 21st century will be summed up as: "The realization that the normal distribution is not the only way to model data". Taleb's favorite topic is the "black swan event" which is something that the normal distribution, and the idea of standard deviation, don't model that well. In a normal distribution very extreme events should only happen once in the lifetime of several universes. Of course assuming variation inline with a Gaussian process is at the heart of how the Black-Sholes model calculates risk/volatility/etc. Benoit Mandelbrot argued that financial markets follow a distribution much more similar to the Cauchy distribution (specifically the Levy distribution) rather than a Gaussian. The problem of course is that the Cauchy distribution is pathological in that it doesn't have a mean or variance, you can calculate similar properties for it (location and scale), but it doesn't obey the central limit theorem so in practice it can be very strange to work with. The normal distribution is fantastic in that it does appear frequently in nature, is very well behaved, and has been extensively studied. However a great amount of future progress is going to come from wrestling with more challenging distributions, and paying more attention to when assumptions of normality need to be questioned. Of course one of the challenges of this is that the normal distribution is baked into a very large number of our existing statistical tools.

jerf 13y ago
This is actually what I expected to read: "The standard deviation is useful because with the average and the standard deviation, one can fully characterize a normal distribution. However, the standard deviation is less useful a statistical summary the farther away from 'normal' you get, and in reality, there is no such thing as a normal distribution, as a true normal distribution is defined on the entire real number line from negative infinity to positive infinity. Reality always provides some bound, and it's often quite distorted from Guassian. For instance, a 'normal' distribution averaging 2 with a standard deviation of 1.4, bounded by 0, is quite non-Gaussian in many important ways! (Not least of which is that you're going to have to do something to replace the missing probability...) "People rarely check how closely their data conform to the standard distribution; indeed, many people blindly apply the standard deviation to their data regardless of its distribution! The resulting number is often more obfuscatory than helpful, to the extent that it crowded out more useful summaries. "It's a useful metric when treated carefully, but it is rare to encounter it treated carefully. Science courses would be well-served to stop teaching it in favor of a stronger emphasis on multiple distributions. (Multiple distributions are usually touched upon, but implicitly our curricula overfavor the Gaussian distribution and end up accidentally implicitly convincing students its the only one.)" But that's just me.

Glyptodon 13y ago

All I know is this reminds me a lot of high school where we had to always compute std dev in problems, homework, and sometimes labs, but nobody really ever explained how to interpret it. It was always like "This is std dev. This is how you compute it. Make sure you put it your tables and report." Eventually someone (or something) did explain it, but once I understood it, it became clear that it wasn't always a sensible thing to be asked to calculate but was instead just an instinctive requirement.

afterburner 13y ago

I've found MAD a potentially useful measure for monitoring whether something gets out of whack; when using STD I needed to modify it to give less weighting to outliers.

randomsample2 13y ago

Standard deviation and mean absolute deviation are both useful, but I think it's silly to suggest that we all adopt exactly one measure of variability to summarize data sets. When in doubt, make a fucking histogram.

michaelhoffman 13y ago
Histograms can be misleading too, especially when the breaks are not set well.
- randomsample2 13y ago
  Anything can be misleading, but in most cases a histogram conveys more information than a single number ever could.
  - michaelhoffman 13y ago
    Absolutely. They just aren't a panacea.

etanazir 13y ago

The minimum uncertainty wave equation is ~ e^(-x^2) ergo the standard measure is in terms of x^2. QED.

vzhang 13y ago

I'm seriously questioning some people's reading comprehension - he NEVER said STD is not useful! He's only saying the name "Standard Deviation" is badly chosen.

bluecalm 13y ago
Read the blog post carefully: > it is time to retire it from common use and replace it with the more effective one of mean deviation > Standard deviation, STD, should be left to mathematicians, physicists and mathematical statisticians deriving limit theorems >There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good He is saying it's not useful for real world things and people are just confused. What you wrote is reasonable view. What Taleb writes isn't.

puranjay 13y ago

NNT is my intellectual superhero but the amount of hate he gets is tremendous. Please understand that NNT's biggest issues are not so much with the way statistical models are applied to economics and finance, but how social scientists sometimes feel compelled to apply them to social fields as well, which is plain unscientific, dumb, and mostly disastrous. So when you bear down on his arguments, please keep this context in mind.

triplesec 13y ago
You're making too many gross generalisations here, as does Mr Taleb. He's right that in all fields there are statistically illiterate people making huge errors, and academic theorists have been making such - and more complete - criticisms for a while now. So, there's a real issue. However, there are a good number of competent social scientists too, beyond the mediocre morass, who understand just what variable construction and limitations to statistical operations are.
Fomite 13y ago
Among other things, I've seen way more appalling applications of statistics in finance and economics than I have in social science. Also, the assertion in your post that the misapplication of statistical models in social science is "disastrous" but somehow giving finance a pass? You've got to be kidding me.
- jessaustin 13y ago
  NNT, at least, doesn't give finance a pass. He excoriates them at every opportunity.
  - Fomite 13y ago
    That's why I was so puzzled - NNT rather eagerly takes finance to task.

dxbydt 13y ago

The notion of area has confused hordes of scientists; it is time to retire it from common use and replace it with the more effective one of circumference. Area should be left to mathematicians, topologists and developers selling real estate. There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good. Say someone just asked you to measure the area of a circle with radius pi. The area is exactly 31. But how do you do it? scala> math.round(math.Pi * math.Pi * math.Pi).toInt res1: Int = 31 Do you pack the circle with n people, count them up and verify n == 31 ? Or do you pour a red liquid into the circle and fill it up, then drain it and measure the amount of red ? For there are serious differences between the two methods. If instead, you were asked to measure the circumference of a circle with radius pi. scala> math.round(2 * math.Pi * math.Pi).toInt res2: Int = 20 You just ask an able-bodied man, perhaps an unemployed migrant, to walk around this circle while another man, an upstanding Stanford sophomore, starts walking from Stanford to meet his maker, I mean VC, well its the same thing... So by the time the migrant finishes walking around the circle, our upstanding Stanford entrepreneur is greeting the VC on the tarmac of the San Francisco International Airport. This leads one to rightfully believe that the circumference of the circle of radius pi is exactly the distance from Stanford to the SF Airport ie. 20 miles. It corresponds to "real life" much better than the first—and to reality. In fact, whenever people make decisions after being supplied with the area, they act as if it were the distance from their university to the airport. It is all due to a historical accident: in 250BC, the Greek mathematician Archimedes introduced Prop 2, the Prevention of Farm Cruelty Act ( http://en.wikipedia.org/wiki/California_Proposition_2_(2008) http://en.wikipedia.org/wiki/California_Proposition_2_(2008) ). No I believe this was a different Prop 2. This Prop 2 states that the area of a circle is to the square on its diameter as 11 to 14 (http://en.wikipedia.org/wiki/Measurement_of_a_Circle http://en.wikipedia.org/wiki/Measurement_of_a_Circle ) .The confusion started then: people thought it meant areas had to do with being cruel to farm animals. But it is not just journalists who fall for the mistake: I recall seeing official documents from the department of data scientists, which found that a high number of data scientists (many with PhDs) also get confused in real life. It all comes from bad terminology for something non-intuitive. Despite this confusion, Archimedes persisted in the folly by drawing circles in the sand, an infantile persuasion, surely. When the Romans waged war, Archimedes was still computing the area of the circle. The Roman soldier asked him to step outside, but Archimedes exclaimed "Do not disturb my circles!" (http://en.wikipedia.org/wiki/Noli_turbare_circulos_meos http://en.wikipedia.org/wiki/Noli_turbare_circulos_meos) He was rightfully executed by the soldier for this grievous offense. It is sad that such a minor mathematician can lead to so much confusion: our scientific tools are way too far ahead of our casual intuitions, which starts to be a problem with a mad Greek. So I close with a statement by famed rapper Sir Joey Bada$$, extolling the virtues of the circumference: "So I keep my circumference of deep fried friends like dumplings, But fuck that nigga we munching, we hungry." (http://rapgenius.com/1931938/Joey-bada-hilary-swank/So-i-keep-my-circumference-of-deep-fried-friends-like-dumplings http://rapgenius.com/1931938/Joey-bada-hilary-swank/So-i-kee...)

al2o3cr 13y ago

Shorter social scientists: "Gaussian distribution sez wut?"

notastartup 13y ago

I've been a long time fan of Dr. Nassim Taleb. First book I've read was the one about his time as a day trader and how on the Black Friday market crash, he made a killing and cleared his desk and never had to work again. There are those that dislike his ideas because it is threatening to their existing assumptions about probability and statistics. He argues that experts and majority of people do not account for the unpredictable but significant impact a single event can have which often shatters the commonly held belief. For example, swans were white until the discovery of black swans in Oceania, too big to fail multi-national corporations going bankrupt like Lehman's brothers and etc. He's not anti-academic, but he is against teachings in the common academia that is based on naive assumptions that is specifically tailored to serve those that thrives most off the limited quantitative measures, such as market callers, hedge funds selling complicated quantitative algorithm trades, academics seeking fame and fortune by writing the most logical and quantitative paper without questioning any of the tools they are using, it is this hypocrisy and laziness that is apparent and those that try to deny to the point of making ad hominem remarks against a man, who simply observes these things and decides to write it in an entertaining manner (otherwise nobody would give a shit because the topic would be dry without lay man's linguo). Keep an open mind, a lot of what he says I do find interesting ideas and it has influenced my thinking process quite a bit, however it's no way in anyway, grounds for cracking jokes or ridicule, in fact when I read some of the comments here, it's a bit shameful. We should be embracing new ideas in order to explore them, regardless of who the explosive nature of the claim, because the black swan event is very real and is not captured or understood completely by our current set of statistical tools and methodology based on questionable assumptions about how the real world operates. For example, 1/2500 chance is not what we really think it means in the real world because black swan events are more common than we think, a percentage probability do not fully reflect it's frequency and the magnitude of it's event. Note the fall of crime rates in the United States following a decision to legalize abortion, economists and experts would come on television and bring up all sorts of random theories and ideas but little did they realize it was a chain effect from a court ruling passed decades ago until two economists came out with a paper that was ridiculed because it suggested that 'killing babies from poor neighbourhoods = lower crime rate' where most poor neighbourhoods is occupied by African Americans. Because such idea was earthshakingly controversial and still denied even to this day. Because Galileo claimed the earth was round instead of flat, he was executed. This is simply the nature of our world, almost all part of life, there exists a hierarchy that people simply do not ask questions either due to blind trust or the fear of reprisal.

MaysonL 13y ago

How often do "six sigma" events occur in financial markets? A hell of a lot more often then the 0.0000001973% that they would in a normally distributed system.

tsax 13y ago
Exactly. I'm astounded that this basic fact - that financial modelers have been using the wrong distribution for decades IS NOT BEING DENOUNCED. For example, the Capital Asset Pricing Model (CAPM) which assumes that returns fall on a normal distribution.

justin66 13y ago

Taleb has a textbook draft up which is more technical than his popular writings: http://www.fooledbyrandomness.com/FatTails.html http://www.fooledbyrandomness.com/FatTails.html There might be something there for the more rabid critics. At least it will keep them off the internet for a few days...

snake_plissken 13y ago

I've always thought his writings were more allegorical than scientific; you can't rely on the standard deviation to never go against you at the worst possible time. But like anything else, it can and it (probably) will. Also, yes, his writing style is grating and he takes opportunistic character swipes at pretty much everyone.

tn13 13y ago

There is nothing wrong with STD or MAD. The real problem is a lot of people apply them without realizing the nature of their data and what kind of analysis they want to do. In this case what matters in the end is the kind of impact deviation from mean has on the real world variable you have. I agree that in most Gaussian experiments MAD might be more useful than STD. STD is more useful when the real world impact of the deviation increases exponentially with the magnitude of deviation and hence it is a good idea of magnify the (x-n) by squaring it. In many cases the impact is linear where MAD clearly works better. For example in cricket where n runs are n times better than 1 run. But in case of shooting. Hitting 9 targets out of 10 might be 100 times better than 1 out of 10 so there MAD will be misleading.

Beliavsky 13y ago

If data is drawn from a Laplace distribution of the form p(x) = exp(-|x|), the mean absolute deviation is more informative than the standard deviation, but if its form is close to the normal, p(x) = exp(-x^2), the standard deviation is more important. So whether to use the mean absolute or standard deviation depends on the distribution of the data. There is a field called robust statistics that looks at this question.

ChristianMarks 13y ago

Climate scientists--among others--have made similar recommendations to use the absolute mean error in place of the standard deviation, depending on the application. Taleb might have cited the extensive methodological literature--for example: Cort J. Willmott, Kenji Matsuuraa, Scott M. Robeson. Ambiguities inherent in sums-of-squares-based error statistics. Atmospheric Environment 43 (2009) 749–752. URL: http://climate.geog.udel.edu/~climate/publication_html/Pdf/WMR_Atmos_Env_09.pdf http://climate.geog.udel.edu/~climate/publication_html/Pdf/W...

tehwalrus 13y ago

at least he's leaving us physicists alone with it...

thetwiceler 13y ago

It is sad that Taleb does not see the value in the standard deviation; standard deviation is far more natural, and more useful, than MAD. For example, if X has a standard deviation of s, and Y has a standard deviation of t, then the standard deviation of X + Y is sqrt(s^2 + t^2). There is a geometry of statistics, and the standard deviation is the fundamental measure of length. To retire the standard deviation is to ignore the wonderful geometry inherent in statistics. Covariance is one of the most important concepts in statistics, and it is a shame to hide it from those who use statistics. Additionally, I will mention that we do not need normal distributions to make special the idea of standard deviations. In fact, it is the geometry of probability - the fact that independent random variables have standard deviations which "point" in orthogonal directions - which causes the normal distribution to be the resulting distribution of the central limit theorem.

bayesianhorse 13y ago

Nassim Taleb somehow likes to beat up on normals... We Bayesians have similar notions, but we usually try not to overly bully frequentist methods, the poor things. Also, being familiar with Bayesian methods, a lot of what Taleb is saying sounds vaguely familiar...

kylebrown 13y ago
Well, frequentist methods are taking the blame for the most recent financial crisis, such as assuming a normal distribution when the empirical one is fat-tailed. Perhaps the Bayesian methods will take the blame in the next financial crisis. Such as the error in estimating a non-stationary distribution and quantifying the uncertainty.
- bayesianhorse 13y ago
  Statistical economists or analysts always said that the normal distribution is a simplification and that this simplification has its own problems. It's just that traders and bankers would fire statisticians who were too vocal about it as wasting their time with unnecessary explanations... Bayesian methods in general can only take the blame if you can prove some other method being more reliable. Also they have another advantage: Bayesian methods are so mind blowing and beautiful, that it is hard to blame them for anything!

deleted 13y ago

[deleted]

beloch 13y ago

I'm a physicist, so I'm one of the people this guy says standard deviation is still good for. However, despite some "oddities" (pointed out by others here) in his article, I'm more than willing to admit a simpler, easier to understand term would be helpful for explaining many things to the general public. Hell, it would be helpful for explaining things to journalists, who we then trust to explain things to the public! Look at an reputable news site or paper. Odds are they post articles based on polls several times a day. How many report confidence intervals or anything of the sort? These are crucial for interpreting polls, but are left out more often than not. Worse yet, many stories make a big deal about a "huge" shift in support for some political policy, party or figure, when the previous month's figure is actually well within the confidence interval of the current month's poll! Standard deviation, confidence intervals, etc. are all ways of expressing uncertainty, and it's become abundantly clear that the average journalist, to say nothing of the average person, has no clue about what the concept means. If the goal is to communicate with the public, then we really need to take a step back and appreciate the stupendously colossal wall of ignorance we're about to butt our heads against. When we talk about the general public, we should keep in mind that rather a lot of people know so little about the scientific method that they interpret the impossibility of proving theories as justification for giving religious fables equal footing in schools. This kind of ignorance isn't a nasty undercurrent lurking in the shadows. It's running the show, as evidenced by many state laws in the U.S.! There is absolutely no hope of explaining uncertainty to most of these people. There is hope of explaining basic statistics to journalists, if only because they are relatively few in number and it's a fundamental part of their job to understand what they are reporting. Yes, I just said that every journalist who has reported a poll result, scientific figure, etc. without the associated uncertainty has failed to adequately perform their job. We need to make journalists understand why they are failing. If simplifying the way we report uncertainties will assist with this, then I'm all for it. Bad journalism is a root cause of a great deal of ignorance, but it's not an insurmountable task to fix it. If you are a scientist who speaks to journalists about your work, make sure they include uncertainties. If you are an editor, slap your peons silly if they write a sensationalistic poll piece when the uncertainties say it's all a bunch of hot air. If you are a reader, please mercilessly mock bad articles and write numerous scornful letters to the editor until those editors pull out their beat-sticks and get slap-happy. We should not tolerate this kind of crap from people who are paid to get it right.

TTPrograms 13y ago

There is some argument that MAD is actually better than RMS for a lot of applications. Apparently it predated RMS, but one of the reasons it was switched to was because RMS minimizing linear regression is much, much simpler to calculate. Also consider comparing the robustness of RMS based regression with MAD based regression. See: http://matlabdatamining.blogspot.com/2007/10/l-1-linear-regression.html http://matlabdatamining.blogspot.com/2007/10/l-1-linear-regr...

yetanotherphd 13y ago

I had hoped this would be about the revolution occurring in statistics/econometrics where confidence intervals based on strong parametric assumptions (e.g. the confidence intervals you would obtain using the standard deviation) are being replaced by confidence intervals obtained using the bootstrap (and other non-parametric methods) that don't rely on such strong assumptions. But no, it is just advocating using Mean absolute distance instead of the standard deviation. Which I guess is to be expected from someone whose work focuses mostly on long-tailed distributions. Still, I think that non-parametric methods are much more valuable as a solution to dealing with non-normal data than what Taleb is proposing.

aredington 13y ago

The way I read it he's proposing two things: 1) Refer to the analysis of Root Mean Square Error always by that name. (RMS is already often used in certain jargon instead of stddev). 2) Stop treating RMS as a default measure of variance. Treat Mean Absolute Deviation as the default measure of variance, because the figure it provides is more consistent with people's psychological interpretation. It's not really retiring RMS, just retiring the idea that it is a good default statistical analysis.