4 ms·

If I hadn't recently read Benoit Mandelbrot's book "The Misbehavior of Markets," I might dismiss this as empty cynicism. But I think there is some truth to it.

If I hadn't recently read Benoit Mandelbrot's book "The Misbehavior of Markets," I might dismiss this as empty cynicism. But I think there is some truth to it. Macro and financial economists have traditionally been the worst offenders.

thanatropism 9y ago
I'm ignorant of "heavy-tailed statistics" as a field, but the normal distribution in quantitative finance is justified by the Levy characterization of Brownian motion (basically every continuous stochastic process is a drift-diffusion driven by a Brownian) and the Levy representation of a discontinuous process as a sum of a diffusion and a pure jump process. Mind, you might find different distributions when solving diffusion equations (the Cox-Ingersoll-Ross process involves a Bessel distribution if I'm not mistaken). But the diffusion-jump paradigm is much better justified there than distributional (or even finite moment) assumptions in discrete land. Put it this way: if finance is abusing distributional assumptions, there's money being left on the table.
- Emilia34444 9y ago
  I'm going to show you how I make a living online! Here is a company that will pay you $100 if you don't make money in 24 hours. Take a look this company has an A+ Business Bureau Rating ¦¦¦¦F¦O¦L¦L¦O¦W¦¦M¦E¦¦¦¦¦http://ow.ly/P0wk30aVXfN http://ow.ly/P0wk30aVXfN
- srean 9y ago
  You are throwing in a lot of technical words around and the specific fact you state arent wrong, but that cannot make a strong case for the choice of the Normal distribution. All you need is that the variance of bounded (in time) increments be unbounded for the Gaussian behavior to fall apart (for example non-rare jumps of sufficient magnitude).
  - thanatropism 9y ago
    I did say "diffusion-jump process". Look, even if we disregard jumps as a possible term in equations, merely considering volatility to be stochastic and driven by a Brownian already gets you variances that may grow arbitrarily fast. This is without considering stranger nonlinearities on the Brownian term itself. People are overly impressed by forceful arguments of the Taleb variety and log-log plots of empirical distributions and start parroting talking points about heavy/fat tails. If I wasn't computerless and on my phone I would link to a paper that does the analog of the Anscombe quartet for log-log distribution plots "proving that data is Pareto/power law/etc." It's good vaccine for fat tail hipsterism, look it up.
    - srean 9y ago
      I dont think we are disagreeing (as long as you do not claim that stock prices are merely a Gaussian process.
  - thanatropism 9y ago
    I thought of a simple example on my way to my commute. (It's a short walk.) The Tanaka equation is an example (admittedly: not a diffusion, not a case o plug-and-chug the Ito lemma) of a process driven by a Brownian with a discontinuous probability distribution. How the hell? From memory, dTNK = dB if dB>0 else -dB Now imagine a model with three equations. X1 is a bog-standard geometric diffusion (we could have picked something that's chi-squared distributed driven by a Brownian from standard interest rate models) as in the 1970s Black-Scholes models. But instead of having an exogenous volatility, it has its Brownian term dB1 multiplied by a second equation, X2. dX2 could be a standard mean-reversion equation, but again its dB2 is multiplied by X3. dX3 is something like abs(dB3 - dB4* X3). Voilà, an equation (X1) with a sudden break driven by the level of a mean-reverting equation (X2, which tells us volatility should come down in finite time even if it grows by a lot at times) that's set to blow up at a X2-dependent but stochastic level. Don't get me wrong, Poisson-like jumps are very common (they're precisely the limiting process for sudden jumps) but people overstate (perhaps because they didn't really read the conditions for Ito isometry) how much a Brownian motion forces a system into normality or smoothness. But hey, people get away with being hipsters about programming languages, why shouldn't they do that for stochastic calculus too, you know?
    - thanatropism 9y ago
      Hey. Why do people upvote a post that says "Taleb is a jerk" and makes a reputation argument in a forceful manner (reminiscent of Taleb's own style) -- and then downvote a post with technical detail?
      - tripzilch 9y ago
        Probably the fact that you flatly ignored the remark about stuffing your post with opaque technical terms--I know a lot of maths, but I can't make heads or tails of what you're saying. Who are you even talking to? I assume it means something to you, but it doesn't even look like you're trying to communicate your point in a clear manner. Just that you enjoy using words. When trying to read around those technical terms, what is left has a rather nasty and arrogant tone. Therefore, doesn't add much to the discussion = downvote.
- skgoa 9y ago
  The main issue with this being that markets don't exhibit behaviour analogous to Brownian motion. It's a nice little assumption that makes the math work, but frequency analysis of real world markets shows that they behave like pink noise instead. And unsurprisingly there are quant funds and prop trading firms that use this very fact to make lots of money. Academic economist's love for the normal distribution is derided by pretty much any fellow real world trader I know.
- BooglyWoo 9y ago
  Can't recommend the Mandelbrot book highly enough - he was a student of Paul Levy, and he wrote extensively about why Gaussian is not a good choice to model financial time series.
lr4444lr 9y ago
I don't mean it to be cynical, it's just my experience that a lot of data gathering in the social sciences (my experience was in education) makes terrible assumptions, often implicitly, the worst being that the variables are IID. And it's very obvious from the publication that the number crunching is some cargo-cult approach they probably learned from their mandated semester or two in research methods. (Okay, that last bit was cynical!)
- lausiant 9y ago
  I could probably be characterized as a social scientist, at least a behavioral scientist. What you're saying is probably part of it, although in my experience that criticism can be leveled as much, if not more, at wet-lab-type biologists who eschew all but the most minimal stats. With the social sciences, though, there's another phenomenon at play, which is that the phenomena are so abstract often that there's not really a good theoretical reason to assume anything in particular. And if that's the case, because the normal is the entropy-maximizing distribution, you're actually better off assuming that rather than some other distribution. You could also use nonparametric stats, but that has its own advantages and disadvantages. Bias-variance dilemma and all that. The truth is, it's hard to beat the normal even when it's wrong. And if you subscribe to the inferential philosophy that every model is wrong, you're better off being conservatively wrong, which implies a normal. I'm not saying everything should be assumed to be normal. But unless things are (1) obviously super non-normal, or (2) you have some very strongly justified model that produces a non-normal distribution, you're probably best off using a normal if you're going to go parametric. And I think those two conditions are met much more often than we like to admit. The normal distribution is kind of over-maligned, I think. I started my stats career being enamoured of rigorously nonparametric stats, and still am (esp. exact tests, bootstrapping/permutation-based inference, and empirical likelihood), but have grown to strongly appreciate normal distributions (or whatever maxent distribution is appropriate).
  - CuriouslyC 9y ago
    From my perspective, either you have data and you have an idea what sort of distributions you're working with, or you don't, and you should fix that problem first rather than going down the theory rabbit hole. With data in hand, a skew/kurtosis scatter plot is a good way to gauge the higher dimensional distribution of your data. Another option is to cluster the variables of the data set using something like HDBSCAN and color the plot points based on cluster membership. If you have to go guessing distributions without evidence, you're better off choosing a low k student's T distribution (for robustness to outliers) or a gamma distribution (if you think your data might be skewed).
lmm 9y ago
Disagree; economists have failed more visibly, but they've got nothing on social scientists.

thanatropism 9y ago

I'm ignorant of "heavy-tailed statistics" as a field, but the normal distribution in quantitative finance is justified by the Levy characterization of Brownian motion (basically every continuous stochastic process is a drift-diffusion driven by a Brownian) and the Levy representation of a discontinuous process as a sum of a diffusion and a pure jump process. Mind, you might find different distributions when solving diffusion equations (the Cox-Ingersoll-Ross process involves a Bessel distribution if I'm not mistaken). But the diffusion-jump paradigm is much better justified there than distributional (or even finite moment) assumptions in discrete land. Put it this way: if finance is abusing distributional assumptions, there's money being left on the table.

Emilia34444 9y ago
I'm going to show you how I make a living online! Here is a company that will pay you $100 if you don't make money in 24 hours. Take a look this company has an A+ Business Bureau Rating ¦¦¦¦F¦O¦L¦L¦O¦W¦¦M¦E¦¦¦¦¦http://ow.ly/P0wk30aVXfN http://ow.ly/P0wk30aVXfN
srean 9y ago
You are throwing in a lot of technical words around and the specific fact you state arent wrong, but that cannot make a strong case for the choice of the Normal distribution. All you need is that the variance of bounded (in time) increments be unbounded for the Gaussian behavior to fall apart (for example non-rare jumps of sufficient magnitude).
- thanatropism 9y ago
  I did say "diffusion-jump process". Look, even if we disregard jumps as a possible term in equations, merely considering volatility to be stochastic and driven by a Brownian already gets you variances that may grow arbitrarily fast. This is without considering stranger nonlinearities on the Brownian term itself. People are overly impressed by forceful arguments of the Taleb variety and log-log plots of empirical distributions and start parroting talking points about heavy/fat tails. If I wasn't computerless and on my phone I would link to a paper that does the analog of the Anscombe quartet for log-log distribution plots "proving that data is Pareto/power law/etc." It's good vaccine for fat tail hipsterism, look it up.
  - srean 9y ago
    I dont think we are disagreeing (as long as you do not claim that stock prices are merely a Gaussian process.
- thanatropism 9y ago
  I thought of a simple example on my way to my commute. (It's a short walk.) The Tanaka equation is an example (admittedly: not a diffusion, not a case o plug-and-chug the Ito lemma) of a process driven by a Brownian with a discontinuous probability distribution. How the hell? From memory, dTNK = dB if dB>0 else -dB Now imagine a model with three equations. X1 is a bog-standard geometric diffusion (we could have picked something that's chi-squared distributed driven by a Brownian from standard interest rate models) as in the 1970s Black-Scholes models. But instead of having an exogenous volatility, it has its Brownian term dB1 multiplied by a second equation, X2. dX2 could be a standard mean-reversion equation, but again its dB2 is multiplied by X3. dX3 is something like abs(dB3 - dB4* X3). Voilà, an equation (X1) with a sudden break driven by the level of a mean-reverting equation (X2, which tells us volatility should come down in finite time even if it grows by a lot at times) that's set to blow up at a X2-dependent but stochastic level. Don't get me wrong, Poisson-like jumps are very common (they're precisely the limiting process for sudden jumps) but people overstate (perhaps because they didn't really read the conditions for Ito isometry) how much a Brownian motion forces a system into normality or smoothness. But hey, people get away with being hipsters about programming languages, why shouldn't they do that for stochastic calculus too, you know?
  - thanatropism 9y ago
    Hey. Why do people upvote a post that says "Taleb is a jerk" and makes a reputation argument in a forceful manner (reminiscent of Taleb's own style) -- and then downvote a post with technical detail?
    - tripzilch 9y ago
      Probably the fact that you flatly ignored the remark about stuffing your post with opaque technical terms--I know a lot of maths, but I can't make heads or tails of what you're saying. Who are you even talking to? I assume it means something to you, but it doesn't even look like you're trying to communicate your point in a clear manner. Just that you enjoy using words. When trying to read around those technical terms, what is left has a rather nasty and arrogant tone. Therefore, doesn't add much to the discussion = downvote.
skgoa 9y ago
The main issue with this being that markets don't exhibit behaviour analogous to Brownian motion. It's a nice little assumption that makes the math work, but frequency analysis of real world markets shows that they behave like pink noise instead. And unsurprisingly there are quant funds and prop trading firms that use this very fact to make lots of money. Academic economist's love for the normal distribution is derided by pretty much any fellow real world trader I know.
BooglyWoo 9y ago
Can't recommend the Mandelbrot book highly enough - he was a student of Paul Levy, and he wrote extensively about why Gaussian is not a good choice to model financial time series.

lr4444lr 9y ago

I don't mean it to be cynical, it's just my experience that a lot of data gathering in the social sciences (my experience was in education) makes terrible assumptions, often implicitly, the worst being that the variables are IID. And it's very obvious from the publication that the number crunching is some cargo-cult approach they probably learned from their mandated semester or two in research methods. (Okay, that last bit was cynical!)

lausiant 9y ago
I could probably be characterized as a social scientist, at least a behavioral scientist. What you're saying is probably part of it, although in my experience that criticism can be leveled as much, if not more, at wet-lab-type biologists who eschew all but the most minimal stats. With the social sciences, though, there's another phenomenon at play, which is that the phenomena are so abstract often that there's not really a good theoretical reason to assume anything in particular. And if that's the case, because the normal is the entropy-maximizing distribution, you're actually better off assuming that rather than some other distribution. You could also use nonparametric stats, but that has its own advantages and disadvantages. Bias-variance dilemma and all that. The truth is, it's hard to beat the normal even when it's wrong. And if you subscribe to the inferential philosophy that every model is wrong, you're better off being conservatively wrong, which implies a normal. I'm not saying everything should be assumed to be normal. But unless things are (1) obviously super non-normal, or (2) you have some very strongly justified model that produces a non-normal distribution, you're probably best off using a normal if you're going to go parametric. And I think those two conditions are met much more often than we like to admit. The normal distribution is kind of over-maligned, I think. I started my stats career being enamoured of rigorously nonparametric stats, and still am (esp. exact tests, bootstrapping/permutation-based inference, and empirical likelihood), but have grown to strongly appreciate normal distributions (or whatever maxent distribution is appropriate).
- CuriouslyC 9y ago
  From my perspective, either you have data and you have an idea what sort of distributions you're working with, or you don't, and you should fix that problem first rather than going down the theory rabbit hole. With data in hand, a skew/kurtosis scatter plot is a good way to gauge the higher dimensional distribution of your data. Another option is to cluster the variables of the data set using something like HDBSCAN and color the plot points based on cluster membership. If you have to go guessing distributions without evidence, you're better off choosing a low k student's T distribution (for robustness to outliers) or a gamma distribution (if you think your data might be skewed).

lmm 9y ago

Disagree; economists have failed more visibly, but they've got nothing on social scientists.