4 ms·

What is the statistical likelihood of this appearing by chance? Note that 2014 to today isn't actually that long of a timeframe, as prices in a time-series are

by dannyw 5y ago

What is the statistical likelihood of this appearing by chance? Note that 2014 to today isn't actually that long of a timeframe, as prices in a time-series are highly linked to the previous data point. Berkshire Hathaway has been trading for several decades; so that's 40 years of data; as compared to 7 years of data. yet the author, by using the same charts, seems to falsely imply that these are remotely comparable. This article seems to have as much logic as the whole "Benford's law proves the elections were rigged!!!" claims by QAnon, who tried to apply Benford's Law on a county level, found that it generally fits a vast majority of the time, but cherry-picked the counties where it didn't fit (which generally tended to be smaller and hence less data points). Well, duh, it's statistics: outliers are to be expected. Pretty poor take from an author at "statmodeling.stat.columbia.edu".

conformist 5y ago
Isn't Andrew Gelman's take more like "Look what they did, that's kind of interesting, what do you think?" rather than "look at this article, it is true"? See also the comment section.
- FabHK 5y ago
  Yes indeed. Most of the article is quoted from Smith, and Gelman (the prolific author of the worthwhile blog at columbia.edu) only really adds a few inconclusive sentences at the end. ("... There’s a lot going on, and people have lots of reasons for doing things.")
mumblemumble 5y ago
The author at statsmodeling.stat.columbia.edu is the person who wrote several of the books on things like this. I don't think he can be finger-wagged away quite so easily as that. There's a huge difference between this and the election one, which is that this one is working with data that satisfies the key statistical assumptions needed to properly apply Benford's Law, and the QAnon elections rigged claims didn't. (Or at least the ones I know about offhand - I don't use Benford's Law at work, so this is not really my area of expertise.) The big one you need to look for is that the variable spans multiple orders of magnitude. Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. Note that we could not use Benford's Law to look at BRK going back only 7 years, but that is not because it's not enough data. It's because that would cover only 1/4 of an order of magnitude. BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. By contrast, a lot of the cases where people thought they were using Benford's Law to uncover electoral fraud, the data only covered a small fraction of an order of magnitude. It's just silly to expect 1 to be the most common leading digit in a set of data that's constrained to the range [300, 600]. I don't know that this is compelling evidence of fraud or manipulation in BTC, but it at least seems like clear evidence that BTC doesn't behave like other securities. There's something unusual going on that merits further investigation. Perhaps it is just a shortcoming in the statistical model being used. But, since this isn't a big data conference, it's not sufficient to just keep hitting the "need more data" number until you finally get the number you want.
- alisonkisk 5y ago
  I don't think you are correctly using "orders of magnitude" in benford's law in this analysis. The orders of magnitude are in percent change over a few days, which is similar for both BTC and BRK.
- CrazyStat 5y ago
  The time series spanning 3 orders of magnitude over a course if years is not sufficient condition for Benford's law to apply. As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. A formal justification for applying Benford's law to a time series like that would depend on some kind of ergodicity argument which has not been made either by Gelman or the original author of the post he's talking about. I'm rather skeptical. As a first step into deciding whether it's reasonable I would do something like the plot for Berkshire Hathaway for each of a large collection of companies, and collect some measure of disagreement between the model (Benford's law) and the observed distribution for each stock. I'm guessing many of these stocks would follow Benford's law no better than Bitcoin.
  - TomSwirly 5y ago
    > As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. You would expect if you deliberately created a security whose growth rate monotonically tends to zero over time that you would get anomalous results in any statistical point, but I don't quite see why you bring up this pathological example?
    - FabHK 5y ago
      If it were to grow by 9 bp (0.09%) every day, it would obey Benford's law. Question is how pathological that would be, but yes, arguably less.
    - CrazyStat 5y ago
      Because the claim was made that spanning several orders of magnitude is sufficient to allow application of Benford's law: > The big one you need to look for is that the variable spans multiple orders of magnitude. > Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. > BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. This claim is false. I provided a counterexample to demonstrate as much. Whether or not it's "pathological" is both subjective and irrelevant.
alisonkisk 5y ago
It always amazes me how people will confidently broadcast their arrogant ignorance in comments like like this.
contravariant 5y ago
Benford's law doesn't give much of a guarantee except in highly specific scenarios. You can however show it's pretty accurate if you've got a 'smooth' probability distribution spanning multiple orders of magnitude (how much of a guarantee you can give depends on how specific you define what it means to be 'smooth'). The rigged elections example you gave was, I believe, mostly explained by the fact that the number of votes they were looking at were all roughly the same order of magnitude [1]. [1]: https://www.youtube.com/watch?v=etx0k1nLn78 https://www.youtube.com/watch?v=etx0k1nLn78
tedunangst 5y ago
Like what does the graph look like for TSLA? Or GOOG?
- CrazyStat 5y ago
  TSLA: https://i.imgur.com/hZVaJG8.png https://i.imgur.com/hZVaJG8.png GOOG: https://i.imgur.com/Wz0am68.png https://i.imgur.com/Wz0am68.png MSFT: https://i.imgur.com/gEYgIN3.png https://i.imgur.com/gEYgIN3.png N.B. these are split-adjusted.
TomSwirly 5y ago
> prices in a time-series are highly linked to the previous data point. And how many data points are there in seven years of BTC? > seems to falsely imply that these are remotely comparable. Why is it "false" that they are "remotely comparable" if we are looking at time series of security prices? You are long on the insults and short on the reasoning. Given the author is a well-known statistician and you are some anonymous individual, I'm going with them.
FabHK 5y ago
Given that Benford's law tends to apply when the numbers span many orders of magnitudes, I would have expected more emphasis on that. Berkshire Hathaway (low 200, high 450,000) covers about 3.4 orders of magnitude (and much more time, ie opportunities to wander up and down), BTC (low 200, high 60,000) covers about about 2.5 orders of magnitude. So, we should expect BTC to have a worse fit than Berkshire. How much worse would require some more sophisticated analysis than looking at the graphs, it seems.
lend000 5y ago
The market is simply too young compared to the "baseline," and he's only testing one ticker. This might make more sense, even with the limited age of data around today, if he took the 20 or so most prominent cryptocurrencies to have a better sample size, since they are all correlated anyways. The progression of charts correlating with Benford is exactly what you would expect if the titles "NYSE, Berkshire Hathaway, Bitcoin" were replaced with "2800 tickers over 40 years, 1 ticker over 40 years, 1 ticker over 7 years."

by dannyw 5y ago

conformist 5y ago

Isn't Andrew Gelman's take more like "Look what they did, that's kind of interesting, what do you think?" rather than "look at this article, it is true"? See also the comment section.

FabHK 5y ago
Yes indeed. Most of the article is quoted from Smith, and Gelman (the prolific author of the worthwhile blog at columbia.edu) only really adds a few inconclusive sentences at the end. ("... There’s a lot going on, and people have lots of reasons for doing things.")

mumblemumble 5y ago

The author at statsmodeling.stat.columbia.edu is the person who wrote several of the books on things like this. I don't think he can be finger-wagged away quite so easily as that. There's a huge difference between this and the election one, which is that this one is working with data that satisfies the key statistical assumptions needed to properly apply Benford's Law, and the QAnon elections rigged claims didn't. (Or at least the ones I know about offhand - I don't use Benford's Law at work, so this is not really my area of expertise.) The big one you need to look for is that the variable spans multiple orders of magnitude. Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. Note that we could not use Benford's Law to look at BRK going back only 7 years, but that is not because it's not enough data. It's because that would cover only 1/4 of an order of magnitude. BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. By contrast, a lot of the cases where people thought they were using Benford's Law to uncover electoral fraud, the data only covered a small fraction of an order of magnitude. It's just silly to expect 1 to be the most common leading digit in a set of data that's constrained to the range [300, 600]. I don't know that this is compelling evidence of fraud or manipulation in BTC, but it at least seems like clear evidence that BTC doesn't behave like other securities. There's something unusual going on that merits further investigation. Perhaps it is just a shortcoming in the statistical model being used. But, since this isn't a big data conference, it's not sufficient to just keep hitting the "need more data" number until you finally get the number you want.

alisonkisk 5y ago
I don't think you are correctly using "orders of magnitude" in benford's law in this analysis. The orders of magnitude are in percent change over a few days, which is similar for both BTC and BRK.
CrazyStat 5y ago
The time series spanning 3 orders of magnitude over a course if years is not sufficient condition for Benford's law to apply. As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. A formal justification for applying Benford's law to a time series like that would depend on some kind of ergodicity argument which has not been made either by Gelman or the original author of the post he's talking about. I'm rather skeptical. As a first step into deciding whether it's reasonable I would do something like the plot for Berkshire Hathaway for each of a large collection of companies, and collect some measure of disagreement between the model (Benford's law) and the observed distribution for each stock. I'm guessing many of these stocks would follow Benford's law no better than Bitcoin.
- TomSwirly 5y ago
  > As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. You would expect if you deliberately created a security whose growth rate monotonically tends to zero over time that you would get anomalous results in any statistical point, but I don't quite see why you bring up this pathological example?
  - FabHK 5y ago
    If it were to grow by 9 bp (0.09%) every day, it would obey Benford's law. Question is how pathological that would be, but yes, arguably less.
  - CrazyStat 5y ago
    Because the claim was made that spanning several orders of magnitude is sufficient to allow application of Benford's law: > The big one you need to look for is that the variable spans multiple orders of magnitude. > Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. > BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. This claim is false. I provided a counterexample to demonstrate as much. Whether or not it's "pathological" is both subjective and irrelevant.

alisonkisk 5y ago

It always amazes me how people will confidently broadcast their arrogant ignorance in comments like like this.

contravariant 5y ago

Benford's law doesn't give much of a guarantee except in highly specific scenarios. You can however show it's pretty accurate if you've got a 'smooth' probability distribution spanning multiple orders of magnitude (how much of a guarantee you can give depends on how specific you define what it means to be 'smooth'). The rigged elections example you gave was, I believe, mostly explained by the fact that the number of votes they were looking at were all roughly the same order of magnitude [1]. [1]: https://www.youtube.com/watch?v=etx0k1nLn78 https://www.youtube.com/watch?v=etx0k1nLn78

tedunangst 5y ago

Like what does the graph look like for TSLA? Or GOOG?

CrazyStat 5y ago
TSLA: https://i.imgur.com/hZVaJG8.png https://i.imgur.com/hZVaJG8.png GOOG: https://i.imgur.com/Wz0am68.png https://i.imgur.com/Wz0am68.png MSFT: https://i.imgur.com/gEYgIN3.png https://i.imgur.com/gEYgIN3.png N.B. these are split-adjusted.

TomSwirly 5y ago

> prices in a time-series are highly linked to the previous data point. And how many data points are there in seven years of BTC? > seems to falsely imply that these are remotely comparable. Why is it "false" that they are "remotely comparable" if we are looking at time series of security prices? You are long on the insults and short on the reasoning. Given the author is a well-known statistician and you are some anonymous individual, I'm going with them.

FabHK 5y ago

Given that Benford's law tends to apply when the numbers span many orders of magnitudes, I would have expected more emphasis on that. Berkshire Hathaway (low 200, high 450,000) covers about 3.4 orders of magnitude (and much more time, ie opportunities to wander up and down), BTC (low 200, high 60,000) covers about about 2.5 orders of magnitude. So, we should expect BTC to have a worse fit than Berkshire. How much worse would require some more sophisticated analysis than looking at the graphs, it seems.

lend000 5y ago

The market is simply too young compared to the "baseline," and he's only testing one ticker. This might make more sense, even with the limited age of data around today, if he took the 20 or so most prominent cryptocurrencies to have a better sample size, since they are all correlated anyways. The progression of charts correlating with Benford is exactly what you would expect if the titles "NYSE, Berkshire Hathaway, Bitcoin" were replaced with "2800 tickers over 40 years, 1 ticker over 40 years, 1 ticker over 7 years."