4 ms·

The author at statsmodeling.stat.columbia.edu is the person who wrote several of the books on things like this. I don't think he can be finger-wagged away quite

The author at statsmodeling.stat.columbia.edu is the person who wrote several of the books on things like this. I don't think he can be finger-wagged away quite so easily as that. There's a huge difference between this and the election one, which is that this one is working with data that satisfies the key statistical assumptions needed to properly apply Benford's Law, and the QAnon elections rigged claims didn't. (Or at least the ones I know about offhand - I don't use Benford's Law at work, so this is not really my area of expertise.) The big one you need to look for is that the variable spans multiple orders of magnitude. Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. Note that we could not use Benford's Law to look at BRK going back only 7 years, but that is not because it's not enough data. It's because that would cover only 1/4 of an order of magnitude. BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. By contrast, a lot of the cases where people thought they were using Benford's Law to uncover electoral fraud, the data only covered a small fraction of an order of magnitude. It's just silly to expect 1 to be the most common leading digit in a set of data that's constrained to the range [300, 600]. I don't know that this is compelling evidence of fraud or manipulation in BTC, but it at least seems like clear evidence that BTC doesn't behave like other securities. There's something unusual going on that merits further investigation. Perhaps it is just a shortcoming in the statistical model being used. But, since this isn't a big data conference, it's not sufficient to just keep hitting the "need more data" number until you finally get the number you want.

alisonkisk 5y ago
I don't think you are correctly using "orders of magnitude" in benford's law in this analysis. The orders of magnitude are in percent change over a few days, which is similar for both BTC and BRK.
CrazyStat 5y ago
The time series spanning 3 orders of magnitude over a course if years is not sufficient condition for Benford's law to apply. As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. A formal justification for applying Benford's law to a time series like that would depend on some kind of ergodicity argument which has not been made either by Gelman or the original author of the post he's talking about. I'm rather skeptical. As a first step into deciding whether it's reasonable I would do something like the plot for Berkshire Hathaway for each of a large collection of companies, and collect some measure of disagreement between the model (Benford's law) and the observed distribution for each stock. I'm guessing many of these stocks would follow Benford's law no better than Bitcoin.
- TomSwirly 5y ago
  > As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. You would expect if you deliberately created a security whose growth rate monotonically tends to zero over time that you would get anomalous results in any statistical point, but I don't quite see why you bring up this pathological example?
  - FabHK 5y ago
    If it were to grow by 9 bp (0.09%) every day, it would obey Benford's law. Question is how pathological that would be, but yes, arguably less.
  - CrazyStat 5y ago
    Because the claim was made that spanning several orders of magnitude is sufficient to allow application of Benford's law: > The big one you need to look for is that the variable spans multiple orders of magnitude. > Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. > BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. This claim is false. I provided a counterexample to demonstrate as much. Whether or not it's "pathological" is both subjective and irrelevant.
- mumblemumble 5y ago
  > As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. That's actually a great illustration of the kind of thing we'd expect Benford's Law to raise red flags about. Your hypothetical posits the kind of behavior that we would not expect to naturally come out of the kind of process that governs stock prices over time. Stocks simply don't have their price go up by exactly one dollar a day, every day, for thirty years. (This is briefly covered in the the article, which describes how the other big assumption of Benford's Law is that you're working with a system where things tend to change by percentages rather than absolute values. So yeah, it's true, covering orders of magnitude is not sufficient. Nor did I claim that it is, mind. I was just pointing out the assumption that I thought he original poster was missing.) So, if we saw the kinds of initial digits that your hypothetical produces, we would be correct to guess that this stock is behaving in an unusual way. And it would be a correct application of Benford's Law, because we can reasonably expect that its key assumptions apply. By contrast, if we hypothesize a stock whose price goes up by exactly 0.0001% every day for 30 years, its record of daily closing prices would obey Benford's Law. In fact, it would match it so well that we would also (correctly) consider it to be suspicious. No, it's not a formal justification in any case. It is just a red flag. It turns out that red flags are useful even when they're not admissible in court, because they provide an easy heuristic that you can use to help direct your search for more compelling evidence.
deleted 5y ago
[deleted]

alisonkisk 5y ago

I don't think you are correctly using "orders of magnitude" in benford's law in this analysis. The orders of magnitude are in percent change over a few days, which is similar for both BTC and BRK.

CrazyStat 5y ago

The time series spanning 3 orders of magnitude over a course if years is not sufficient condition for Benford's law to apply. As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. A formal justification for applying Benford's law to a time series like that would depend on some kind of ergodicity argument which has not been made either by Gelman or the original author of the post he's talking about. I'm rather skeptical. As a first step into deciding whether it's reasonable I would do something like the plot for Berkshire Hathaway for each of a large collection of companies, and collect some measure of disagreement between the model (Benford's law) and the observed distribution for each stock. I'm guessing many of these stocks would follow Benford's law no better than Bitcoin.

TomSwirly 5y ago
> As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. You would expect if you deliberately created a security whose growth rate monotonically tends to zero over time that you would get anomalous results in any statistical point, but I don't quite see why you bring up this pathological example?
- FabHK 5y ago
  If it were to grow by 9 bp (0.09%) every day, it would obey Benford's law. Question is how pathological that would be, but yes, arguably less.
- CrazyStat 5y ago
  Because the claim was made that spanning several orders of magnitude is sufficient to allow application of Benford's law: > The big one you need to look for is that the variable spans multiple orders of magnitude. > Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. > BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. This claim is false. I provided a counterexample to demonstrate as much. Whether or not it's "pathological" is both subjective and irrelevant.
mumblemumble 5y ago
> As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. That's actually a great illustration of the kind of thing we'd expect Benford's Law to raise red flags about. Your hypothetical posits the kind of behavior that we would not expect to naturally come out of the kind of process that governs stock prices over time. Stocks simply don't have their price go up by exactly one dollar a day, every day, for thirty years. (This is briefly covered in the the article, which describes how the other big assumption of Benford's Law is that you're working with a system where things tend to change by percentages rather than absolute values. So yeah, it's true, covering orders of magnitude is not sufficient. Nor did I claim that it is, mind. I was just pointing out the assumption that I thought he original poster was missing.) So, if we saw the kinds of initial digits that your hypothetical produces, we would be correct to guess that this stock is behaving in an unusual way. And it would be a correct application of Benford's Law, because we can reasonably expect that its key assumptions apply. By contrast, if we hypothesize a stock whose price goes up by exactly 0.0001% every day for 30 years, its record of daily closing prices would obey Benford's Law. In fact, it would match it so well that we would also (correctly) consider it to be suspicious. No, it's not a formal justification in any case. It is just a red flag. It turns out that red flags are useful even when they're not admissible in court, because they provide an easy heuristic that you can use to help direct your search for more compelling evidence.

deleted 5y ago

[deleted]