5 ms·

The time series spanning 3 orders of magnitude over a course if years is not sufficient condition for Benford's law to apply. As a trivial example, a stock whos

The time series spanning 3 orders of magnitude over a course if years is not sufficient condition for Benford's law to apply. As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. A formal justification for applying Benford's law to a time series like that would depend on some kind of ergodicity argument which has not been made either by Gelman or the original author of the post he's talking about. I'm rather skeptical. As a first step into deciding whether it's reasonable I would do something like the plot for Berkshire Hathaway for each of a large collection of companies, and collect some measure of disagreement between the model (Benford's law) and the observed distribution for each stock. I'm guessing many of these stocks would follow Benford's law no better than Bitcoin.

TomSwirly 5y ago
> As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. You would expect if you deliberately created a security whose growth rate monotonically tends to zero over time that you would get anomalous results in any statistical point, but I don't quite see why you bring up this pathological example?
- FabHK 5y ago
  If it were to grow by 9 bp (0.09%) every day, it would obey Benford's law. Question is how pathological that would be, but yes, arguably less.
- CrazyStat 5y ago
  Because the claim was made that spanning several orders of magnitude is sufficient to allow application of Benford's law: > The big one you need to look for is that the variable spans multiple orders of magnitude. > Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. > BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. This claim is false. I provided a counterexample to demonstrate as much. Whether or not it's "pathological" is both subjective and irrelevant.
mumblemumble 5y ago
> As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. That's actually a great illustration of the kind of thing we'd expect Benford's Law to raise red flags about. Your hypothetical posits the kind of behavior that we would not expect to naturally come out of the kind of process that governs stock prices over time. Stocks simply don't have their price go up by exactly one dollar a day, every day, for thirty years. (This is briefly covered in the the article, which describes how the other big assumption of Benford's Law is that you're working with a system where things tend to change by percentages rather than absolute values. So yeah, it's true, covering orders of magnitude is not sufficient. Nor did I claim that it is, mind. I was just pointing out the assumption that I thought he original poster was missing.) So, if we saw the kinds of initial digits that your hypothetical produces, we would be correct to guess that this stock is behaving in an unusual way. And it would be a correct application of Benford's Law, because we can reasonably expect that its key assumptions apply. By contrast, if we hypothesize a stock whose price goes up by exactly 0.0001% every day for 30 years, its record of daily closing prices would obey Benford's Law. In fact, it would match it so well that we would also (correctly) consider it to be suspicious. No, it's not a formal justification in any case. It is just a red flag. It turns out that red flags are useful even when they're not admissible in court, because they provide an easy heuristic that you can use to help direct your search for more compelling evidence.
- CrazyStat 5y ago
  > So, if we saw the kinds of initial digits that your hypothetical produces, we would be correct to guess that this stock is behaving in an unusual way. Sure > And it would be a correct application of Benford's Law, because we can reasonably expect that its key assumptions apply. No, absolutely not. The fact that Benford's law would trigger on that one example is not evidence that using Benford's law in this situation is actually a reasonable thing to do. Applying Benford's law to observations over time from a highly correlated time series (e.g. a stock price over time) is fundamentally different from applying Benford's law to a bunch of observations from different time series at a single point in time (e.g. the NYSE prices in the first figure from the original post). As I pointed out in my comment above, applying Benford's law to the single time series must be justified by an ergodicity argument which nobody has attempted to make. I don't know if Smith has any background in stochastic processes, but given Gelman's work on MCMC convergence (which deals with similar issues) he should have flagged that. > It turns out that red flags are useful even when they're not admissible in court, because they provide an easy heuristic that you can use to help direct your search for more compelling evidence. No. A "red flag" like this with no theoretical justification and unknown operating characteristics is a good way to get led down false paths. About the only thing it's good for is confirmation bias.
HWR_14 5y ago
> A formal justification for applying Benford's law to a time series like that would depend on some kind of ergodicity argument I don't think Benford's law relies on ergodicity at all. I thought it applied to all cases of exponential growth.

TomSwirly 5y ago

> As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. You would expect if you deliberately created a security whose growth rate monotonically tends to zero over time that you would get anomalous results in any statistical point, but I don't quite see why you bring up this pathological example?

FabHK 5y ago
If it were to grow by 9 bp (0.09%) every day, it would obey Benford's law. Question is how pathological that would be, but yes, arguably less.
CrazyStat 5y ago
Because the claim was made that spanning several orders of magnitude is sufficient to allow application of Benford's law: > The big one you need to look for is that the variable spans multiple orders of magnitude. > Berkshire Hathaway going back to 1980 spans three orders of magnitude, so we're good there. > BTC going back 7 years is also decent. It's not quite as good - only spanning two orders of magnitude - but it's enough that we would expect Benford's Law to apply better than it does in this graph. This claim is false. I provided a counterexample to demonstrate as much. Whether or not it's "pathological" is both subjective and irrelevant.

mumblemumble 5y ago

> As a trivial example, a stock whose price started at $1 and increased by $1 every day for 30 years would span 4 orders of magnitude over that time period, but would not follow Benford's law. That's actually a great illustration of the kind of thing we'd expect Benford's Law to raise red flags about. Your hypothetical posits the kind of behavior that we would not expect to naturally come out of the kind of process that governs stock prices over time. Stocks simply don't have their price go up by exactly one dollar a day, every day, for thirty years. (This is briefly covered in the the article, which describes how the other big assumption of Benford's Law is that you're working with a system where things tend to change by percentages rather than absolute values. So yeah, it's true, covering orders of magnitude is not sufficient. Nor did I claim that it is, mind. I was just pointing out the assumption that I thought he original poster was missing.) So, if we saw the kinds of initial digits that your hypothetical produces, we would be correct to guess that this stock is behaving in an unusual way. And it would be a correct application of Benford's Law, because we can reasonably expect that its key assumptions apply. By contrast, if we hypothesize a stock whose price goes up by exactly 0.0001% every day for 30 years, its record of daily closing prices would obey Benford's Law. In fact, it would match it so well that we would also (correctly) consider it to be suspicious. No, it's not a formal justification in any case. It is just a red flag. It turns out that red flags are useful even when they're not admissible in court, because they provide an easy heuristic that you can use to help direct your search for more compelling evidence.

CrazyStat 5y ago
> So, if we saw the kinds of initial digits that your hypothetical produces, we would be correct to guess that this stock is behaving in an unusual way. Sure > And it would be a correct application of Benford's Law, because we can reasonably expect that its key assumptions apply. No, absolutely not. The fact that Benford's law would trigger on that one example is not evidence that using Benford's law in this situation is actually a reasonable thing to do. Applying Benford's law to observations over time from a highly correlated time series (e.g. a stock price over time) is fundamentally different from applying Benford's law to a bunch of observations from different time series at a single point in time (e.g. the NYSE prices in the first figure from the original post). As I pointed out in my comment above, applying Benford's law to the single time series must be justified by an ergodicity argument which nobody has attempted to make. I don't know if Smith has any background in stochastic processes, but given Gelman's work on MCMC convergence (which deals with similar issues) he should have flagged that. > It turns out that red flags are useful even when they're not admissible in court, because they provide an easy heuristic that you can use to help direct your search for more compelling evidence. No. A "red flag" like this with no theoretical justification and unknown operating characteristics is a good way to get led down false paths. About the only thing it's good for is confirmation bias.

HWR_14 5y ago

> A formal justification for applying Benford's law to a time series like that would depend on some kind of ergodicity argument I don't think Benford's law relies on ergodicity at all. I thought it applied to all cases of exponential growth.