4 ms·
Bootstrap resampling is such a black magic thing
by ThouYS 5y ago
Bootstrap resampling is such a black magic thing
- CrazyStat 5y agoOne way to approach the bootstrap is as sampling from the posterior mean of a Dirichlet Process model with a noninformative prior (alpha=0).
- derbOac 5y agoIt's just Monte Carlo simulation using the observed distribution as the population distribution.
- civilized 5y agoIt's not the mere description of the procedure that people find mysterious.
- btown 5y agoThis has big "A monad is just a monoid in the category of endofunctors" energy
- grayclhn 5y ago1) It’s not — there are lots of procedures called “the bootstrap” that act differently. 2) The fact that “substitute the data for the population distribution” both works and is sometimes provably better than other more sensible approaches is a little mind blowing. Most things called the bootstrap feel like cheating, ie “this part seems hard, let’s do the easiest thing possible instead and hope it works.”
- derbOac 5y agoI think it's brilliant as an idea but not particularly mysterious after the fact or something. I love it, but think it's brilliant in part because it's so transparent and simple. I agree "bootstrap" has expanded a bit in meaning but I think it's basically the same idea. I used to think it was cheating but have realized there is a cost to it, which is replication. For many things it's just impractical in terms of computation time. So although it's great, it requires a lot.
- swyx 5y agohow does it deal with tail risk that hasn't been observed? i assume not well?
- spekcular 5y agoYes. It only tells you about variability within the sampled values. If you don't sample outliers, or get unlucky and same many non-representative vvalues, it can't tell you what you're missing out on (obviously).
- disgruntledphd2 5y agoYeah, like your bootstrap estimate of max or min will be pretty crap. Works well for central tendencies though.
- graycat 5y agoThere is a nice treatment of resampling, i.e., permutation tests, in (from my TeX format bibliography) Sidney Siegel, {\it Nonparametric Statistics for the Behavioral Sciences,\/} McGraw-Hill, New York, 1956.\ \ Right, there was already a good book on such tests over 50 years ago. Can also justify it with an independence, identically distributed assumption. But a weaker assumption of exchangeability can also work -- I published a paper with that. The broad idea of such a statistical hypothesis test is to decide on the null hypothesis, null as in no effect (if looking for an effect, then want to reject the null hypothesis of no effect) and to make assumptions to permit calculating the probability of what you observe. If that probability is way too small then reject the null hypothesis and conclude that there was an effect. Right, it's fishy.
- jll29 5y agoThe 2nd edition is never far from my desk: Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.) New York: McGraw-Hill.