3 ms·
> One thing that troubles me in the paper is that the researchers appear to have gone looking for precursor patterns in an ad hoc way, with no physical theory i
by bryan0 3y ago
> One thing that troubles me in the paper is that the researchers appear to have gone looking for precursor patterns in an ad hoc way, with no physical theory in mind, just trying different binning techniques and delays until they got a signal.
I believe in particle physics this is known as the “look-elsewhere effect”. Basically if look long and hard enough for a pattern, you will eventual find it if your parameter space is large enough: https://en.m.wikipedia.org/wiki/Look-elsewhere_effect https://en.m.wikipedia.org/wiki/Look-elsewhere_effect
- GolDDranks 3y agoThe obligatory XKCD: https://xkcd.com/882/ https://xkcd.com/882/
- jaggederest 3y agoThis is related to the fact that in a higher-dimensional space the hamming distance between two points is compact, compared to 2d/3d intuition. This problem is related to the way that e.g. embedding text into an AI for comparison purposes often produces surprisingly closely related vector distances for relatively unrelated strings.
- Ringz 3y agoCan we transfer this to the financial markets?
- madrox 3y agoWhen I was taught statistics, this was regularly brought up as a big no-no in science. However, I read a guide to practical statistics that had a gem about predicting the stock market. If we discovered surges in the market correlated to newspaper sales, we wouldn't discard this as look-elsewhere. In fact, we'd follow newspaper sales very closely. Predicting earthquakes has a big upside for humanity. If there is even a small correlation -- even if we don't yet understand it -- we can benefit from it.
- rcxdude 3y agoIt can be useful, it's just you need to keep monitoring it: if the effect seems to shrink or disappear on future tests, it's probably spurious.
- breck 3y agoIn GWAS studies there's a nice visualization for dealing with this called Manhattan Plots. https://en.wikipedia.org/wiki/Manhattan_plot https://en.wikipedia.org/wiki/Manhattan_plot Basically if you test a lot of hypotheses plot all the p-values and look to see if there is something that is a true outlier or whether it is expected given so many tests.
- mnky9800n 3y agoMost of particle physics is people searching for jobs that give meaning to their life. This often biases them against finding results against meaning.