3 ms·
The more data you have, the more creative we can get with it. Gather enough statistics, and you're bound to find something that backs your theory. Traditional
by unabst 11y ago
The more data you have, the more creative we can get with it. Gather enough statistics, and you're bound to find something that backs your theory.
Traditional intuition insists this isn't suppose to happen. But that's why statistics isn't physics. Data does not have to be botched or erroneous to get creative. It can all be true, and the backing of your theory may also be valid. The issue is whether your theory itself holds any weight or precedence in light of all other possible theories. So when taking big data and statistics into account, "all possible theories" is the big data picture, not any specific theory. Searching for one theory is already misguided, because information chaos/noise mounts with scale, as other data scientists will consistently tell you.
But if we consider theories as abstractions of evidence, then this should all make intuitive sense. A shitload of theories should emerge from a shitload of evidence.