3 ms·
That gif is wildly intriguing to me. Probability theory is the bread and butter of mainstream science. Have people been doing it wrong this whole time? I wan
by macawfish 9y ago
That gif is wildly intriguing to me. Probability theory is the bread and butter of mainstream science. Have people been doing it wrong this whole time?
I want to dig deeper into this but two things come to mind:
1. George Sugihara's and Floris Takens' works on quantifying causation in non-linear systems. Correlation doesn't imply causation, so why not look at causation? Here's a writeup on it: https://www.quantamagazine.org/chaos-theory-in-ecology-predicts-future-populations-20151013/ https://www.quantamagazine.org/chaos-theory-in-ecology-predi...
2. Andrei Khrennikov argues that quantum models can be expressed with Kolmogorovian (classical) probability theory. He points out that many probability theorists have glossed over an entire subtlety of Kolmogorov's work: that probability spaces need not be finite dimensional, that they can be subspaces of higher dimensional, even infinite dimensional probability spaces. Khrennikov's philosophical convictions are not widely shared by quantum theorists, but his math is solid (he seems to be in pursuit of quantum theories with "hidden variables", like Einstein was looking for). http://www.sciencedirect.com/science/article/pii/S0375960103011964 http://www.sciencedirect.com/science/article/pii/S0375960103...
Even in the gif shown in this article, you might suppose that each point set could be modeled in higher dimensional terms, for example adding some kind of discrete curl-like measure. But even without going there: something as simple as a changing to a polar basis should reveal different statistics.
And isn't this kind of what deep learning is all about? Going between higher and lower dimensional representations?
- Invictus0 9y agoI'm not sure why that gif is so interesting to you; that wasn't the reason I submitted this piece. Average and standard deviation are hardly useful data points in evaluating a dataset, and also don't have anything to do with probability theory. The second half of the post contains the bulk of its interesting content (as I read it).
- macawfish 9y agoMaybe I'm missing something? To me, that gif illustrates sets of samples that are on the surface compatible with the the same Gaussian distribution, even if a Gaussian distribution is "obviously" lacking.
- wenc 9y agoI think 0xBABAD00C's comment is on the money. The gif is almost a non-sequitur and does not seem to support the conclusion the article is trying to reach. The second part of the article makes a claim that is worth discussing -- that probabilistic inference does not work well in nonlinear domains. I think this is true in one sense, but in many practical situations we can usually to get something to work well enough for it to be useful. Assuming we can (approximately) split up the domain into multiple regions and get probability distributions for each region (with transition functions between them), we can continue to make inferences under nonlinearity. That said, this is a concession. Perhaps there are techniques in deep learning that more general in nature.
- deltux 9y agoFor those who want the source of the gif : https://www.autodeskresearch.com/publications/samestats https://www.autodeskresearch.com/publications/samestats The paper is actually really interesting, as it provides a useful insight into what summary statistics mean and the importance of data visualization.
- wenc 9y agoI'm a bit slow today but I'm not sure I got what the GIF was about. It is well known that the mean (1st moment) and stdev/variance (2nd moment) are insufficient to uniquely describe an arbitrary probability distribution except in the case of the normal distribution. Therefore it shouldn't be surprising that that you can find probability distributions that share the first two moments, but differ wildly in the higher moments (skewness, kurtosis) or even the number of modes (bimodal, trimodal, etc.). Not everything in the world is Gaussian, but we knew that.