3 ms·

My random stream-of-consciousness reference reactions ... Tegmark & Lin's discussion of how deep learning maps onto the physical world: Why does deep and cheap

My random stream-of-consciousness reference reactions ... Tegmark & Lin's discussion of how deep learning maps onto the physical world: Why does deep and cheap learning work so well? [1][2] The work of Aerts et al. more than 10 years ago, including how vector space models of human categorization show QM structure [3]. Lucien Hardy's exquisite teasing out of the difference between classical and quantum probability: Quantum Theory From Five Reasonable Axioms [4] It turns out that the innovation, power and strangeness of QM is related to separating physical processes into: 1. Linear continuous unitary reversible evolution relying on complex amplitudes (wavefunction propagation). 2. Non-linear discontinuous irreversible 'collapse' or 'measurement' or 'entanglement' or 'correlated branching', with probabilities based on the Born Rule (real values from the square of wave function amplitudes). Neural nets also decompose a problem into successive alternating layers of reversible continuous functions and discrete irreversible categorical decisions (e.g. softmax/sigmoid logistic classifiers). An even more obscure tangent is that most of the financial market is now constructed from 'options', which track continuous values with a continuous payoff, but offer a choice of execution to collapse the option and create a real financial result, e.g. see N.N.Taleb's dicussion of optionality [5]. [1] http://arxiv.org/pdf/1608.08225.pdf http://arxiv.org/pdf/1608.08225.pdf [2] http://www.youtube.com/watch?v=5MdSE-N0bxs http://www.youtube.com/watch?v=5MdSE-N0bxs [3] http://en.wikipedia.org/wiki/Diederik_Aerts#Quantum_structure_in_psychology http://en.wikipedia.org/wiki/Diederik_Aerts#Quantum_structur... [4] http://arxiv.org/abs/quant-ph/0101012 http://arxiv.org/abs/quant-ph/0101012 [5] http://fooledbyrandomness.com/ConvexityScience.pdf http://fooledbyrandomness.com/ConvexityScience.pdf

cs702 9y ago
Thank you for posting this. Similar thoughts have been percolating in my stream-of-consciousness for a while, especially after coming across an earlier version of Tegmark & Lin's paper some months ago. My take is that deep neural nets work so well in practice because not all distributions of natural data are equally likely (and therefore, the no-free-lunch theorem(s), which assumes all distributions are equally likely, doesn't hold in the real world!); and that, in turn, is because the distribution of natural data is a consequence of the laws of Physics / symmetries of the universe in which we happen to live. PS. You will enjoy the following papers too: "An exact mapping between the Variational Renormalization Group and Deep Learning" - https://arxiv.org/abs/1410.3831 https://arxiv.org/abs/1410.3831 ; and "Mutual Information, Neural Networks and the Renormalization Group" - https://arxiv.org/abs/1704.06279 https://arxiv.org/abs/1704.06279 .

cs702 9y ago

Thank you for posting this. Similar thoughts have been percolating in my stream-of-consciousness for a while, especially after coming across an earlier version of Tegmark & Lin's paper some months ago. My take is that deep neural nets work so well in practice because not all distributions of natural data are equally likely (and therefore, the no-free-lunch theorem(s), which assumes all distributions are equally likely, doesn't hold in the real world!); and that, in turn, is because the distribution of natural data is a consequence of the laws of Physics / symmetries of the universe in which we happen to live. PS. You will enjoy the following papers too: "An exact mapping between the Variational Renormalization Group and Deep Learning" - https://arxiv.org/abs/1410.3831 https://arxiv.org/abs/1410.3831 ; and "Mutual Information, Neural Networks and the Renormalization Group" - https://arxiv.org/abs/1704.06279 https://arxiv.org/abs/1704.06279 .