As a complement to all the other comments so far: It is plausible that any goal directed inference must "compress" the raw data into a small number of bits requ

As a complement to all the other comments so far: It is plausible that any goal directed inference must "compress" the raw data into a small number of bits required to make a decision on the goal. For physicists "renormalization group flow" (RG) is nothing but a theory behind that process of "simplifying" a theory enough to make predictions (towards a specific goal). It is also natural that this compression process is iterative. There are many implementations of RG that fall within the framework, each with their own pros and cons, which makes each of them suitable for solving different problems. One of the more recent examples is the "Multi-scale Entanglement Renormalization Ansatz" (MERA) tensor network which accounts quantum entanglement in the RG framework (none of the previous formulations did, to my knowledge). In my opinion, that is a significant qualitative breakthrough, whose consequences are starting to bear fruit, and will continue for the next decade. OP compares MERA with RBMs and claims that they're similar. For an information theoretic perspective of this relation between deep learning and RG, refer to this fascinating paper by Naftali Tishby and Noga Zaslavsky, that brings together ideas from different fields: https://arxiv.org/abs/1503.02406 https://arxiv.org/abs/1503.02406 Regarding the implications: 1. For deep learning: We might have a shot at coming up with a theory for how DNNs work. This will help us design (much) better DNNs because today's designs are pretty ad-hoc and might be far from optimal. It might also give us some explanator power in reasoning about the way DNNs work. 2. For physics: It's not clear what the payoffs would be. There has been some recent work trying to use DNNs to solve physics-ish problems (eg: guessing the phase of the Ising mode, given MCMC generated samples). The hope is that in trying to study this relation, we uncover a better understanding of how RG works. That would have potentially tremendous consequences, for every thing from designing superconductors to understanding quantum gravity. PS: I'm a physicist and I've been exploring this area over the last year or so; if anyone is interested in discussing details, feel free to ping me.