3 ms·
simulating complex behavior requires at least 10mb of data The author has not shown that that much is required. At most, the author has shown that it has been
by kd0amg 10y ago
simulating complex behavior requires at least 10mb of data
The author has not shown that that much is required. At most, the author has shown that it has been done with that much. Seeing as having only thousands of states is apparently enough to make a Turing machine's behavior independent of ZFC (http://www.scottaaronson.com/blog/?p=2725 http://www.scottaaronson.com/blog/?p=2725), I really doubt this "10 megabyte" lower bound.
- gwern 10y agoOP's mind would probably be absolutely blown by the demo scene if he thinks 10MB of data is impressive. Also, the NN example is interesting because no one in deep learning thinks that you actually need all those parameters: most NNs can be shrunk by 90%+ with some simple tweaks like quantizing neural weights and pruning them (speculation is that you need to train large parameterized NNs to make the path through high-dimensional space to the optimal program smooth enough for gradient descent to travel).