3 ms·
If you expand a NN to decision trees, the resulting decision tree(s) can take up orders of magnitude more space and take orders of magnitude more time to run th
by wolfium3 4y ago
If you expand a NN to decision trees, the resulting decision tree(s) can take up orders of magnitude more space and take orders of magnitude more time to run than the original NN.
The paper is discussed here: https://www.youtube.com/watch?v=_okxGdHM5b8 https://www.youtube.com/watch?v=_okxGdHM5b8
- titzer 4y agoSo...compressed decision trees?
- deleted 4y ago[deleted]
- sigstoat 4y agoat that point most things are compressed decision trees.
- TeMPOraL 4y agoPossibly related: learning is isomorphic to compression, and the two may be in fact fundamentally the same thing.
- mncharity 4y agoOh, wow. I'd long thought of my own learning as "when reality surprises, isn't merely what you'd expect, isn't "obvious", then fix that, with emphasis on simplicity and generality". I'd not recognized that as compression. Tnx!
- m3047 4y agoCounter argument: For every trainable NN, there exists an equivalent, sparser [edit] trained NN. https://numenta.com/blog/2019/08/30/case-for-sparsity-in-neural-networks-part-1-pruning https://numenta.com/blog/2019/08/30/case-for-sparsity-in-neu... https://web.archive.org/web/20211130211241/https://www.bhauth.com/blog/programming/sparsity.html https://web.archive.org/web/20211130211241/https://www.bhaut... https://arxiv.org/abs/1803.03635 https://arxiv.org/abs/1803.03635