3 ms·
> getting compressed into the network The idea that neural network weights are compressed data is a distressingly common human hallucination.
by mr_toad 3y ago
> getting compressed into the network
The idea that neural network weights are compressed data is a distressingly common human hallucination.
- Terr_ 3y agoThe overlaps are obvious, so on what grounds are you claiming the overall system not like compression? (Dynamic, lossy, predictive.)
- mr_toad 3y agoNeural networks are no more like compressed data than a Huffman tree, or frequency counts of letters, or the weights of a linear model, or even a sample average. You can use them to build a compression algorithm (and people have), but they are not compressed data.
- comex 3y agoNeural networks can repeat significant amounts of their training data verbatim or near-verbatim. None of the other things you mentioned are capable of doing that. They don’t store nearly enough information.