4 ms·
Who'd think compression works so well? (yes, neural networks are compression engines)
by chewxy 7y ago
Who'd think compression works so well?
(yes, neural networks are compression engines)
- FartyMcFarter 7y agoDo you mean in the same way that any mathematical function is a compression engine? That is, you implement something that can handle many cases (1+1, 2+3, 5+6) in a concise form? It seems to me like the real magic of neural networks is that they make it easier to search for a function that solves (to some extent) a particular problem.
- agumonkey 7y agoI always thought programming and even theory were knowledge compression
- andbberger 7y agoNot necessarily
- tanilama 7y agoCompression in your context is as meaningless as Generalization. Yes, you can say generalization is compression.
- fxtentacle 7y agoExcept that "generalization" implies that it works for previously unseen problems, which is usually not the case for AI. Compression, on the other hand, nicely captures the "learn and reproduce" approach that using AI entails.
- tanilama 7y agoUnseen problems is a ill defined term. There is a distinction between in domain and out of domain, both can be unseen by the model before. Even human as agent requires training before being deployed to unseen problems. Generalization is conditioned on experience, after all. AI generalizes to unseen in domain data given a specific task. That is why it is useful in the first place.
- nabla9 7y agoThis is not meaningful analogy due to being too generic. Using it does not add anything to discussion. Any mathematical model is 'compressed' form of reality and that's why they works well. Instead of compressed, simplified or abstracted, is better term. Machine Learning adds heuristic data driven model to scientific model.