4 ms·
Lets rename "Deep learning" to "Statistical Compression". Its essentially distilled data from a dataset, limited by size of the neural network. There isn't anyt
by FrozenVoid 8y ago
Lets rename "Deep learning" to "Statistical Compression".
Its essentially distilled data from a dataset, limited by size of the neural network. There isn't anything "deep" there, the goal of NN is always towards overfitting the data.
- dschuetz 8y agoThat approach didn't work for "cyber", "quantum" or similar buzzwords. Calling something by its actual meaning isn't easy to sell. Hell, even "hacking" still has the connotation of some criminal activity, although it is widely known that the process of hacking in itself cannot be classified as benign or malign! But it doesn't matter. What matters is what public thinks of it hearing the buzzword. "Deep learning" is the same thing: "I have no idea what it is, but you know what I mean right?"
- IshKebab 8y agoThere is something deep there - the neural networks are deep. They have lots of layers. It's in contrast to shallow neural networks. It doesn't mean "deep and meaningful".
- gaius 8y agoIt doesn't mean "deep and meaningful". It would not be nearly as well funded were it not for this misconception on the part of budget holders, and everyone involved in ML knows it.
- garmaine 8y agoI have never once encountered this misconception until now. Every explanation of deep learning I’ve seen begins with an explanation of the origin of the phrase.
- IshKebab 8y agoA few weeks ago someone submitted an entire article to HN that consisted of someone complaining that deep learning wasn't really deep, in the deep & meaningful sense. They clearly misunderstood. Was pretty funny.
- gaius 8y agoLets rename "Deep learning" to "Statistical Compression" The previous name for what is now called “machine learning” was predictive statistics. But a better name for deep learning is “machine intuition” since it seems to work pretty well but can’t be readily explained
- FrozenVoid 8y ago> can’t be readily explained It works "pretty well" in well-defined complete-information domains like Go. The training in that case is actually about improving the next dataset from which higher-quality patterns are extracted. 1.We have a NN that doesn't know how to play and makes random moves. 2.Blank slate dataset filled from random games. 3.NN learns to produce better games by weighting wins vs losses, making losing moves less likely and winning moves more likely. 4.NN produces slightly better dataset. 5.Repeat #3 until you have covered a large chunk of go openings and have pretty good estimation for move quality. 6.NN becomes the condensed statistic pattern data extracted from billions of games, able to estimate value of a move more accurately than humans. Still in some rare cases it "condensed pattern database" will make mistakes due gaps in data or wrong correlation that NN hallucinated from its connections.
- gaius 8y agoworks "pretty well" in well-defined complete-information domains like Go But no one can definitively say “my AI credit scoring system didn’t decline his mortgage because he is black” and that’s the explainability bar that needs to be passed. Right now the state of it is “gut feel” with an unknown amount of unconscious bias.
- tim333 8y agoSame as humans really.
- garmaine 8y ago> since it seems to work pretty well but can’t be readily explained That is becoming less and less true.
- viewtransform 8y ago"Dimension reduction" is what I would call it. Going from R^n to R^m n>m using parameterized non-linear high dimensional mappings.