3 ms·
Machine learning (or more specifically deep learning) is shaking the world precisely because the models at the highest end are able to take in mostly unlabeled
by KRAKRISMOTT 4y ago
Machine learning (or more specifically deep learning) is shaking the world precisely because the models at the highest end are able to take in mostly unlabeled garbage and spit out useful results in an unsupervised manner. The world is not full of i.i.d. spherical cows.
100 years of statistical advances have not been able to solve image recognition at scale until convolutional neural networks. Large language models may be somewhat Bayesian but they do not resemble any traditional Bayesian techniques except at the highest level. Clean data and good sampling are incompatible with the real world outside of laboratory conditions (or large budgets).
- avs733 4y agoThat is a fairly brave statement given some of the research looking at how failed attempts at ML modeling fall apart [1] [1] https://ai.googleblog.com/2021/06/data-cascades-in-machine-learning.html https://ai.googleblog.com/2021/06/data-cascades-in-machine-l...
- wongarsu 4y agoSome of them do, but others depend on labeled and augmented data. For example training facial recognition starts with images labeled by who appears in them, then you run a model to find the face in the image and crop to that, then another image tells you where the eyes are and you use that to rotate the image to make the eyes level, and that's the input to the actual facial recognition neural network. And that's for the "naive" approaches that don't have special handling for faces don't look at the camera. And the field still routinely has problems because it turns out that somebody's data set only contained certain lighting conditions, or severely underrepresented certain races, etc.
- KRAKRISMOTT 4y agoNot sure about image recognition but for the language models and audio space, semi/self-supervised methods are currently doing quite well.