3 ms·

Machine Learning Deserves Better Than This

28 points by jasonhansel 5y ago

axelroze 5y ago
Machine Learning, as understood in this context of Image Analysis with DeepLearning, is a pretty young field. I'd place 2012 and Hinton's work then as the starting point and maybe even 2015 when ResNet was introduced. That is in the best case 9 years. (AI and ML exist since Turing in 50s but in many forms) Fortran was made public in 1956 and only 12 years later use of goto considered harmful was published. It is still a young field with everyone wanting to get in, many without any clear goals or standards, just to publish anything. Trash certainly and from what I've seen myself by reading papers a lot of very badly hidden plagiarism. A ML analog to use of goto considered harmful is very much needed to raise the bar. Scientific integrity, consideration for the greater whole, 100% replicability and basis on sound principles of mathematical theory, most notably probability and statistics. Yet that is the price of freedom from corporations and big universities. Science for everyone and by everyone. We can't let bad apples, even if they are the vast majority, force us into willing deposition of our rights. If sites like ArXiV went away then access to knowledge would cease under the boots of Springer and Elsevier. Some sort of blind rating papers incognito could perhaps help to have the good rise above the rest. No information on authors, no information on raters. Just votes mirroring the quality, or lack of, w.r.t the paper.
kubb 5y ago
Machine learning has been overhyped and many people have overpromised what it can do. It's useful for recognizing objects on images, it improves automatic translation, it can rate chess positions, in general it's doing well in certain types of problems. The thing is, people were expecting it to revolutionize every discipline and it turns out it's incredibly hard to build something useful with it in a domain badly suited to it.
- saeranv 5y ago
  If you know what you're doing, having access to a universal approximator is lethal. But if you don't, it's true, you can only replicate previous successes in the domain, and that will be the limiting factor.
zozbot234 5y ago
Machine learning used to be called 'data mining' back in the 1990s and early 2000s. We should revive that name, since it describes the field quite well in many ways.
- dagw 5y ago
  Data mining was more like what we call 'data science' today. There wasn't that much Machine Learning in data mining field back in 90s. But I agree that the 'data mining' name was a good name and should be revived.
  - thecal 5y ago
    Sorry, I agree with the GP. This was a popular book for learning ML with Weka (which is still around): https://www.amazon.com/Data-Mining-Practical-Techniques-Management/dp/0120884070/ https://www.amazon.com/Data-Mining-Practical-Techniques-Mana... There is also the Knowledge Discovery in Databases (KDD) term which is still around via: https://www.kdd.org/ https://www.kdd.org/
pyentropy 5y ago
I've taken a look at a handful of CNN based models for COVID diagnosis via chest scans and I believe they are pure noise. The convolutional layers often get activated nowhere near the actual chest, and they do not seem to grasp the anatomy in scans. The reason is, they were trained as pure "covid-not covid" detectors. Unless you have billions of real scans I don't think that's a good idea, which is why such models need some kind of an organ feature extractor first. Even trained humans suck at this task, and it's not because they miss details.

28 points by jasonhansel 5y ago

Machine Learning, as understood in this context of Image Analysis with DeepLearning, is a pretty young field. I'd place 2012 and Hinton's work then as the starting point and maybe even 2015 when ResNet was introduced. That is in the best case 9 years. (AI and ML exist since Turing in 50s but in many forms) Fortran was made public in 1956 and only 12 years later use of goto considered harmful was published. It is still a young field with everyone wanting to get in, many without any clear goals or standards, just to publish anything. Trash certainly and from what I've seen myself by reading papers a lot of very badly hidden plagiarism. A ML analog to use of goto considered harmful is very much needed to raise the bar. Scientific integrity, consideration for the greater whole, 100% replicability and basis on sound principles of mathematical theory, most notably probability and statistics. Yet that is the price of freedom from corporations and big universities. Science for everyone and by everyone. We can't let bad apples, even if they are the vast majority, force us into willing deposition of our rights. If sites like ArXiV went away then access to knowledge would cease under the boots of Springer and Elsevier. Some sort of blind rating papers incognito could perhaps help to have the good rise above the rest. No information on authors, no information on raters. Just votes mirroring the quality, or lack of, w.r.t the paper.

kubb 5y ago

Machine learning has been overhyped and many people have overpromised what it can do. It's useful for recognizing objects on images, it improves automatic translation, it can rate chess positions, in general it's doing well in certain types of problems. The thing is, people were expecting it to revolutionize every discipline and it turns out it's incredibly hard to build something useful with it in a domain badly suited to it.

saeranv 5y ago
If you know what you're doing, having access to a universal approximator is lethal. But if you don't, it's true, you can only replicate previous successes in the domain, and that will be the limiting factor.

zozbot234 5y ago

Machine learning used to be called 'data mining' back in the 1990s and early 2000s. We should revive that name, since it describes the field quite well in many ways.

dagw 5y ago
Data mining was more like what we call 'data science' today. There wasn't that much Machine Learning in data mining field back in 90s. But I agree that the 'data mining' name was a good name and should be revived.
- thecal 5y ago
  Sorry, I agree with the GP. This was a popular book for learning ML with Weka (which is still around): https://www.amazon.com/Data-Mining-Practical-Techniques-Management/dp/0120884070/ https://www.amazon.com/Data-Mining-Practical-Techniques-Mana... There is also the Knowledge Discovery in Databases (KDD) term which is still around via: https://www.kdd.org/ https://www.kdd.org/

pyentropy 5y ago

I've taken a look at a handful of CNN based models for COVID diagnosis via chest scans and I believe they are pure noise. The convolutional layers often get activated nowhere near the actual chest, and they do not seem to grasp the anatomy in scans. The reason is, they were trained as pure "covid-not covid" detectors. Unless you have billions of real scans I don't think that's a good idea, which is why such models need some kind of an organ feature extractor first. Even trained humans suck at this task, and it's not because they miss details.