27 ms·
There was a similar discussion in another thread, someone pointed to the piekniewski.info blog and I found that article that I think is spot-on (though I don't
by YeGoblynQueenne 6y ago
There was a similar discussion in another thread, someone pointed to the
piekniewski.info blog and I found that article that I think is spot-on (though
I don't like the blog overall because it seems to have a real bone to pick).
Scientific progress is marked by hypotheses that explain observations and
generate predictions that can then be verified (or falsified) by new
observations. Machine learning research has produced scant few such verifiable
hypotheses in its very long history. To a great extent the same goes for the
entire field of AI. I quote from John McCarthy's article on the Lighthill
Report [1]:
Much work in AI has the "look ma, no hands" disease. Someone programs a
computer to do something no computer has done before and writes a paper
pointing out that the computer did it. The paper is not directed to the
identification and study of intellectual mechanisms and often contains no
coherent account of how the program works at all. As an example, consider that
the SIGART Newseletter prints the scores of the games in the ACM Computer
Chess Tournament just as though the programs were human players and their
innards were innacessible. We need to know why one program missed the right
move in a position - what was it thinking about all that time? We also need an
analysis of what class of positions the particular one belonged to and how a
future program might recognize this class and play better".
McCarthy wrote that in 1974. The criticism is every bit as valid today as it
was back then. Machine learning and AI research remains an endeavour that is
rarely and only incidentally scientific. Despite 70 years of AI research and
the recent amazement at the "progress" in machine learning (a "progress" only
in the context of the field's own measures of progress) we have learned very,
very little from AI research that we didn't know already. Big machines
can compute big programs. So, what?
_________
[1] The quote is from McCarthy's review of the Lighthill report. The review was
published in the journal Artificial Intelligence Vol. 5, No. 3, 1974. A pdf
copy is here:
jmc.stanford.edu/artificial-intelligence/reviews/lighthill.html
McCarthy is best known to computer scientists as the father of Lisp but he
is also one of the founders of AI and the man who named it. The Lighthill
Report was a report by James Lighthill on the progress of AI research,
commissionned by the UK government and published in 1973. It was extremely
negative and caused AI funding to freeze for years, thus bringing on the first
"AI winter" and bascially killing Good, Old-Fashioned, logic-based AI.
- syrrim 6y agoScience isn't about understanding how something works. It is, as you say, about generating and testing hypotheses. In the case of machine learning, those hypotheses take the form of a program and an associated criteria. The hypothesis is verified by demonstrating the efficacy of the program. That a machine learning specialist is able to consistently produce effective programs demonstrates that they have a good understanding of the relationship between the program and its functioning. This is true even if they couldn't explain how the program "works" to the satisfaction of an observer.
- YeGoblynQueenne 6y ago>> In the case of machine learning, those hypotheses take the form of a program and an associated criteria. The hypothesis is verified by demonstrating the efficacy of the program. This is the first time I hear anything like that. Can you say where this idea comes from? I've never seen it mentioned in any machine learning paper or textbook etc.