4 ms·

This is one of those rare moments in intellectual life where being in the room and now seeing the debate develop, it becomes clear that the resulting hype isn't

• _delirium 14y ago
  I don't think I would take that strident battle-against-dinosaurs view, in part because I think intellectually understanding things is useful in and of itself, not some kind of "just build it and shut up" anti-intellectual view; and also because I don't think it's an accurate summary of the history of AI. There's no particular reason we can't both build and study things in various ways, and the history of AI has been full of people doing many takes on both. In particular, statistical approaches have been used for a long time, but were not practical until fairly recently; it was the lack of "big data" computing power holding them back more than anything. Statistical machine translation and parsing experiments have been tried on and off for decades, but with 1950s-era data they produced total garbage as output, even worse than the (also bad) symbolic approaches. Hence why Shannon's work on text processing didn't produce practical NLP or NLG systems. It took Google-sized data to produce statistical translation that was actually usable. What numerical approaches were possible on computers of the time were fairly extensively investigated when they became possible (e.g. the 1980s focus on "sub-symbolic AI", with perceptrons, neural networks, numerical regression methods, etc.). Some were shelved for years because they just didn't work as well, e.g. symbolic game-tree search massively outperformed machine learning in board games in the early experiments, which is why Samuels's 1950s ML-based checkers player was theoretically intriguing but not considered very practical.
  • jerf 14y ago
    I often feel bad for computer science researchers in the 50s and 60s. It isn't that they were any less smart than today, it's just that where they had kilo-X, we have giga-X, if not tera-X or even peta-X. One of my evolutionary computation professors said that he had a deck of cards that he carried from job to job, university to university, running bits of his evolutionary computation experiment on spare cycles, an experiment that runs in fractions of a second on hardware at the time he told the story in 2001. Presumably now it would run in even more fractional fractions of a second.
  • slurgfest 14y ago
    Intellectually understanding things is useful. There's no anti-intellectualism here. It's become increasingly clear that Chomsky's approach to language is not going to generalize to AI successes or explanations of many domains of human performance, as was promised. That approach has not yielded fruitfully anywhere except the narrow realm of syntax (in particular typologies of syntax). While machine learning has kicked butt left and right. There is no reason people can't go on studying syntax at the same time machine learning expands. But it would be dishonest to ignore the extreme and exclusive claims laid down by Chomsky's school at the outset of the cognitive revolution. These claims were given a very liberal benefit of the doubt for decades. A lot of good work has been done in syntax. But now those claims about AI and psychology-in-general are clearly threadbare, since they have not yielded the promised fruits either in AI or in explaining human functions. They have not even yielded plausible IOUs. Remember, this was supposed to explain pretty much everything. Not even language acquisition has been explained. That is empirical inadequacy. Who cares if Chomsky thinks it's beautiful? And it's equally clear that learning of various kinds has to occur and also gives us the more parsimonious and elegant solutions to problems (no-solution and no-explanation is not elegant even if it is simpler). Machine learning has a great deal of conceptual beauty - if you study it rather than pooh-poohing it because of some facile abductive argument to UG. Computing power isn't a problem. The scale of the human brain is enormous and we are still nowhere near reaching it with our computational resources.
    • ddd571 14y ago
      What about research showing things like intermediate traces exist, we parse sentences in accordance with binding principles, we don't postulate gaps for fillers in sentences in island contexts unless there'll later be a potential filler gap? All of these are facts we know from abstract work in Chomskyan syntax, and yet they've been replicated in laboratories. So, your claim that there's no fruits is just simply empirically false
• pnathan 14y ago
  "Who cares about the ontology when we can recreate? It's like saying airplanes don't properly explain flight because they don't replicate how birds do it. Who cares?" Well, I do. Being able to understand something from first principle is very different than being able to model it and gain an approximation of it. One might point out that this debate crops up in a different form in developer circles, but reframed as "Learn assembly & cs theory vs. use IDE & ignore assembly and theory".
• JabavuAdams 14y ago
  > There isn't one algorithm in the human mind to decode all of our mental capabilities. Careful here. There could be one algorithm, but we may not be able to express its parameters in the way we like to for simple models. These statistical approaches are algorithms. It's the fact that we can't make sense of the parameters that seems to lead people to believe that they don't explain anything. The fact that we can't categorize all life in an unambiguous way that makes sense to humans doesn't mean that the elegant and simple algorithm of evolution by natural selection is wrong. Model vs. initial conditions. Knowledge vs. search.
  • slurgfest 14y ago
    You are right, these statistical approaches are algorithms. And they themselves are very simple and do not necessarily require a lot of initial state. And you are also right that it is extra hard to reverse-engineer these. But they amount to learning and search. And because of the politics of Chomsky's movement, it has always fought the concept of learning tooth and nail in favor of innate knowledge or ready-made models. This is why we talk about language "acquisition" and why we read Chomsky saying that contemporaries who studied learning were facilitating totalitarianism. It's also worth mentioning that Chomsky has stated that he does not think the brain has evolved. So however many algorithms there are (and however they are employed to get the vast array of intelligent human actions) many things you have said here are totally anathema to Chomsky's school. It was pushed too hard, it overextended and only now has any challenge come to it - from the quarter of AI. (Up until a few years ago, fighting it in academic Psychology meant you were OK with killing your career, and anyone less respectable than Norvig would get torn apart - look at what they are saying about Norvig now).
    • JabavuAdams 14y ago
      Nauseating. Trashing my academic career is probably the best "mistake" I've made.
• fusiongyro 14y ago
  > It's clear that Chomsky and Universal Syntax has held back the entire field of AI That may be true, but there is value to understanding language for the sake of language outside the practical goals of improving AI. There is plenty of evidence for parameters playing a role in human language whether or not a parametric implementation of NLP is possible, desirable or necessary. It's certainly true that a statistical approach that bears plenty of fruit in AI applications is going to be very strained to provide anything of value back to linguistics or developmental psychology.
• meric 14y ago
  "It's like saying airplanes don't properly explain flight because they don't replicate how birds do it." The analogy doesn't hold. We don't want planes to fly like birds, but we may want machines to thinking like humans.
  • eyko 14y ago
    Also, the statistical approach would be to imitate flying creatures until we can fly, rather than understanding aerodynamics in order to create "vessels that float".