5 ms·

Overall, I'll say that the general sentiment of this panel is half true. On the one hand, machine learning research has grown to such a large field that the si

by tansey 15y ago

Overall, I'll say that the general sentiment of this panel is half true. On the one hand, machine learning research has grown to such a large field that the signal to noise ratio has dropped dramatically. Lots of people try to squeak out another ICML or AAAI paper by making an incremental improvement that gets 94% accuracy instead of 92% on some set of benchmark tasks. This phenomenon is true across almost all academic disciplines, however, and is more an indictment of the "publish or perish" environment than anything else. On the other hand, some of the things these (famous) researchers are noting is complete FUD: >The answer is that there was a lot of progress in the 1960s and 1970s. Then something went wrong. Yep, things got hard. People early on thought that the difficulty of picking fruit would increase linearly over time. If they could pick all this low-hanging fruit in such a short span of time then surely in X years we'd be at point Y! Unfortunately, it turned out that the landscape was much steeper and fraught with local optima. As a machine learning researcher, I do try to focus on high-level problems that haven't be tackled before. My startup[1] is an example of that, and the extensions to it that I'm researching are as well. But does it really count as revolutionary from an academic sense? Probably not. The fact is that at this point, everyone has thought of something closely related to whatever you want to work on. Even if you've found an institution that enables you to explore freely, big impacts are really hard to come by these days. And when they do, older academics like the ones on this panel don't want to give credit because it's just another incremental improvement in their eyes. I suppose it's just frustrating to hear these guys sit at a panel and complain that AI researchers need to get to work-- they're AI researchers! Why aren't they doing anything? They have tenure and all the free time in the world. But they don't want to do that. They want to sit back and judge people while pointing to contributions they made forty years ago as proof that they can judge. Being critical of others' work while not producing anything of value is just mean-spirited. Put up or shut up. [1] http://effectcheck.com http://effectcheck.com

Groxx 15y ago
>Being critical of others' work while not producing anything of value is just mean-spirited. Put up or shut up. That's awfully close to "it takes one to know one", which is a fallacy at best. I can tell good tea from bad tea, but I've never grown it. Closer to the programming realm, any theoretical computer scientist will lament the huge delay between theory and practice - does this mean they don't have a valid complaint, simply because they aren't fixing everything? Specific to the piece: >Chomsky derided researchers in machine learning who use purely statistical methods to produce behavior that mimics something in the world, but who don't try to understand the meaning of that behavior. Chomsky compared such researchers to scientists who might study the dance made by a bee returning to the hive, and who could produce a statistically based simulation of such a dance without attempting to understand why the bee behaved that way. A valid complaint from a researcher, I think, and Chomsky has hardly "not produced anything of value". The first round of AI work in the 60s and 70s was a lot of theoretical work getting, as you mentioned, the low-hanging fruit in a new field. And then it turned out to be harder than the initial extreme success implied, and took more computational power. In more recent years, there have been business reasons to use relatively simple AI models to do fairly basic tasks - predicting your tastes in movies, for example. I think they're complaining about that focus, taking a known and just tweaking it until it works without doing any real foundational work, and they're not claiming that everybody in the field today is lazy. It's a complaint you see in every field that gains attention. The original researchers see a decreased focus on research, and little progress for how many people are in the field, and lament for the good old days when people thought instead of made money. A rose-tinted view, to be sure, but the viewers aren't wholly value-less for doing so.
- tansey 15y ago
  >I can tell good tea from bad tea, but I've never grown it. That's a straw man argument. If you used to grow tea, got famous for some awesome blend, started writing books about tea, stopped growing it, then joined a panel with a bunch of other former tea growers and said people need to start growing good tea again... then that'd be closer. Even better would be if in the time since you last grew tea, the plant had become endangered and complicated processes had to be undertaken by current manufacturers to extract sufficient quantities for production. You want better tea? You're in luck! You happen to have been an expert tea-grower in the past-- welcome back! Get to work. >Chomsky has hardly "not produced anything of value" Quite the opposite. In fact, all of these guys are brilliant researchers who have made huge impacts on their fields. My point was that they are not actively contributing anything, and haven't been for the last 30+ years. When was the last time Chomsky did any serious investigation that wasn't about how evil governments are? If Geoffrey Hinton, David Fogel, Jonathan Schaeffer, or any of the other "second-wave" pioneers[1] who have remained active in the last 20 years want to form a panel and complain about people not focusing on high-level research topics, then sure. Because I know when they do so, they'll point to their own research being published in the last few years as an example. [1] The guys who got us out of the AI winter.
  - woodson 15y ago
    > When was the last time Chomsky did any serious investigation that wasn't about how evil governments are? Well, he is still working on generative syntactic theory in linguistics. In his 2005 paper "On phases" he (again...) overhauled his minimalist program. So, yes, he is actively working (or at least has been until recently), but perhaps not in a field you are interested in.
  - Maro 15y ago
    Chomsky is 82 years old. We're lucky he still has enough energy and drive to sit on panels. I think it's disrespectful to demand serious investigation from an 82 year old man who has produced more of that in his earlier life than we all will combined.
alan-crowe 15y ago
Part of this is the difference between intra-generational progress and inter-generational progress. intRA-generational: You look at what the 30 year old researchers are doing. Wait twenty years. Look at what the 50 year old researchers are doing. It is mostly the same guys and your concept of progress includes the personal accumulation of knowledge in the field. intER-generational: You look at what the 30 year old researchers are doing. Wait twenty years. Look at what the 30 year old researchers are doing. It is a new crop of researchers and your concept of progress labours under the burden of human mortality; knowledge has to be transferred to younger researchers before it is lost. When a field is new, young people come in and pioneer it. We see the intra-generational progress. Fifty years later the age structure has stabilized and the kind of progress that is most visible changes from intra-generational to inter-generational. This is much slower.

by tansey 15y ago

Groxx 15y ago

>Being critical of others' work while not producing anything of value is just mean-spirited. Put up or shut up. That's awfully close to "it takes one to know one", which is a fallacy at best. I can tell good tea from bad tea, but I've never grown it. Closer to the programming realm, any theoretical computer scientist will lament the huge delay between theory and practice - does this mean they don't have a valid complaint, simply because they aren't fixing everything? Specific to the piece: >Chomsky derided researchers in machine learning who use purely statistical methods to produce behavior that mimics something in the world, but who don't try to understand the meaning of that behavior. Chomsky compared such researchers to scientists who might study the dance made by a bee returning to the hive, and who could produce a statistically based simulation of such a dance without attempting to understand why the bee behaved that way. A valid complaint from a researcher, I think, and Chomsky has hardly "not produced anything of value". The first round of AI work in the 60s and 70s was a lot of theoretical work getting, as you mentioned, the low-hanging fruit in a new field. And then it turned out to be harder than the initial extreme success implied, and took more computational power. In more recent years, there have been business reasons to use relatively simple AI models to do fairly basic tasks - predicting your tastes in movies, for example. I think they're complaining about that focus, taking a known and just tweaking it until it works without doing any real foundational work, and they're not claiming that everybody in the field today is lazy. It's a complaint you see in every field that gains attention. The original researchers see a decreased focus on research, and little progress for how many people are in the field, and lament for the good old days when people thought instead of made money. A rose-tinted view, to be sure, but the viewers aren't wholly value-less for doing so.

tansey 15y ago
>I can tell good tea from bad tea, but I've never grown it. That's a straw man argument. If you used to grow tea, got famous for some awesome blend, started writing books about tea, stopped growing it, then joined a panel with a bunch of other former tea growers and said people need to start growing good tea again... then that'd be closer. Even better would be if in the time since you last grew tea, the plant had become endangered and complicated processes had to be undertaken by current manufacturers to extract sufficient quantities for production. You want better tea? You're in luck! You happen to have been an expert tea-grower in the past-- welcome back! Get to work. >Chomsky has hardly "not produced anything of value" Quite the opposite. In fact, all of these guys are brilliant researchers who have made huge impacts on their fields. My point was that they are not actively contributing anything, and haven't been for the last 30+ years. When was the last time Chomsky did any serious investigation that wasn't about how evil governments are? If Geoffrey Hinton, David Fogel, Jonathan Schaeffer, or any of the other "second-wave" pioneers[1] who have remained active in the last 20 years want to form a panel and complain about people not focusing on high-level research topics, then sure. Because I know when they do so, they'll point to their own research being published in the last few years as an example. [1] The guys who got us out of the AI winter.
- woodson 15y ago
  > When was the last time Chomsky did any serious investigation that wasn't about how evil governments are? Well, he is still working on generative syntactic theory in linguistics. In his 2005 paper "On phases" he (again...) overhauled his minimalist program. So, yes, he is actively working (or at least has been until recently), but perhaps not in a field you are interested in.
- Maro 15y ago
  Chomsky is 82 years old. We're lucky he still has enough energy and drive to sit on panels. I think it's disrespectful to demand serious investigation from an 82 year old man who has produced more of that in his earlier life than we all will combined.

alan-crowe 15y ago

Part of this is the difference between intra-generational progress and inter-generational progress. intRA-generational: You look at what the 30 year old researchers are doing. Wait twenty years. Look at what the 50 year old researchers are doing. It is mostly the same guys and your concept of progress includes the personal accumulation of knowledge in the field. intER-generational: You look at what the 30 year old researchers are doing. Wait twenty years. Look at what the 30 year old researchers are doing. It is a new crop of researchers and your concept of progress labours under the burden of human mortality; knowledge has to be transferred to younger researchers before it is lost. When a field is new, young people come in and pioneer it. We see the intra-generational progress. Fifty years later the age structure has stabilized and the kind of progress that is most visible changes from intra-generational to inter-generational. This is much slower.