3 ms·

Please, no. While he provides some food for thought, asks a few important questions (and provokes even more), this text is full of pseudointellectualism. As in:

by stared 8y ago

Please, no. While he provides some food for thought, asks a few important questions (and provokes even more), this text is full of pseudointellectualism. As in: fancy words but utter lack of understanding of the core subjects one writes about. Most importantly: what is science (no, it is not a trivial question; I recommend going Ludwik Fleck's "Genesis and development of a scientific fact" route, http://www.evolocus.com/Textbooks/Fleck1979.pdf http://www.evolocus.com/Textbooks/Fleck1979.pdf) and what is AI (it is a vast field; some of it IS math/CS (as in: proving things), but it is a small part). For example, many of things he says would fit other practical disciplines, e.g. medicine. Yes, experiments are on a group of people. Yes, criterion that "drug X works better than drug Y, but we don't know why" is sufficient. > I don’t know data science folks well, but my impression is that they find the inexplicability and unreliability of AI methods frustrating. Well, it is not the main issue (speaking as a data scientist working with AI). At least he acknowledges his lack of expertise. > These failures of scientific practice seem as common in AI research now as they were in social psychology a decade ago. From psychology’s experience, we should expect that many supposed AI results are scientifically false. Also - no, it is not at level of psychology when it comes to the replication crisis. A lot of code (though, unfortunately, not all) is shared online, by the authors or other contributors, and people do replicate it (or there is an absence of replication, which also conveys a message).

Radim 8y ago
Speaking as both a ML researcher and an applied ML business owner (one who hired stared at one point — hi Piotr :), I respectfully disagree. The replication crisis in "AI" may not be as bad as psychology (I wouldn't know), but it's not great. Sadly, my brain has somehow learned to equate "SOTA" with "hot-stitched crap, stay away". Too many painful lessons. On the subject of publishing code: this is useful to the degree that it removes bad faith as the possible reason for the lack of replicability. But otherwise helps little in practical terms. You just have the privilege to sift through the bugs and bad design in more close-up. "I am afraid you are right. I used to reach ~72% via the given random seed on an old version of pytorch, but now with the new version of pytorch, I wasn't able to reproduce the result. My personal opinion is that the model is neither deep or sophisticated, and usually for such kind of model, tuning hyper parameters will change the results a lot (although I don't think it's worthy to invest time tweaking an unstable model structure)." = quote [1] for one of the "new SOTA" papers from NLP (WikiQA question answering), where the replication scores came out 62% instead of claimed 72%. I generally call this the "AI Mummy Effect" — looks great but crumbles to dust on touch. [1] https://github.com/pcgreat/SeqMatchSeq/issues/1 https://github.com/pcgreat/SeqMatchSeq/issues/1
- stared 8y ago
  Hi Radim! To make it clear, I am not happy with the current state of reproducibility in AI. Yet, it is still better that in all disciplines I interacted with (quantum physics, mathematical psychology). There the standard prectice was to not include any code, even if the paper bases on it. Vide my answer to "Why are papers without code but with results accepted?" (https://academia.stackexchange.com/questions/23237/why-are-papers-without-code-but-with-results-accepted/23238#23238 https://academia.stackexchange.com/questions/23237/why-are-p...). So, I was so happy to see that in Deep Learninig a lot of code appears on GitHub (I am the most happy if it appears in different frameworks, implemented y different people). Dirty code provides limited value. It's hard to learn from it, it's hard to re-use it, and its performance may depend on the phase of the Moon (and system setting, software versions, etc). Yet, IMHO, is much better than no code. It is not only about good faith, but about including all details. Some of them may seem unimportant (even to the author), yet crucial for the results. The next level is resonably well written code, with clear environment setting (e.g. Dockerfile/requirements.txt), and the dataset. Otherwise it is hard to proof it against "on my environment it works": > where the replication scores came out 62% instead of claimed 72%
- jononor 8y ago
  The cited example was attempted reproduced and found wanting. A plausible reason for this was discovered, and is accessible online. This in much better than no replication being performed at all. But yes, we need to continuously up the game. Adopting a standard that ensures model stability would be a good next target, and not accepting papers that don't uphold it. I'd also like to see open-source runnable code the default for paper acceptance, with some sort of 'punishment' for not having it. Maybe even make it prerequisite of empirical papers.
arvinjoar 8y ago
FYI: The author of this blog post also wrote "How to do research at the MIT AI Research Lab". While it's fair to suspect he may not be fully up to scratch with how things are done, he should have a pretty good overview
- stared 8y ago
  My criticism is towards this paper, not necessarily - the author. Surely, he knows something about AI (otherwise it would be impossible write anything gaining such publicity) and philosophy (AFAIK it is his field). Though, even if someone is accomplished scientist in a given field, it does not mean they are incapable of making (to put it mildly) questionable statements (Noam Chomsky on data-driven NLP, Judea Pearl on Deep Learning, Roger Penrose on quantum measurement and consciousness; from historical - Albert Einstein on quantum physics). Yet, there are many errors which won't be noticed by newcomers, but are demonstrably false for researchers and practitioners. It is dangerous as novices may be prone to "appeal to authority" and mistake witty style for knowledge. Don't take me wrong - I am all for sharing ideas, even half-baked. But I think that it works well better when there isn't artificially boosted confidence.
  - Aqua 8y ago
    On the one hand you make this sound extremely bad, while at the same time you describe it as just "making questionable statements". Also, maybe I misunderstood the analogy, but I think you're being very unfair putting Albert Einstein who was wrong on quantum physics in the same basket as Roger Penrose with his view on consciousness, which may be questionable, but hasn't been disproved.
    - stared 8y ago
      You are right that I shouldn't have put them in the same basket. While Penrose's ideas on consciousness are not considered mainstream (neither by cognitive scientists nor quantum physicists), they don't fall in the infertile basket of: - people gravitating to the state of science they were "raised into" - people talking about things they are don't mastered In this case it is a healthy scientific peculiarity. And who knows, it may turn out true. Or false, yet fertile. As ideas of faster-than-light communication with quantum states - which was flawed, yet gave birth to quantum information (more to this story, and an interesting overlap of non-science and science, in http://www.hippiessavedphysics.com/ http://www.hippiessavedphysics.com/).

by stared 8y ago

Radim 8y ago

Speaking as both a ML researcher and an applied ML business owner (one who hired stared at one point — hi Piotr :), I respectfully disagree. The replication crisis in "AI" may not be as bad as psychology (I wouldn't know), but it's not great. Sadly, my brain has somehow learned to equate "SOTA" with "hot-stitched crap, stay away". Too many painful lessons. On the subject of publishing code: this is useful to the degree that it removes bad faith as the possible reason for the lack of replicability. But otherwise helps little in practical terms. You just have the privilege to sift through the bugs and bad design in more close-up. "I am afraid you are right. I used to reach ~72% via the given random seed on an old version of pytorch, but now with the new version of pytorch, I wasn't able to reproduce the result. My personal opinion is that the model is neither deep or sophisticated, and usually for such kind of model, tuning hyper parameters will change the results a lot (although I don't think it's worthy to invest time tweaking an unstable model structure)." = quote [1] for one of the "new SOTA" papers from NLP (WikiQA question answering), where the replication scores came out 62% instead of claimed 72%. I generally call this the "AI Mummy Effect" — looks great but crumbles to dust on touch. [1] https://github.com/pcgreat/SeqMatchSeq/issues/1 https://github.com/pcgreat/SeqMatchSeq/issues/1

stared 8y ago
Hi Radim! To make it clear, I am not happy with the current state of reproducibility in AI. Yet, it is still better that in all disciplines I interacted with (quantum physics, mathematical psychology). There the standard prectice was to not include any code, even if the paper bases on it. Vide my answer to "Why are papers without code but with results accepted?" (https://academia.stackexchange.com/questions/23237/why-are-papers-without-code-but-with-results-accepted/23238#23238 https://academia.stackexchange.com/questions/23237/why-are-p...). So, I was so happy to see that in Deep Learninig a lot of code appears on GitHub (I am the most happy if it appears in different frameworks, implemented y different people). Dirty code provides limited value. It's hard to learn from it, it's hard to re-use it, and its performance may depend on the phase of the Moon (and system setting, software versions, etc). Yet, IMHO, is much better than no code. It is not only about good faith, but about including all details. Some of them may seem unimportant (even to the author), yet crucial for the results. The next level is resonably well written code, with clear environment setting (e.g. Dockerfile/requirements.txt), and the dataset. Otherwise it is hard to proof it against "on my environment it works": > where the replication scores came out 62% instead of claimed 72%
jononor 8y ago
The cited example was attempted reproduced and found wanting. A plausible reason for this was discovered, and is accessible online. This in much better than no replication being performed at all. But yes, we need to continuously up the game. Adopting a standard that ensures model stability would be a good next target, and not accepting papers that don't uphold it. I'd also like to see open-source runnable code the default for paper acceptance, with some sort of 'punishment' for not having it. Maybe even make it prerequisite of empirical papers.

arvinjoar 8y ago

FYI: The author of this blog post also wrote "How to do research at the MIT AI Research Lab". While it's fair to suspect he may not be fully up to scratch with how things are done, he should have a pretty good overview

stared 8y ago
My criticism is towards this paper, not necessarily - the author. Surely, he knows something about AI (otherwise it would be impossible write anything gaining such publicity) and philosophy (AFAIK it is his field). Though, even if someone is accomplished scientist in a given field, it does not mean they are incapable of making (to put it mildly) questionable statements (Noam Chomsky on data-driven NLP, Judea Pearl on Deep Learning, Roger Penrose on quantum measurement and consciousness; from historical - Albert Einstein on quantum physics). Yet, there are many errors which won't be noticed by newcomers, but are demonstrably false for researchers and practitioners. It is dangerous as novices may be prone to "appeal to authority" and mistake witty style for knowledge. Don't take me wrong - I am all for sharing ideas, even half-baked. But I think that it works well better when there isn't artificially boosted confidence.
- Aqua 8y ago
  On the one hand you make this sound extremely bad, while at the same time you describe it as just "making questionable statements". Also, maybe I misunderstood the analogy, but I think you're being very unfair putting Albert Einstein who was wrong on quantum physics in the same basket as Roger Penrose with his view on consciousness, which may be questionable, but hasn't been disproved.
  - stared 8y ago
    You are right that I shouldn't have put them in the same basket. While Penrose's ideas on consciousness are not considered mainstream (neither by cognitive scientists nor quantum physicists), they don't fall in the infertile basket of: - people gravitating to the state of science they were "raised into" - people talking about things they are don't mastered In this case it is a healthy scientific peculiarity. And who knows, it may turn out true. Or false, yet fertile. As ideas of faster-than-light communication with quantum states - which was flawed, yet gave birth to quantum information (more to this story, and an interesting overlap of non-science and science, in http://www.hippiessavedphysics.com/ http://www.hippiessavedphysics.com/).