5 ms·

A blog post leads to doubts about Cornell’s food laboratory

77 points by nkurz 10y ago

danso 10y ago
This is what Andrew German has been covering as "Pizzagate" http://andrewgelman.com/2017/02/03/pizzagate-curious-incident-researcher-response-people-pointing-150-errors-four-papers-2/ http://andrewgelman.com/2017/02/03/pizzagate-curious-inciden... I had never heard of Wansink but it sounds like he's dominated the food science field, and has been the main purveyor of "common sense" concepts such as how serving food on a small plate will trick you into feeling fuller, and all of these findings are suddenly under attack [0]. Even worse, these attacks arise from blatantly obvious errors or anomalies that are apparent without even looking at the data -- which implies severe weaknesses in the peer review process. Given the scope and severity of the errors, it's really hard to imagine Wansink or the Food Lab surviving this. I wonder if one of the main takeaways for other academics is to never blog? All of this only came about because Wansink seemed to sincerely want to give advice on how to succeed in academia. Though the blog post's tone of "just work really hard for free!" was already boiling into a scandal in itself, before people started actually reading what his hard working grad student actually produced. [0] https://mobile.twitter.com/StuartJRitchie/status/846863854843891712 https://mobile.twitter.com/StuartJRitchie/status/84686385484...
flamingcow 10y ago
A researcher with hundreds of publications hadn't heard of p-hacking or the reproducibility crisis until a month ago? Lay people who occasionally read science news have been hearing about those for years now. This is the "I'm too stupid to be a fraud" defense, apparently.
qeternity 10y ago
> It’s also led to an undercurrent of anxiety among scientists who fear that their labs and their publication records might come under attack from a feisty cadre of freelance critics. Formerly known as "peer review".
- lutusp 10y ago
  > Formerly known as "peer review". An in-depth review of a paper in advance of publication is common (although that system isn't as reliable as had one been assumed). Such an exhaustive review after publication is rare, for multiple reasons. One serious problem with post-publication review is that the original data are rarely made available to reviewers. For example, academic critics of the work of Diederik Stapel couldn't justify revoking his Ph.D. because the original data that might have proved his fraud no longer existed. He eventually voluntarily gave up his degree[1], possibly to try to avoid a deeper investigation into his behavior. 1. http://www.science20.com/cool-links/dutch_psychologist_diederik_stapel_relinquishes_his_phd-84542 http://www.science20.com/cool-links/dutch_psychologist_diede...
  - apathy 10y ago
    I love how you believe that most reviewers have time to conduct a thorough review for a journal, most of the time. As opposed to something much closer to "LGTM!" With some "but be sure to cite my vaguely-relevant work in Foo et al, 1952".
    - lutusp 10y ago
      > I love how you believe that most reviewers have time to conduct a thorough review for a journal, most of the time. Your objection is to a position I never took -- in fact I suggested the opposite. The depth of a review is gauged by its length, not its accuracy or objectivity.
      - apathy 10y ago
        Ah, OK. I think we are on the same page. My mistake.
- Jweb_Guru 10y ago
  Bluntly, no. Laypeople are not the intended audience for scientific papers and usually lack the background to provide an informed review (hell, even most researchers in the field do). They do lack certain biases and incentives that researchers familiar with the topic usually have, but that doesn't compensate for what they don't know (for example, whether the claimed phenomenon has been tested many other times and not shown up; whether the claimed mechanism is bunk; whether the supposedly novel idea has already been published many other times). That being said, laypeople can (and should) absolutely call out research malpractice like what's described in the article.
dingbat 10y ago
> His insights have also attracted interest — and, more importantly, money — from policy makers. In 2014 the U.S. Department of Agriculture forked over a $5.5-million grant to support Wansink’s Smarter Lunchrooms program, which has since been put into practice in more than 30,000 schools across the country. there's something painfully appropriate about that quote appearing on the same page with a "trending" article titled: "Trump Proposal to Cut Indirect Research Payments Would Hit State Universities Hardest"
erikpukinskis 10y ago
Is this another example of why journals should publish negative results? Is there a way to review papers where you are just evaluating the soundness of the methodology regardless of results? "Do good science" is easier to set up reliable cultural practices around than "discover something new".
- apathy 10y ago
  Preregistration, as demanded for clinical trials, is essentially the only way to ensure that researcher degrees of freedom are avoided. Buy the ticket, take the ride. Needless to say, it's a scary ride, so most groups won't do it.

77 points by nkurz 10y ago

danso 10y ago

This is what Andrew German has been covering as "Pizzagate" http://andrewgelman.com/2017/02/03/pizzagate-curious-incident-researcher-response-people-pointing-150-errors-four-papers-2/ http://andrewgelman.com/2017/02/03/pizzagate-curious-inciden... I had never heard of Wansink but it sounds like he's dominated the food science field, and has been the main purveyor of "common sense" concepts such as how serving food on a small plate will trick you into feeling fuller, and all of these findings are suddenly under attack [0]. Even worse, these attacks arise from blatantly obvious errors or anomalies that are apparent without even looking at the data -- which implies severe weaknesses in the peer review process. Given the scope and severity of the errors, it's really hard to imagine Wansink or the Food Lab surviving this. I wonder if one of the main takeaways for other academics is to never blog? All of this only came about because Wansink seemed to sincerely want to give advice on how to succeed in academia. Though the blog post's tone of "just work really hard for free!" was already boiling into a scandal in itself, before people started actually reading what his hard working grad student actually produced. [0] https://mobile.twitter.com/StuartJRitchie/status/846863854843891712 https://mobile.twitter.com/StuartJRitchie/status/84686385484...

flamingcow 10y ago

A researcher with hundreds of publications hadn't heard of p-hacking or the reproducibility crisis until a month ago? Lay people who occasionally read science news have been hearing about those for years now. This is the "I'm too stupid to be a fraud" defense, apparently.

qeternity 10y ago

> It’s also led to an undercurrent of anxiety among scientists who fear that their labs and their publication records might come under attack from a feisty cadre of freelance critics. Formerly known as "peer review".

lutusp 10y ago
> Formerly known as "peer review". An in-depth review of a paper in advance of publication is common (although that system isn't as reliable as had one been assumed). Such an exhaustive review after publication is rare, for multiple reasons. One serious problem with post-publication review is that the original data are rarely made available to reviewers. For example, academic critics of the work of Diederik Stapel couldn't justify revoking his Ph.D. because the original data that might have proved his fraud no longer existed. He eventually voluntarily gave up his degree[1], possibly to try to avoid a deeper investigation into his behavior. 1. http://www.science20.com/cool-links/dutch_psychologist_diederik_stapel_relinquishes_his_phd-84542 http://www.science20.com/cool-links/dutch_psychologist_diede...
- apathy 10y ago
  I love how you believe that most reviewers have time to conduct a thorough review for a journal, most of the time. As opposed to something much closer to "LGTM!" With some "but be sure to cite my vaguely-relevant work in Foo et al, 1952".
  - lutusp 10y ago
    > I love how you believe that most reviewers have time to conduct a thorough review for a journal, most of the time. Your objection is to a position I never took -- in fact I suggested the opposite. The depth of a review is gauged by its length, not its accuracy or objectivity.
    - apathy 10y ago
      Ah, OK. I think we are on the same page. My mistake.
Jweb_Guru 10y ago
Bluntly, no. Laypeople are not the intended audience for scientific papers and usually lack the background to provide an informed review (hell, even most researchers in the field do). They do lack certain biases and incentives that researchers familiar with the topic usually have, but that doesn't compensate for what they don't know (for example, whether the claimed phenomenon has been tested many other times and not shown up; whether the claimed mechanism is bunk; whether the supposedly novel idea has already been published many other times). That being said, laypeople can (and should) absolutely call out research malpractice like what's described in the article.

dingbat 10y ago

> His insights have also attracted interest — and, more importantly, money — from policy makers. In 2014 the U.S. Department of Agriculture forked over a $5.5-million grant to support Wansink’s Smarter Lunchrooms program, which has since been put into practice in more than 30,000 schools across the country. there's something painfully appropriate about that quote appearing on the same page with a "trending" article titled: "Trump Proposal to Cut Indirect Research Payments Would Hit State Universities Hardest"

erikpukinskis 10y ago

Is this another example of why journals should publish negative results? Is there a way to review papers where you are just evaluating the soundness of the methodology regardless of results? "Do good science" is easier to set up reliable cultural practices around than "discover something new".

apathy 10y ago
Preregistration, as demanded for clinical trials, is essentially the only way to ensure that researcher degrees of freedom are avoided. Buy the ticket, take the ride. Needless to say, it's a scary ride, so most groups won't do it.