7 ms·

A Cornell Scientist’s Downfall

70 points by melling 8y ago

melling 8y ago
Abusing statistics: “Mr. Wansink’s fall from grace began with a 2016 blog post in which he blithely confessed to using improper research techniques known as p-hacking and HARKing. P-hacking involves running statistical analyses until they produce a statistically significant result; HARKing stands for “hypothesizing after the results are known.” The post prompted a small group of skeptics to take a hard look at Mr. Wansink’s past scholarship. Their analysis, published in January 2017, turned up an astonishing variety and quantity of errors in his statistical procedures and data.”
- rossdavidh 8y ago
  Oh, if only we had a system to pass articles that are submitted to scientific journals to some sort of expert committee, that could look at it and see if it was likely to be valid. We could call it, I don't know, "peer checkup", or "expert review", or...oh never mind.
  - ble 8y ago
    Peer review can't fix broken incentives.
  - sn41 8y ago
    The issue is that peer review never pays off directly. The time spent in reviews mostly goes down a black hole. Editorships are awarded based on publication record, and not quality of reviews. As a result, more and more, reviews will tend to be shallow and uncritical.
    - rossdavidh 8y ago
      Truth.
- soperj 8y ago
  >The post prompted a small group of skeptics to take a hard look at Mr. Wansink’s past scholarship It's funny that this is basically the peer-review version of harking.
  - theshadowknows 8y ago
    You just wrecked my mind
  - pcrh 8y ago
    It's also pretty close to the Baysian approach. The difference being that in the Baysian approach one is "permitted" to revise the hypothesis as new data is acquired.
    - antidesitter 8y ago
      Correction: Bayesian, not Baysian.
- kradroy 8y ago
  I've never heard the term "HARKing" before. This is exactly what my advisor had me do on our experiments that couldn't confirm our initial hypotheses. I knew it was wrong because we set up the experiments in a way to eliminate confounding variables. So any conclusions drawn on data for hypotheses outside of those would be suspect. The experiments we ran were costly in time to do because we were using feedback from students using our learning system over a semester. I quit working with her because of that, and also because she was a total asshole. I refuse to donate to my university until she's no longer on the faculty. I don't want her benefiting from my hard-earned money.
  - MrEfficiency 8y ago
    This is why I believe in Engineering and not the modern "Academia Science". Totally corrupt, Ive seen it when I talked to professors about getting my PhD. You prove what they want proven. One is going to have millions of customers using it, the other is going to have a few biased professors review the data.
    - vkou 8y ago
      If you think this is any different from how industry works, you have not spent enough time in industry. After-the-fact-hypothesis is constantly used to retroactively justify business decisions.
      - MrEfficiency 8y ago
        I have 10 years in Industry, testing hypothesis is testing. Building a product that is bad because 'X person wanted it' is bad Capitalism and will be punished. Academia doesnt have this same customer focused delivery.
        rossdavidh 8y ago
        Hmmm...I think it depends on how fast the feedback loop of bad sales is. There are lots of cases where somebody inside a corporation wants the answer to be "X", because that's what they had advocated. It may be worse for the corporation as a whole, but it's better for that guy's career, and if the hit to the corporation is not immediately visible.
- nonbel 8y ago
  He also shamed a post-doc who refused to do it: https://web.archive.org/web/20170312041524/http:/www.brianwansink.com/phd-advice/the-grad-student-who-never-said-no https://web.archive.org/web/20170312041524/http:/www.brianwa...
  - sampo 8y ago
    "In the meantime, this Turkish woman's resume will always have the five papers below." Turns out it won't: 2 of the 5 papers have been retracted. https://www.frontiersin.org/articles/10.3389/fpsyg.2017.02140/full https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0214... https://bmcnutr.biomedcentral.com/articles/10.1186/s40795-017-0195-6 https://bmcnutr.biomedcentral.com/articles/10.1186/s40795-01...
sremani 8y ago
We replace people who are curious about science with people who are good at getting grants. Iron law of bureaucracy at work.
- RcouF1uZ4gsC 8y ago
  "If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible result being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate a different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want one kind of research, but, if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is to pay him for doing something else and give him enough leisure to do research for the love of it." Quote attributed to J.J. Thompson via a previous HN comment https://news.ycombinator.com/item?id=14022158 https://news.ycombinator.com/item?id=14022158
  - iguy 8y ago
    You may enjoy the Connes interview linked here, discussing (in the 2nd half) French vs US vs Soviet systems: http://infoproc.blogspot.com/2018/09/the-french-way-alain-connes-interview.html http://infoproc.blogspot.com/2018/09/the-french-way-alain-co...
  - ataturk 8y ago
    That quote is very profound. I know from my own doctoral research how difficult innovation is and how stressful that becomes. I want to also have a personal life outside of just working and what I saw at my university was unhealthy to the extreme. I got my doctorate, but I quit academia. You only get one life to live and I couldn't see why I should work endless hours for shit wages, worrying about obtaining funding year in and year out, dealing with the terrible communists who run US campuses these days, and so on. It was a completely undesirable lifestyle with only a few redeeming qualities. Bottom line, for me it was less headache to become a software engineer and take longer vacations once in awhile. I would rather do hard research or R&D, but it is what it is.
adamc 8y ago
Eh, paywall.
- elektor 8y ago
  Install this extension to use Outline. https://chrome.google.com/webstore/detail/outline-read-without-clut/daoolpmoieinofbnddaofhkhmbagfmnj?hl=en-US https://chrome.google.com/webstore/detail/outline-read-witho...
  - ur-whale 8y ago
    Only works on Chrome, so not an option.
  - prairiedock 8y ago
    You don't need the extension to get the benefit. Just go to https://outline.com https://outline.com
- dang 8y ago
  Please don't post unsubstantive comments here. Re paywalls: if there's a workaround, it's ok. Users usually post workarounds in the thread. This is in the FAQ at https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html and there's more explanation here: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989 https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix&page=0&query=by:dang%20paywall https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
sn41 8y ago
http://archive.is/II8xf http://archive.is/II8xf
- misiti3780 8y ago
  off topic, but does anyone know how the technology behind archive.is works and how they make money (assuming they do). it seem pretty impressive to me that it always gets the article right.
  - cheeko1234 8y ago
    The site is a commercial enterprise, and as such can go kaputt at any given point, especially if it does not find a lucrative business model. Although it's not a strong indication of long-term issues; in October 2016 the site "made transparent" the server costs, and started to accept donations. A weekly crowdfunded target of $800[4] is set to maintain the site. Prior to this, the site actively refused donations. A donation link took the user to an animal shelter donation page. Stated in January 2017, through donations the site only receives "more than $1.50 every day, enough for a bowl of phở". Src: https://www.archiveteam.org/index.php/Archive.is https://www.archiveteam.org/index.php/Archive.is
    - true_religion 8y ago
      I thought archive.is was part of the internet archive project which is a public library of California.
      - sp332 8y ago
        Nope, archive.org is a whole different organization. Also note that the "Wayback Machine" is only one project at the Internet Archive. They also have 78 RPM records, video games, tons of books, CD's, etc. https://en.wikipedia.org/wiki/Lists_of_Internet_Archive%27s_collections https://en.wikipedia.org/wiki/Lists_of_Internet_Archive%27s_... You can donate at https://archive.org/donate https://archive.org/donate Edit: Archive Team is also a separate group.
    - 8y ago
misiti3780 8y ago
also relevant: https://andrewgelman.com/2016/12/15/hark-hark-p-value-heavens-gate-sings/ https://andrewgelman.com/2016/12/15/hark-hark-p-value-heaven...
sampo 8y ago
Also e.g. The Atlantic wrote about this 2 days ago: https://www.theatlantic.com/health/archive/2018/09/what-is-food-science/571105/ https://www.theatlantic.com/health/archive/2018/09/what-is-f...
fmajid 8y ago
The guy has a BS in Business Administration, a MA In journalism and a PhD in Marketing. He is a professor of Marketing at Cornell's School of Economics. In what universe would such a person be called a scientist and not a huckster doing what hucksters do?
- wmf 8y ago
  I guess he has marketing down to a science.
  - helpfulTroll 8y ago
    Trolling is a art.
- qubax 8y ago
  "Social science". The term science has been appropriated by so many pseudosciences that it has lost any meaning. Because real science has such a good reputation, all the academic frauds have decided to link their field to science to enhance their own reputation. There is political "science", legal "science", economics, etc. Here is richard feynmann's poignant take on the matter. BTW, it has gotten way worse since feynmann's interview. https://www.youtube.com/watch?v=tWr39Q9vBgo https://www.youtube.com/watch?v=tWr39Q9vBgo
  - mhuffman 8y ago
    Also watching a youtube video on Javascript makes you a "Software Engineer". Bit of a slap to what an actual engineer (in any concentration) has to go through to get a degree.
  - deleted 8y ago
    [deleted]
  - 7402 8y ago
    See also "Harary's Law" <https://quoteinvestigator.com/2015/06/26/not-science/> https://quoteinvestigator.com/2015/06/26/not-science/>, which says that any field that has the word “science” in its name is guaranteed thereby not to be a science.
DanielleMolloy 8y ago
Starting to get mixed feelings about these cases. Impostors and fraudsters certainly need to be exposed. But publicly ripping apart few individuals who, together with their peer reviewers, in some cases may simply not know better (e.g. because their whole field has low standards – nutrition science seems to be one of these fields –; or because they are inexperienced and their co-authors and supervisors don't care about details and correct science and just want to have their names on the paper) can not be the solution for an overall malfunctioning scientific communication and incentive system, or for bad methods that are used by entire fields. The current situation also invites all sorts of abuse. Need to say that certain parts of the open science movement appear to be a bit angry, not to say sadistic. Wansinks case was one that was ripped apart on Twitter. Wansinks research has received lots of media attention before - would his statistical methods have been investigated and debunked without that? How many Wansinks are out there, still having their job, continuing to apply bad methods (while not committing active fraud), never to be called out for their mistakes? I've been in Japan during the Obokata stem cell case. Her public apology was broadcasted on national TV. In konbinis you could see various tabloid newspapers with her name and her crying face on the cover for weeks. There seemed to be different mechanisms at play than a genuine interest in correct science. One of Obokata's supervisors, Yoshiki Sasai, committed suicide as a consequence of the case.
- nonbel 8y ago
  These people ostracize and hassle those who try to do a honest job and point out the issues with their methods. They also drive the people trying to do a good job out of the field because they can't compete with the fake productivity people like Wansink enjoyed. Also, the unnecessary suffering caused by this behavior is basically unimaginable. Who knows what advances we would have made if 90+% of resources dedicated to science during this golden era of cheap energy hadn't been wasted? We can't even estimate that. But yea, Wansink is just the tip of the iceberg. I've been saying for awhile that future generations will need to double check everything since about 1940, because the professional researchers have not been checking each others work, and have been using known incorrect methods to draw conclusions (NHST).
- jrumbut 8y ago
  I think in situations like this an idea worth considering is a truth and reconciliation process. The idea is everyone is given a year or so to go back, identify anything that they may have p-hacked or otherwise been unethical, and if they are radically transparent and work to correct the record then they can continue on reputation and position intact. It requires some serious compromise, it's imperfect, but it may be the best way to get the truth and prevent an ugly process from dragging on indefinitely.

70 points by melling 8y ago

Abusing statistics: “Mr. Wansink’s fall from grace began with a 2016 blog post in which he blithely confessed to using improper research techniques known as p-hacking and HARKing. P-hacking involves running statistical analyses until they produce a statistically significant result; HARKing stands for “hypothesizing after the results are known.” The post prompted a small group of skeptics to take a hard look at Mr. Wansink’s past scholarship. Their analysis, published in January 2017, turned up an astonishing variety and quantity of errors in his statistical procedures and data.”

rossdavidh 8y ago
Oh, if only we had a system to pass articles that are submitted to scientific journals to some sort of expert committee, that could look at it and see if it was likely to be valid. We could call it, I don't know, "peer checkup", or "expert review", or...oh never mind.
- ble 8y ago
  Peer review can't fix broken incentives.
- sn41 8y ago
  The issue is that peer review never pays off directly. The time spent in reviews mostly goes down a black hole. Editorships are awarded based on publication record, and not quality of reviews. As a result, more and more, reviews will tend to be shallow and uncritical.
  - rossdavidh 8y ago
    Truth.
soperj 8y ago
>The post prompted a small group of skeptics to take a hard look at Mr. Wansink’s past scholarship It's funny that this is basically the peer-review version of harking.
- theshadowknows 8y ago
  You just wrecked my mind
- pcrh 8y ago
  It's also pretty close to the Baysian approach. The difference being that in the Baysian approach one is "permitted" to revise the hypothesis as new data is acquired.
  - antidesitter 8y ago
    Correction: Bayesian, not Baysian.
kradroy 8y ago
I've never heard the term "HARKing" before. This is exactly what my advisor had me do on our experiments that couldn't confirm our initial hypotheses. I knew it was wrong because we set up the experiments in a way to eliminate confounding variables. So any conclusions drawn on data for hypotheses outside of those would be suspect. The experiments we ran were costly in time to do because we were using feedback from students using our learning system over a semester. I quit working with her because of that, and also because she was a total asshole. I refuse to donate to my university until she's no longer on the faculty. I don't want her benefiting from my hard-earned money.
- MrEfficiency 8y ago
  This is why I believe in Engineering and not the modern "Academia Science". Totally corrupt, Ive seen it when I talked to professors about getting my PhD. You prove what they want proven. One is going to have millions of customers using it, the other is going to have a few biased professors review the data.
  - vkou 8y ago
    If you think this is any different from how industry works, you have not spent enough time in industry. After-the-fact-hypothesis is constantly used to retroactively justify business decisions.
    - MrEfficiency 8y ago
      I have 10 years in Industry, testing hypothesis is testing. Building a product that is bad because 'X person wanted it' is bad Capitalism and will be punished. Academia doesnt have this same customer focused delivery.
      - rossdavidh 8y ago
        Hmmm...I think it depends on how fast the feedback loop of bad sales is. There are lots of cases where somebody inside a corporation wants the answer to be "X", because that's what they had advocated. It may be worse for the corporation as a whole, but it's better for that guy's career, and if the hit to the corporation is not immediately visible.
nonbel 8y ago
He also shamed a post-doc who refused to do it: https://web.archive.org/web/20170312041524/http:/www.brianwansink.com/phd-advice/the-grad-student-who-never-said-no https://web.archive.org/web/20170312041524/http:/www.brianwa...
- sampo 8y ago
  "In the meantime, this Turkish woman's resume will always have the five papers below." Turns out it won't: 2 of the 5 papers have been retracted. https://www.frontiersin.org/articles/10.3389/fpsyg.2017.02140/full https://www.frontiersin.org/articles/10.3389/fpsyg.2017.0214... https://bmcnutr.biomedcentral.com/articles/10.1186/s40795-017-0195-6 https://bmcnutr.biomedcentral.com/articles/10.1186/s40795-01...

sremani 8y ago

We replace people who are curious about science with people who are good at getting grants. Iron law of bureaucracy at work.

RcouF1uZ4gsC 8y ago
"If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible result being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate a different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want one kind of research, but, if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is to pay him for doing something else and give him enough leisure to do research for the love of it." Quote attributed to J.J. Thompson via a previous HN comment https://news.ycombinator.com/item?id=14022158 https://news.ycombinator.com/item?id=14022158
- iguy 8y ago
  You may enjoy the Connes interview linked here, discussing (in the 2nd half) French vs US vs Soviet systems: http://infoproc.blogspot.com/2018/09/the-french-way-alain-connes-interview.html http://infoproc.blogspot.com/2018/09/the-french-way-alain-co...
- ataturk 8y ago
  That quote is very profound. I know from my own doctoral research how difficult innovation is and how stressful that becomes. I want to also have a personal life outside of just working and what I saw at my university was unhealthy to the extreme. I got my doctorate, but I quit academia. You only get one life to live and I couldn't see why I should work endless hours for shit wages, worrying about obtaining funding year in and year out, dealing with the terrible communists who run US campuses these days, and so on. It was a completely undesirable lifestyle with only a few redeeming qualities. Bottom line, for me it was less headache to become a software engineer and take longer vacations once in awhile. I would rather do hard research or R&D, but it is what it is.

adamc 8y ago

Eh, paywall.

elektor 8y ago
Install this extension to use Outline. https://chrome.google.com/webstore/detail/outline-read-without-clut/daoolpmoieinofbnddaofhkhmbagfmnj?hl=en-US https://chrome.google.com/webstore/detail/outline-read-witho...
- ur-whale 8y ago
  Only works on Chrome, so not an option.
- prairiedock 8y ago
  You don't need the extension to get the benefit. Just go to https://outline.com https://outline.com
dang 8y ago
Please don't post unsubstantive comments here. Re paywalls: if there's a workaround, it's ok. Users usually post workarounds in the thread. This is in the FAQ at https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html and there's more explanation here: https://news.ycombinator.com/item?id=10178989 https://news.ycombinator.com/item?id=10178989 https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix&page=0&query=by:dang%20paywall https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...

sn41 8y ago

http://archive.is/II8xf http://archive.is/II8xf

misiti3780 8y ago
off topic, but does anyone know how the technology behind archive.is works and how they make money (assuming they do). it seem pretty impressive to me that it always gets the article right.
- cheeko1234 8y ago
  The site is a commercial enterprise, and as such can go kaputt at any given point, especially if it does not find a lucrative business model. Although it's not a strong indication of long-term issues; in October 2016 the site "made transparent" the server costs, and started to accept donations. A weekly crowdfunded target of $800[4] is set to maintain the site. Prior to this, the site actively refused donations. A donation link took the user to an animal shelter donation page. Stated in January 2017, through donations the site only receives "more than $1.50 every day, enough for a bowl of phở". Src: https://www.archiveteam.org/index.php/Archive.is https://www.archiveteam.org/index.php/Archive.is
  - true_religion 8y ago
    I thought archive.is was part of the internet archive project which is a public library of California.
    - sp332 8y ago
      Nope, archive.org is a whole different organization. Also note that the "Wayback Machine" is only one project at the Internet Archive. They also have 78 RPM records, video games, tons of books, CD's, etc. https://en.wikipedia.org/wiki/Lists_of_Internet_Archive%27s_collections https://en.wikipedia.org/wiki/Lists_of_Internet_Archive%27s_... You can donate at https://archive.org/donate https://archive.org/donate Edit: Archive Team is also a separate group.
  - 8y ago

misiti3780 8y ago

also relevant: https://andrewgelman.com/2016/12/15/hark-hark-p-value-heavens-gate-sings/ https://andrewgelman.com/2016/12/15/hark-hark-p-value-heaven...

sampo 8y ago

Also e.g. The Atlantic wrote about this 2 days ago: https://www.theatlantic.com/health/archive/2018/09/what-is-food-science/571105/ https://www.theatlantic.com/health/archive/2018/09/what-is-f...

fmajid 8y ago

The guy has a BS in Business Administration, a MA In journalism and a PhD in Marketing. He is a professor of Marketing at Cornell's School of Economics. In what universe would such a person be called a scientist and not a huckster doing what hucksters do?

wmf 8y ago
I guess he has marketing down to a science.
- helpfulTroll 8y ago
  Trolling is a art.
qubax 8y ago
"Social science". The term science has been appropriated by so many pseudosciences that it has lost any meaning. Because real science has such a good reputation, all the academic frauds have decided to link their field to science to enhance their own reputation. There is political "science", legal "science", economics, etc. Here is richard feynmann's poignant take on the matter. BTW, it has gotten way worse since feynmann's interview. https://www.youtube.com/watch?v=tWr39Q9vBgo https://www.youtube.com/watch?v=tWr39Q9vBgo
- mhuffman 8y ago
  Also watching a youtube video on Javascript makes you a "Software Engineer". Bit of a slap to what an actual engineer (in any concentration) has to go through to get a degree.
- deleted 8y ago
  [deleted]
- 7402 8y ago
  See also "Harary's Law" <https://quoteinvestigator.com/2015/06/26/not-science/> https://quoteinvestigator.com/2015/06/26/not-science/>, which says that any field that has the word “science” in its name is guaranteed thereby not to be a science.

DanielleMolloy 8y ago

Starting to get mixed feelings about these cases. Impostors and fraudsters certainly need to be exposed. But publicly ripping apart few individuals who, together with their peer reviewers, in some cases may simply not know better (e.g. because their whole field has low standards – nutrition science seems to be one of these fields –; or because they are inexperienced and their co-authors and supervisors don't care about details and correct science and just want to have their names on the paper) can not be the solution for an overall malfunctioning scientific communication and incentive system, or for bad methods that are used by entire fields. The current situation also invites all sorts of abuse. Need to say that certain parts of the open science movement appear to be a bit angry, not to say sadistic. Wansinks case was one that was ripped apart on Twitter. Wansinks research has received lots of media attention before - would his statistical methods have been investigated and debunked without that? How many Wansinks are out there, still having their job, continuing to apply bad methods (while not committing active fraud), never to be called out for their mistakes? I've been in Japan during the Obokata stem cell case. Her public apology was broadcasted on national TV. In konbinis you could see various tabloid newspapers with her name and her crying face on the cover for weeks. There seemed to be different mechanisms at play than a genuine interest in correct science. One of Obokata's supervisors, Yoshiki Sasai, committed suicide as a consequence of the case.

nonbel 8y ago
These people ostracize and hassle those who try to do a honest job and point out the issues with their methods. They also drive the people trying to do a good job out of the field because they can't compete with the fake productivity people like Wansink enjoyed. Also, the unnecessary suffering caused by this behavior is basically unimaginable. Who knows what advances we would have made if 90+% of resources dedicated to science during this golden era of cheap energy hadn't been wasted? We can't even estimate that. But yea, Wansink is just the tip of the iceberg. I've been saying for awhile that future generations will need to double check everything since about 1940, because the professional researchers have not been checking each others work, and have been using known incorrect methods to draw conclusions (NHST).
jrumbut 8y ago
I think in situations like this an idea worth considering is a truth and reconciliation process. The idea is everyone is given a year or so to go back, identify anything that they may have p-hacked or otherwise been unethical, and if they are radically transparent and work to correct the record then they can continue on reputation and position intact. It requires some serious compromise, it's imperfect, but it may be the best way to get the truth and prevent an ugly process from dragging on indefinitely.