13 ms·

I Fooled Millions Into Thinking Chocolate Helps Weight Loss

205 points by brown-dragon 11y ago

oconnor663 11y ago
http://slatestarcodex.com/2015/05/30/that-chocolate-study/ http://slatestarcodex.com/2015/05/30/that-chocolate-study/ > For me, the takeaway from this affair is that there is no one-size-fits-all solution to make statistics impossible to hack. Getting rid of p-values is appropriate sometimes, but not other times. Demanding large sample sizes is appropriate sometimes, but not other times. Not trusting silly conclusions like “chocolate causes weight loss” works sometimes but not other times. At the end of the day, you have to actually know what you’re doing. Also, try to read more than one study.
- allendoerfer 11y ago
  Another good indicator is, that in Germany Bild and the Huffington Post reported about it. You generally want to avoid to write a story based on these two. Everybody knows, that Bild consists of sex, hate, tits and the weather forecast as the Ärzte put it in a song mocking people that get their information from that paper.
- NyxWulf 11y ago
  For me the take away is that non-trivial independent replication of results still stands as the gold standard for experimentation. P-hacking shows up clearly when the experiment is replicated. P-values in my experience are only a starting point for further experiments. Take the cold-fusion debacle. When independent groups attempted to replicate the results they were unable to do so. Of course the other big take away is that people really don't understand statistics at a basic level, even the "scientific reporters".
  - digi_owl 11y ago
    In essence "science" reporters have made it their business to not understand statistics. That way they can turn weak correlations into strong causations, rake in the eyeballs, and then go "oops" whenever someone points out problems.
  - mcguire 11y ago
    Replication is hard and expensive, particularly in the medical field, and does not get the researcher any real science points. Besides, it's more fun to do a meta-analysis on a flock of sorta-vaguely-similar studies.
  - hootener 11y ago
    > For me the take away is that non-trivial independent replication of results still stands as the gold standard for experimentation. Agreed. Too bad funding agencies rarely if ever give you the money to do it :(. The takeaway here is that statistics is arguably one of the most nuanced quantitative fields out there. It's really easy to shoot yourself in the foot, particularly with p-values. I think every statistical test has its place, but my personal favorite lambasting of p-value testing is Steiger and Fouladi's 1997 paper on non-centrality interval estimation [1]. As an aside, Steiger was my graduate statistics professor several years ago, and probably the primary reason I know this paper even exists. If you enjoy the harsh treatment of significance testing in the paper, just imagine hearing it straight from the horse's mouth during lecture :). [1]: http://www.statpower.net/Steiger%20Biblio/Steiger&Fouladi97.PDF http://www.statpower.net/Steiger%20Biblio/Steiger&Fouladi97....
- reddog 11y ago
  > Also, try to read more than one study. Even if I had the time to read several studies and expertise to understand them, I couldn't afford it. Out of curiosity I just went to the Journal of Nutrition (I know nothing about it, picked that journal randomly) and tried to download one random issue. It was $273. Going to multiple original sources isn't practical for most of us. I doubt that the typical mainstream journalist writing a popular story on a scientific subject could justify this expense to her boss either.
  - leephillips 11y ago
    If you need regular access to the scientific literature you have or make an arrangement with an institution. You don't pay per article or issue. It is a real problem for unaffiliated or freelance folks, though.
    - _delirium 11y ago
      Even as a freelance person it's not a huge problem for me. It can be annoying, but if I'm doing real work rather than just looking up one article on a whim, it's not a showstopper. Mostly I just use the local university library. Non-affiliates can't check out books, but anyone can walk in and get on the wifi. And once you're on their wifi, you get whatever database access they've negotiated for their IP range. Not every university library allows this, but quite a lot do.
  - takluyver 11y ago
    I'd hope that news outlets covering any kind of science pay for access to at least some major journals as a matter of course. The subscription prices are probably much lower per issue than the price to buy an issue individually. Most journals also show you abstracts for free, which are overviews of what each paper says. If you find a paper that you want to read in full, email the author - most scientists are very happy to know that people are interested in their work, so they'll give out copies to anyone who asks. This isn't to defend the paywalls - I think open access is important. But I don't think the paywalls are a valid excuse for journalists.
seszett 11y ago
By the way, this is the documentary for which this was done: In French: http://www.arte.tv/guide/fr/052711-000/pour-maigrir-mangez-du-chocolat http://www.arte.tv/guide/fr/052711-000/pour-maigrir-mangez-d... And in German: http://www.arte.tv/guide/de/052711-000/schlank-durch-schokolade http://www.arte.tv/guide/de/052711-000/schlank-durch-schokol...
hartator 11y ago
Finally, the most accurate reactions come from the web itself: http://forum.bodybuilding.com/showthread.php?t=166912491 http://forum.bodybuilding.com/showthread.php?t=166912491 > study invalid. no control for calories at all. > Why are calories not counted on any of the individuals? > Free reign is not a way to go about doing a study. They might very well have, we don't know. We all know cocoa has weight loss properties (Theobromine), but to do such a poorly constructed study was pointless.
Jedd 11y ago
'everyone'? I don't recall being fooled into thinking this. Slightly less flippantly - the assertion, conclusion, and title are wrong.
jessaustin 11y ago
It's difficult to understand why anyone would ever pay any attention to nutrition "science". Everything is wrong. TFA complimented the meatheads on some bodybuilding site for their skepticism. People who pay very close attention to their own habits and results are probably going to be more right about their own nutrition than all the scientists in the world put together.
- vezzy-fnord 11y ago
  I upvoted you because even though you're too far in outright rejecting empiricism about nutrition, the state of nutrition science is indeed quite poor and shrouded in mythology.
  - jessaustin 11y ago
    Of course one can say basic things like "average daily calories should be between 800 and 3000", but even a range that covers 95% of the population isn't right for everyone, and isn't precise enough to really help anyone. Individuals differ not only in terms of lifestyle, which conceivably could be studied, but also in terms of genetics, which no study to date has examined in more than a rudimentary way. The various statistics sins that we complain about on HN are exacerbated when important factors like genetic background are left unstudied. And we haven't even mentioned the fact that most published (and unpublished) nutrition studies are funded by the food and supplement industries! A rejection of a particularly poor practice of science is not a rejection of science. I doubt we'll see decent nutrition science until we're all equipped with a network of bloodstream nanobots that measures the actual physical response we each display to nutritional input.
- beachstartup 11y ago
  > the meatheads on some bodybuilding site that's not very nice. a lot of people on HN lift weights.
  - pavlov 11y ago
    I wouldn't have a problem if posters on said bodybuilding forum referred to HN as "the nerds on some programming site". It's the truth, isn't it?
SmokyBourbon 11y ago
Skepticism of science is healthy. Prior to 1987, temperature recordings were not taken at the same time of day. Any time someone cites historical temperature data, they are referring to adjusted temperatures. This is what happened when NASA and NOAA made changes to those adjustments in 1998: http://i.imgur.com/UGk9YTY.gifv http://i.imgur.com/UGk9YTY.gifv Now stop calling climate skeptics "anti-science".
reamworks 11y ago
This is a particularly good area to promote fake/weak conclusions because "fatties gonna fat." Obese people, suffering from food addictions, have no basis in reality. Simply counting calories, truthfully, will make anyone lose weight. But it's not politically correct to say this. I was kicked off of reddit for _thinking_ this.
- orf 11y ago
  No, /r/fatpeoplehate was kicked off Reddit because it was a mockery of a subreddit full of (you guessed it) hate and personal abuse, with the mods encouraging it.
  - reamworks 11y ago
    Found the fatty!
- navait 11y ago
  Your personal cause was shown the door on reddit. Please don't think that means it's welcome here, or on other websites.
  - reamworks 11y ago
    Found the fatty!
reamworks 11y ago
This is a particularly good area to promote fake/weak conclusions because "fatties gonna fat." Obese people, suffering from food addictions, have no basis in reality. Simply counting calories, truthfully, will make anyone lose weight. But it's not politically correct to say this. I was kicked off of reddit for _thinking_ this.
ddebernardy 11y ago
The real story here is that, even after being shown proof of that it was a hoax all along, a chunk of the same population will continue to hold that eating chocolate helps to lose weight.
- javajosh 11y ago
  I laughed. But more seriously, I wonder if Bohannon's media experiment would have succeeded if you change a detail. E.g. if it was about something truly gross, like drinking curdled milk to lose weight. Journalists want to tell people what they want to hear, because people will tend to buy the paper that says what they are already thinking (weird, I know). It's human nature that people are more skeptical about claims they don't like.
  - tdicola 11y ago
    Imagine how mad the people in the study would be after they found out it was a sham. "I drank curdled milk for 3 weeks so you could get your big break!"
    - javajosh 11y ago
      That's an interesting angle even for this study - how do the original participants feel about it I wonder? I'd guess some mixture of amused and cynical.
- CyberDildonics 11y ago
  It is a lot easier to convince someone of something they want to believe than of something they don't.
  - wpietri 11y ago
    When I read this, I nodded. But now you've got me wondering if I just want to believe you.
ak217 11y ago
Actually, the takeaway for me is that John Bohannon behaved unethically and made a bunch of invalid conclusions.
- javajosh 11y ago
  Bohannon tested a system, it failed, and he reported exactly how he tested the system and how it failed. 'The system' in this case is the global food science and health journalism industry. He found that ignorance and laziness is rampant, that journal peer review is non-existent, and skepticism about results and methods is non-existent. He has done a tremendous public service exposing the flaws in a system. With any luck consumers of these information sources will take everything with an enormous, skeptical grain of salt, because the last and best line of defense is your own skepticism critical thinking. You can't delegate thinking.
  - leephillips 11y ago
    He did not find that "journal peer review is non-existent". He targeted a low-quality journal that does not send out papers for review to make sure he could claim that the research was "published" when sending out his press releases. That's explicit in the article, which you might find an interesting read
  - alexhill 11y ago
    > that journal peer review is non-existent This isn't about peer review, because the paper was knowingly submitted only to journals known not to conduct peer review. Quoting from the article: > We needed to get our study published pronto, but since it was such bad science, we needed to skip peer review altogether. Conveniently, there are lists of fake journal publishers. And later: > The new publisher’s CEO, Carlos Vasquez, emailed Johannes to let him know that we had produced an “outstanding manuscript,” and that for just 600 Euros it “could be accepted directly in our premier journal.” This is really more about science journalism than about science. These kinds of journals are difficult for a lay person to tell apart from actual, scientific journals, but distinguishing between the two is exactly the kind of job a competent science journalist would do. Inept science journalists, on the other hand, give these fake journals a reason to exist by citing them unchallenged.
spacehome 11y ago
@reamworks [Your post is flagged/dead, so I'm replying here. I hope you see it.] The relationship between counting calories and weight loss is not quite as simple as the formula [Calories Injested] - [Calories Burned] = [Delta body fat] might indicate. Details here: http://slatestarcodex.com/2015/01/12/the-physics-diet/ http://slatestarcodex.com/2015/01/12/the-physics-diet/
Mikeb85 11y ago
To be honest, chocolate may help in an indirect way... If someone's on a diet, and never gets any 'reward', they might be tempted to cheat or give up. If they're told a little chocolate every day is good, they'll stick with their diet, have their chocolate, and forego worse 'cheat' items (even though chocolate has plenty of fat and sugar, the amount you consume before you're content is relatively small compared to say, ice-cream, cookies, etc...). Studies (and anecdotes) have shown that chocolate makes people happy, so that little happiness at the end might just help people stay on their diet and not stress out (stress can cause the body to store fat). Maybe it's something like the French paradox... The study was pretty funny though, and I appreciate their honesty. And it goes to show why pretty much all research on fad diets is nonsense. Still though, if you choose between drinking a gallon of pop per day or chocolate, chocolate wins. So maybe their 'research' didn't go to waste.
bernardlunn 11y ago
My takeaway is you can always sell people something they want to buy.
tgb 11y ago
Did any major news organizations retract their articles or apologize?
- edc117 11y ago
  Do they ever? This is likely one of many reasons for eroding confidence in those same organizations.
deleted 11y ago
[deleted]
bjourne 11y ago
It's not news that the peer-review system isn't safe from exploits. People have been using Markov chains to generate studies which have then been published. And journalists uncritically picks up everything they can. All functions in society are based on trust and are easily exploitable if a person is not behaving trustworthy. So making a fake study about chocolate and getting lots of magazines to publish it I think is a little unethical. It's not news, and the reason most people don't do it is because they want to act trustworthy. For example, someone could, if they wanted, submit patches to some free software projects which would introduce awful security holes and get them merged or just write shit on Wikipedia.
- thomasahle 11y ago
  It wasn't actually a fake study. That's the point. They used the same statistical methods as too many 'honest' groups out there, and got results thereafter.
  - bjourne 11y ago
    I disagree. The study was fake because the intent was to deceive. He used flawed methods, but could have used 10 times as sophisticated methods and still would have been able to publish completely invalid results.
    - olifante 11y ago
      I wasn't a fake study, it was a dishonest one.
- munro 11y ago
  To continue this line of though, distributed computation is hard problem, and that's just computers. Here, people have to do real work to statically verify a paper's procedure, which is very cognitive overloading. Imagine reviewing a PR and deciding with certainty if it will work or not. Even then they should also "recompute" the experiment to completely verify it. Of course add human error on top of all this. It seems there's a problem here, the article made it seem anyone can produce a bad study for lulz, and get everyone's attention. What is more scary is people doing it for vested interests. Maybe there's something that could be learned from projects like BIONC, Wikipedia, or Bitcoin, but better science is going to cost more money, and require more cultural attention. Not sure how to fix those things :)
liquidcool 11y ago
On the flip side, you have some well-heeled corporations and industry groups working to discredit good science that's bad for them. Would be interesting to read a piece on how that's done.
- GhotiFish 11y ago
  Unfortunately the investment to make good science just to get it discredited by interested parties isn't as cost effective.
  - liquidcool 11y ago
    I don't think you need to go that far, you just need a PR person with a conscience to spill the beans on what they did (or dissect what others have done).

205 points by brown-dragon 11y ago

http://slatestarcodex.com/2015/05/30/that-chocolate-study/ http://slatestarcodex.com/2015/05/30/that-chocolate-study/ > For me, the takeaway from this affair is that there is no one-size-fits-all solution to make statistics impossible to hack. Getting rid of p-values is appropriate sometimes, but not other times. Demanding large sample sizes is appropriate sometimes, but not other times. Not trusting silly conclusions like “chocolate causes weight loss” works sometimes but not other times. At the end of the day, you have to actually know what you’re doing. Also, try to read more than one study.

allendoerfer 11y ago
Another good indicator is, that in Germany Bild and the Huffington Post reported about it. You generally want to avoid to write a story based on these two. Everybody knows, that Bild consists of sex, hate, tits and the weather forecast as the Ärzte put it in a song mocking people that get their information from that paper.
NyxWulf 11y ago
For me the take away is that non-trivial independent replication of results still stands as the gold standard for experimentation. P-hacking shows up clearly when the experiment is replicated. P-values in my experience are only a starting point for further experiments. Take the cold-fusion debacle. When independent groups attempted to replicate the results they were unable to do so. Of course the other big take away is that people really don't understand statistics at a basic level, even the "scientific reporters".
- digi_owl 11y ago
  In essence "science" reporters have made it their business to not understand statistics. That way they can turn weak correlations into strong causations, rake in the eyeballs, and then go "oops" whenever someone points out problems.
- mcguire 11y ago
  Replication is hard and expensive, particularly in the medical field, and does not get the researcher any real science points. Besides, it's more fun to do a meta-analysis on a flock of sorta-vaguely-similar studies.
- hootener 11y ago
  > For me the take away is that non-trivial independent replication of results still stands as the gold standard for experimentation. Agreed. Too bad funding agencies rarely if ever give you the money to do it :(. The takeaway here is that statistics is arguably one of the most nuanced quantitative fields out there. It's really easy to shoot yourself in the foot, particularly with p-values. I think every statistical test has its place, but my personal favorite lambasting of p-value testing is Steiger and Fouladi's 1997 paper on non-centrality interval estimation [1]. As an aside, Steiger was my graduate statistics professor several years ago, and probably the primary reason I know this paper even exists. If you enjoy the harsh treatment of significance testing in the paper, just imagine hearing it straight from the horse's mouth during lecture :). [1]: http://www.statpower.net/Steiger%20Biblio/Steiger&Fouladi97.PDF http://www.statpower.net/Steiger%20Biblio/Steiger&Fouladi97....
reddog 11y ago
> Also, try to read more than one study. Even if I had the time to read several studies and expertise to understand them, I couldn't afford it. Out of curiosity I just went to the Journal of Nutrition (I know nothing about it, picked that journal randomly) and tried to download one random issue. It was $273. Going to multiple original sources isn't practical for most of us. I doubt that the typical mainstream journalist writing a popular story on a scientific subject could justify this expense to her boss either.
- leephillips 11y ago
  If you need regular access to the scientific literature you have or make an arrangement with an institution. You don't pay per article or issue. It is a real problem for unaffiliated or freelance folks, though.
  - _delirium 11y ago
    Even as a freelance person it's not a huge problem for me. It can be annoying, but if I'm doing real work rather than just looking up one article on a whim, it's not a showstopper. Mostly I just use the local university library. Non-affiliates can't check out books, but anyone can walk in and get on the wifi. And once you're on their wifi, you get whatever database access they've negotiated for their IP range. Not every university library allows this, but quite a lot do.
- takluyver 11y ago
  I'd hope that news outlets covering any kind of science pay for access to at least some major journals as a matter of course. The subscription prices are probably much lower per issue than the price to buy an issue individually. Most journals also show you abstracts for free, which are overviews of what each paper says. If you find a paper that you want to read in full, email the author - most scientists are very happy to know that people are interested in their work, so they'll give out copies to anyone who asks. This isn't to defend the paywalls - I think open access is important. But I don't think the paywalls are a valid excuse for journalists.

seszett 11y ago

By the way, this is the documentary for which this was done: In French: http://www.arte.tv/guide/fr/052711-000/pour-maigrir-mangez-du-chocolat http://www.arte.tv/guide/fr/052711-000/pour-maigrir-mangez-d... And in German: http://www.arte.tv/guide/de/052711-000/schlank-durch-schokolade http://www.arte.tv/guide/de/052711-000/schlank-durch-schokol...

hartator 11y ago

Finally, the most accurate reactions come from the web itself: http://forum.bodybuilding.com/showthread.php?t=166912491 http://forum.bodybuilding.com/showthread.php?t=166912491 > study invalid. no control for calories at all. > Why are calories not counted on any of the individuals? > Free reign is not a way to go about doing a study. They might very well have, we don't know. We all know cocoa has weight loss properties (Theobromine), but to do such a poorly constructed study was pointless.

Jedd 11y ago

'everyone'? I don't recall being fooled into thinking this. Slightly less flippantly - the assertion, conclusion, and title are wrong.

jessaustin 11y ago

It's difficult to understand why anyone would ever pay any attention to nutrition "science". Everything is wrong. TFA complimented the meatheads on some bodybuilding site for their skepticism. People who pay very close attention to their own habits and results are probably going to be more right about their own nutrition than all the scientists in the world put together.

vezzy-fnord 11y ago
I upvoted you because even though you're too far in outright rejecting empiricism about nutrition, the state of nutrition science is indeed quite poor and shrouded in mythology.
- jessaustin 11y ago
  Of course one can say basic things like "average daily calories should be between 800 and 3000", but even a range that covers 95% of the population isn't right for everyone, and isn't precise enough to really help anyone. Individuals differ not only in terms of lifestyle, which conceivably could be studied, but also in terms of genetics, which no study to date has examined in more than a rudimentary way. The various statistics sins that we complain about on HN are exacerbated when important factors like genetic background are left unstudied. And we haven't even mentioned the fact that most published (and unpublished) nutrition studies are funded by the food and supplement industries! A rejection of a particularly poor practice of science is not a rejection of science. I doubt we'll see decent nutrition science until we're all equipped with a network of bloodstream nanobots that measures the actual physical response we each display to nutritional input.
beachstartup 11y ago
> the meatheads on some bodybuilding site that's not very nice. a lot of people on HN lift weights.
- pavlov 11y ago
  I wouldn't have a problem if posters on said bodybuilding forum referred to HN as "the nerds on some programming site". It's the truth, isn't it?

SmokyBourbon 11y ago

Skepticism of science is healthy. Prior to 1987, temperature recordings were not taken at the same time of day. Any time someone cites historical temperature data, they are referring to adjusted temperatures. This is what happened when NASA and NOAA made changes to those adjustments in 1998: http://i.imgur.com/UGk9YTY.gifv http://i.imgur.com/UGk9YTY.gifv Now stop calling climate skeptics "anti-science".

reamworks 11y ago

This is a particularly good area to promote fake/weak conclusions because "fatties gonna fat." Obese people, suffering from food addictions, have no basis in reality. Simply counting calories, truthfully, will make anyone lose weight. But it's not politically correct to say this. I was kicked off of reddit for _thinking_ this.

orf 11y ago
No, /r/fatpeoplehate was kicked off Reddit because it was a mockery of a subreddit full of (you guessed it) hate and personal abuse, with the mods encouraging it.
- reamworks 11y ago
  Found the fatty!
navait 11y ago
Your personal cause was shown the door on reddit. Please don't think that means it's welcome here, or on other websites.
- reamworks 11y ago
  Found the fatty!

reamworks 11y ago

ddebernardy 11y ago

The real story here is that, even after being shown proof of that it was a hoax all along, a chunk of the same population will continue to hold that eating chocolate helps to lose weight.

javajosh 11y ago
I laughed. But more seriously, I wonder if Bohannon's media experiment would have succeeded if you change a detail. E.g. if it was about something truly gross, like drinking curdled milk to lose weight. Journalists want to tell people what they want to hear, because people will tend to buy the paper that says what they are already thinking (weird, I know). It's human nature that people are more skeptical about claims they don't like.
- tdicola 11y ago
  Imagine how mad the people in the study would be after they found out it was a sham. "I drank curdled milk for 3 weeks so you could get your big break!"
  - javajosh 11y ago
    That's an interesting angle even for this study - how do the original participants feel about it I wonder? I'd guess some mixture of amused and cynical.
CyberDildonics 11y ago
It is a lot easier to convince someone of something they want to believe than of something they don't.
- wpietri 11y ago
  When I read this, I nodded. But now you've got me wondering if I just want to believe you.

ak217 11y ago

Actually, the takeaway for me is that John Bohannon behaved unethically and made a bunch of invalid conclusions.

javajosh 11y ago
Bohannon tested a system, it failed, and he reported exactly how he tested the system and how it failed. 'The system' in this case is the global food science and health journalism industry. He found that ignorance and laziness is rampant, that journal peer review is non-existent, and skepticism about results and methods is non-existent. He has done a tremendous public service exposing the flaws in a system. With any luck consumers of these information sources will take everything with an enormous, skeptical grain of salt, because the last and best line of defense is your own skepticism critical thinking. You can't delegate thinking.
- leephillips 11y ago
  He did not find that "journal peer review is non-existent". He targeted a low-quality journal that does not send out papers for review to make sure he could claim that the research was "published" when sending out his press releases. That's explicit in the article, which you might find an interesting read
- alexhill 11y ago
  > that journal peer review is non-existent This isn't about peer review, because the paper was knowingly submitted only to journals known not to conduct peer review. Quoting from the article: > We needed to get our study published pronto, but since it was such bad science, we needed to skip peer review altogether. Conveniently, there are lists of fake journal publishers. And later: > The new publisher’s CEO, Carlos Vasquez, emailed Johannes to let him know that we had produced an “outstanding manuscript,” and that for just 600 Euros it “could be accepted directly in our premier journal.” This is really more about science journalism than about science. These kinds of journals are difficult for a lay person to tell apart from actual, scientific journals, but distinguishing between the two is exactly the kind of job a competent science journalist would do. Inept science journalists, on the other hand, give these fake journals a reason to exist by citing them unchallenged.

spacehome 11y ago

@reamworks [Your post is flagged/dead, so I'm replying here. I hope you see it.] The relationship between counting calories and weight loss is not quite as simple as the formula [Calories Injested] - [Calories Burned] = [Delta body fat] might indicate. Details here: http://slatestarcodex.com/2015/01/12/the-physics-diet/ http://slatestarcodex.com/2015/01/12/the-physics-diet/

Mikeb85 11y ago

To be honest, chocolate may help in an indirect way... If someone's on a diet, and never gets any 'reward', they might be tempted to cheat or give up. If they're told a little chocolate every day is good, they'll stick with their diet, have their chocolate, and forego worse 'cheat' items (even though chocolate has plenty of fat and sugar, the amount you consume before you're content is relatively small compared to say, ice-cream, cookies, etc...). Studies (and anecdotes) have shown that chocolate makes people happy, so that little happiness at the end might just help people stay on their diet and not stress out (stress can cause the body to store fat). Maybe it's something like the French paradox... The study was pretty funny though, and I appreciate their honesty. And it goes to show why pretty much all research on fad diets is nonsense. Still though, if you choose between drinking a gallon of pop per day or chocolate, chocolate wins. So maybe their 'research' didn't go to waste.

bernardlunn 11y ago

My takeaway is you can always sell people something they want to buy.

tgb 11y ago

Did any major news organizations retract their articles or apologize?

edc117 11y ago
Do they ever? This is likely one of many reasons for eroding confidence in those same organizations.

deleted 11y ago

[deleted]

bjourne 11y ago

It's not news that the peer-review system isn't safe from exploits. People have been using Markov chains to generate studies which have then been published. And journalists uncritically picks up everything they can. All functions in society are based on trust and are easily exploitable if a person is not behaving trustworthy. So making a fake study about chocolate and getting lots of magazines to publish it I think is a little unethical. It's not news, and the reason most people don't do it is because they want to act trustworthy. For example, someone could, if they wanted, submit patches to some free software projects which would introduce awful security holes and get them merged or just write shit on Wikipedia.

thomasahle 11y ago
It wasn't actually a fake study. That's the point. They used the same statistical methods as too many 'honest' groups out there, and got results thereafter.
- bjourne 11y ago
  I disagree. The study was fake because the intent was to deceive. He used flawed methods, but could have used 10 times as sophisticated methods and still would have been able to publish completely invalid results.
  - olifante 11y ago
    I wasn't a fake study, it was a dishonest one.
munro 11y ago
To continue this line of though, distributed computation is hard problem, and that's just computers. Here, people have to do real work to statically verify a paper's procedure, which is very cognitive overloading. Imagine reviewing a PR and deciding with certainty if it will work or not. Even then they should also "recompute" the experiment to completely verify it. Of course add human error on top of all this. It seems there's a problem here, the article made it seem anyone can produce a bad study for lulz, and get everyone's attention. What is more scary is people doing it for vested interests. Maybe there's something that could be learned from projects like BIONC, Wikipedia, or Bitcoin, but better science is going to cost more money, and require more cultural attention. Not sure how to fix those things :)

liquidcool 11y ago

On the flip side, you have some well-heeled corporations and industry groups working to discredit good science that's bad for them. Would be interesting to read a piece on how that's done.

GhotiFish 11y ago
Unfortunately the investment to make good science just to get it discredited by interested parties isn't as cost effective.
- liquidcool 11y ago
  I don't think you need to go that far, you just need a PR person with a conscience to spill the beans on what they did (or dissect what others have done).