18 ms·
“Mindless Eating,” or how to send an entire life of research into question
- js2 9y agoThe article makes Wansink seem negligent and/or incompetent. Also, the embedded "CBS This Morning" video is cringeworthy. It appears that the recommendations in his book amount to "study normal weight people; do what they do." Isn't that fraught with survivorship bias? Do the recommendations in his book control for people that have the same habits but still end up overweight? Anecdote: my household is a family of four. Myself and my wife, and our two teenage kids, daughter and son. Since we live together, the answers to the 10 questions on http://www.slimbydesign.com/get-scored/ http://www.slimbydesign.com/get-scored/ are the same for the household. My son and I are normal weight. My wife and daughter are overweight. Conclusion: weight has very little to do with the design of your kitchen. Caveat: this conclusion has not been peer reviewed and is not scientifically sound.
- closed 9y agoWansink strikes me as an extremely bright, creative researcher. His problem seems to be that he didn't collaborate with people who were strong methodologists. If anything, he reminds me of a lot of brilliant researchers, who are too confident in their methodological prowess. (And this is the outcome that should have been avoided through collaboration).
- killedbydeath 9y agoWell that's a lot of data you have there. I did some GLM analysis and clearly weight is correlated with gender. Let me go tell some journalists.
- npsimons 9y ago> Isn't that fraught with survivorship bias? It is, but it's not a completely invalid starting point. Even better is to study people who were once fat, but lost weight and keep it off. This is what the National Weight Control Registry does. It's similar to studies that have tried to pin down why marriages fail. Most studies focus on what went wrong in failed marriages, but some of the most helpful suggestions have come from what people who stay married do. > My son and I are normal weight. My wife and daughter are overweight. Do your wife and daughter eat the same portions as you and your son? It's a common complaint of women, who are on average shorter than men and therefore need fewer calories, that they can't eat as much as their significant others.
- js2 9y ago> Do your wife and daughter eat the same portions as you and your son No. I expand on that a bit here - https://news.ycombinator.com/item?id=14196631 https://news.ycombinator.com/item?id=14196631
- tryitnow 9y agoTragically, I bet this is all too common. Wansink's mistake was blogging about it. This emblematic of a larger problem with how science is practiced: the obsessive focus on p-value thresholds leads to irrational practices like trawling data for interesting "findings." But on a certain level Wansink was right: a data set is not completely worthless if it showed a null result. So we need to start thinking about how to communicate the value of data even when the null is not rejected. One way to do this is to encourage widespread sharing of data sets regardless of the outcome of the experiment. Maybe for a given study the data did not show a definitive result - but does the data point to potential future paths of study? Maybe another researcher could get ideas for new experiments.
- StavrosK 9y ago> a data set is not completely worthless if it showed a null result. Why is a null result worthless? I don't understand. It goes against my common sense that paying more for food doesn't make you eat more. The fact that this is not the case is valuable knowledge to me. I understand that it's not as glamorous as "paying more makes you eat less!", but it's still valuable knowledge that should be published.
- draw_down 9y agoI think you're saying the same thing as GP
- ad-hominem 9y ago> a data set is not completely worthless if it showed a null result. A null result is not worthless. A data set is not worthless if it showed a null result. Nobody is claiming a null result is worthless. Even the original article gives a good reason for publishing a null result.
- KingMob 9y agoNull results are "worthless" only in the sense that they're harder, if not impossible, to publish, and do less for your career. As a former neuroscientist, I had a unicorn data set of intracranial EEG data, and we'd spent so much time collecting it, that we were determined to find something. I left grad school before we found anything of interest. I believe my former prof eventually published something on it, but I analyzed the shit out of it, even knowing I was fishing, because so much time would have been lost to not use it. Fishing like this was one of the reasons I left. The pursuit of knowledge and the pursuit of your career don't align often enough. To me, what's unusual is not that Wansink pushed to keep analyzing the data, but that he was called out for it. Everyone seemed to be doing it when I was in academia.
- astrobase_go 9y agoI think what's needed is a strategy for making actual data analysis an easier and more visible component of the peer-review and publishing process. Submissions are typically sent to three people for review, and the decision to accept or reject is made based on the feedback or criticism given. If the reviewers are busy, overworked professors, they may not have the time to really perform a deep dive into the data and conduct an independent analysis. Furthermore, it's likely these submissions don't include enough data for a proper statistical review. This is the 21st century, where most if not all reputable journals have an online presence and submission portal. It would be great if authors had to upload their anonymized data sets in a common format (.csv?) that can easily be imported into statistical analysis packages (ex: Minitab, read into R, etc). Journals provide analysis and test recommendations, reviewers run the tests, upload their independent analysis results as part of the review process. The idea here is to produce something like an auditable paper trail. There has to be some sort of solution for this problem, especially in 2017. Hiding behind shitty (or worse, deceitful) data analysis shouldn't be possible.
- candiodari 9y agoInside universities, it does not seem realistic that this will lower people's opinions on the value of social science statistical research, or in the related fields of psychology (esp. management/organisational psychology), or even all of the humanities. That opinion is already "reject it all". Even when it comes to medicine that opinion is strongly represented. A few people even ascribe the placebo effect to massive and widespread statistical error.
- draw_down 9y agoThe Last Psychiatrist talked about this sometimes. The problem is not just that the original studies need to be rescinded, or discredited going forward. The problem is what do you do about everything that cited them, that took the knowledge in the faulty papers and used it as a foundation for more knowledge? The faulty ideas in the paper have permeated out to researchers and (sometimes) the broader culture generally, what do you do about that? http://thelastpsychiatrist.com/2009/10/the_problem_with_science_is_sc.html http://thelastpsychiatrist.com/2009/10/the_problem_with_scie...
- panglott 9y ago'Why did peer review not catch this? “Because peer review doesn’t do this,” Heathers told Ars. The point of peer review has always been for fellow scientists to judge whether a paper is of reasonable quality; reviewers aren't expected to perform an independent analysis of the data. ...In fact, without open data—something that’s historically been hit-or-miss—it would be impossible for peer reviewers to validate any numbers.' It doesn't seem unreasonable to think that the peer review process should include a statistician or someone who can review the statistics.
- jobu 9y agoThis was my thought exactly. "Wansink added another note to his blog post, acknowledging the problems and announcing that a statistician would be redoing the analyses." With all of the issues lately around replication of studies it seems like having a statistician review the math should be step one of any peer-review process.
- closed 9y agoInterestingly, a few years ago Nature planned to include statisticians... > as consultants on the rare papers in which there are serious issues about the statistical techniques used, at the editor's discretion and based on the referees' suggestions. I think the crux is that editors already have enough trouble finding reviewers, and see getting a statistician as often unreasonable. It's too bad, because the nature of statistical issues is that people tend to be unaware of them until a statistician (used broadly) points them out.
- deleted 9y ago[deleted]
- jm__87 9y agoFeel like we really approach the whole dieting process incorrectly. People eat to fulfill a need. They are either hungry and eat to no longer feel hungry or they eat because it produces positive feelings (whether this is to negate negative feelings or just to enhance your already good mood). Hunger is your body signalling your brain that you need nutrients. If you eat foods that cover all your nutrient requirements for a low number of calories and your body is taking up these nutrients properly, then you should have no problems. Thus it makes sense to inform people which foods they need to be eating as well as educate people on which disorders can cause poor nutrient uptake. If you suffer from stress and you eat high calorie foods to feel better, changing your diet is likely going to exacerbate your stress, not make you feel better. I feel this is the case for many who are overweight.. stress management strategies should be the first priority as you won't be able to handle changing your diet for the long term until you get your stress under control.
- temp246810 9y agoThis doesn't even begin to get into the effects different macronutrients have on you. It's a complicated subject that has reached religion-like levels of zealotry.
- ysavir 9y agoYes, but if we started acknowledging that everyone's different and functions on their own level, people will stop giving statisticians money! Who wants to live in that world? [/sarcasm]
- tonmoy 9y agoIf the academic world let scientists publish their validation of the null hypothesis, then they wouldn't go out of their way to "deep dive" into data to look for other hypothesis to prove!
- Animats 9y ago"Trawling through data, running lots of statistical tests, and looking only for significant results is bound to turn up some false positives." That's most of the field of macroeconomics. It's all data analysis, not controlled experimentation.
- mcguire 9y agoI was just about to ask, isn't that how data science works?
- beagle3 9y agoProper data science (for example, bayesian analysis done right) takes that into account, and would generally not score random finds as remarkable. But doing that is hard, so most people don't even try ... yes, practically, that's how modern data science works.
- cm2012 9y agoPSA I always post into weight threads: Every legitimate long term study of non surgical weight loss shows that it doesn't happen for the vast, vast majority of people. 1) ["In controlled settings, participants who remain in weight loss programs usually lose approximately 10% of their weight. However, one third to two thirds of the weight is regained within 1 year, and almost all is regained within 5 years. "](http://www.ncbi.nlm.nih.gov/pubmed/1580453 http://www.ncbi.nlm.nih.gov/pubmed/1580453) 2) Giant meta study of long term weight loss: ["Five years after completing structured weight-loss programs, the average individual maintained a weight loss of >3% of initial body weight."](http://ajcn.nutrition.org/content/74/5/579.full http://ajcn.nutrition.org/content/74/5/579.full) 3) Less Scientific: [Weight Watcher's Failure - "about two out of a thousand Weight Watchers participants who reached goal weight stayed there for more than five years."](https://fatfu.wordpress.com/2008/01/24/weight-watchers/ https://fatfu.wordpress.com/2008/01/24/weight-watchers/) 4) [The reason why it's impossible seems to be that although calories in < calories out works, the body of a fat person makes it extremely difficult psychologically to eat less.](http://www.nytimes.com/2012/01/01/magazine/tara-parker-pope-fat-trap.html?_r=2&pagewanted=all http://www.nytimes.com/2012/01/01/magazine/tara-parker-pope-...) This is borne out by the above data. 5) [The only thing that does seem to work in the long term is gastric surgery.](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1421028/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1421028/) Moreover, you won't find any reputable study on the web where the average person lost 10%+ of their body weight and kept it off for five years. Not even one.
- drewcrawford 9y ago> Moreover, you won't find any reputable study on the web where the average person lost 10%+ of their body weight and kept it off for five years. Not even one. Of course there is: http://www.nwcr.ws/Research/published%20research.htm http://www.nwcr.ws/Research/published%20research.htm. Here's a good question: other than the fact that these people lost weight, what is identifiably unusual about them?
- npsimons 9y agoWas going to link to the National Weight Control Registry, thanks! I'll just add that all those studies in GP seem to prove is that A) weight loss programs (especially fad diets) don't work and B) it is a psychological issue. There are plenty of people on MFP, /r/loseit or just counting calories themselves that have successfully kept off weight for years. I'm one of them. In case someone out there is serious about losing weight and not making excuses, here's how you calculate your actual TDEE: https://www.reddit.com/r/leangains/comments/2rv09z/this_is_how_you_calculate_your_tdee/ https://www.reddit.com/r/leangains/comments/2rv09z/this_is_h...
- badosu 9y agoThis is something that irritates me to the point of irrationality. Why so many people spend so much time and resources on $name diets, questionable research, news about how 'xyz' is good for you while 'abc' is bad, exercises to eliminate calories in 30 days, exercises to remove localized fat etc? All of this is shown to be evidently bullshit for decades for the minimally rational observer. Meanwhile the knowledge of what works is blatantly obvious: the energy expenditure must be higher than the intake. Having a healthy diet and exercise regimen along that is highly desirable but not strictly necessary, stating this just because understanding the core concept is more important than sheepishly believing the latest fad. Also that this is a process that takes a lot of time and is not a one-off procedure but a process of learning life-changing habits. I'd like to remark that this is not easy at all! In fact is really hard due to the inherent biological and societal tricks that play on our minds. But if people are already suffering psychologically and financially with this, why don't just try the basics? I understand how this is incentivized by an industry that extracts money from desperate people trying very hard to feel accepted by what society indicates as an acceptable and desirable appearance. The irony is that the failure of their latest hope is what makes them unable to understand that the problem is much simpler (not easy), and only try again on a more desperate attempt with the latest extreme measure. At least if people would be honest enough to blame themselves for their bad habits (lack of a healthy diet and exercise regimen) they could start the process of accepting what they are as a result of their choices, and finally notice that they have some agency on this. I am not talking here about existent mental and biological disorders of course and would not downplay their role. Sorry for the rant, the only way I can rationalize that this exists is that there are people that deny the existence of climate change.
- CharlesW 9y ago> Meanwhile the knowledge of what works is blatantly obvious: the energy expenditure must be higher than the intake. This is like saying about the poor, "What works is blatantly obvious: Poor people just need to increase income or reduce expenditures. Boom! Poverty solved." The secret is that this isn't the problem. The actual problem that fat people must solve is not what to do, but how to do it, both in the short- and long-term. > Having a healthy diet and exercise regimen along that is highly desirable but not strictly necessary... 1,000 calories of cupcake are 1,000 calories of chicken breast are, in theory, the same amount of energy. In practice, diet and exercise considerations are integral aspects of the actual problem that fat people must solve. Example: For some, exercise is actually counter-productive and causes more hunger. For others, exercise is a crucial part of success. Example: For some, calorie restriction can be done with no particular attention paid to the balance of carbs, proteins, and fats. For others, a low-carb diet means lower hunger/higher satiation for longer and is a must for success.
- kwhitefoot 9y ago> Many scientists receive only cursory training in statistics, This seems to be true (at least I have many younger colleagues who are woefully ill informed about even basic stats.) but it's very strange. Most of what I learned of statistics, including tests for significance and regression, I learned in senior high school at the age of 17. Learning not to search of correlations like this is pretty much equivalent to ensuring that you do not over fit your data set which is surely such conventional wisdom that no one getting a scientific education in the last sixty or seventy years should have missed it. I'm aware that there is a lot of highly sophisticated statistical analysis that I don't know about but things like p value are not among them.
- deleted 9y ago[deleted]