5 ms·

n=17, so read nothing into this result. Also, what they tested is having no fixed eating schedule. The effects (if any) of skipping breakfast every day consist

n=17, so read nothing into this result. Also, what they tested is having no fixed eating schedule. The effects (if any) of skipping breakfast every day consistently are not measured.

jasonhansel 4y ago
I agree that we shouldn't read much into this. But I do suspect that, if an n=17 study came to the opposite conclusions, supporters of intermittent fasting would never stop citing it.
- mrighele 4y ago
  I doubt so, since the study is not about intermittent fasting, but intermittent meal skipping at best :-). I think that skipping a meal every once in a while, and regularly skipping meals according to a specific pattern for one, two or ten years, are completely different activities with very different outcomes
paulpauper 4y ago
i don't get why some studies have so few partipants. How hard is it to recruit people for something like this? We're not talking rare diseases.
- deleted 4y ago
  [deleted]
- subharmonicon 4y ago
  From what I have read in the past this is just how the funding tends to work. You write a grant proposal for a study with 20 people, get funding, get results, use that to get funding for another study with 200, then 2000, etc. Frequently they don’t manage to make it past the first couple levels because the results aren’t interesting.
  - 3pt14159 4y ago
    Any study less than 200 people is essentially false advertising unless the population size is incredibly small (e.g., rare diseases) or the intervention has a crazy high prior (e.g., does getting shot in the chest lead to blood loss leading to passing out) so I consider these grants completely counter productive. There is a real benefit in having fewer pieces of junk to sort out and so many of these medical research studies are just noise that well meaning members of the public cite and adopt into practice because they don't know any better. It's the modern day version of leeches, essentially.
    - subharmonicon 4y ago
      Rather than “well meaning members of the public”, I think the larger problem is with health and science reporting which frequently reports findings of very small studies as if they are definitive fact, with very prominent and provocative headlines.
      - 3pt14159 4y ago
        > health and science reporting Is part of "well meaning members of the public" just as "YouTube commentary" is and "random celebrity that Googles something and repeats it on Twitter." And quite frankly, so are scientists themselves. I have met far, FAR too many scientists that do not know the very first thing about statistics and yet feel competent to reference other people's research. This is why I say it's a big pile of leeches for more than half this stuff. Sure the incentives are misaligned, sure its reporting, sure this and that. But what we really need here are people to say "no you idiot leeches don't do dittly squat to fix this problem."
- belltaco 4y ago
  All that screening and testing costs a lot of money.
- daviddavis 4y ago
  Nutritional studies are notoriously difficult to conduct. You either have to isolate subjects and monitor everything they eat and when. Or you have to rely on self reported data and people do an atrocious job of tracking food, estimating portion size, etc. This study did the former: > The duration of the study is 7 days including 3 days with controlled diet and 4 days (including 5 nights) in a metabolic chamber at the Institute of Nutritional Medicine at the University of Hohenheim. I imagine it was prohibitively expensive to have more participants.
- bee_rider 4y ago
  Try plinking around on a sample size calculator a bit, the results might be surprising. For example: https://www.surveymonkey.com/mp/sample-size-calculator/ https://www.surveymonkey.com/mp/sample-size-calculator/ If you are ok with a 25% margin of error and a 95% confidence level, 16 people is good enough. You have to make sure there isn’t some skew in your population, but if your method of sampling the population produces a skew, adding people won’t really help there.
weakfortress 4y ago
These comments are always so hilarious. It demonstrates a significant lack of understand of how statistics actually works. You see this on reddit all the time, you'd think it'd be better here. Power and sample size are determined by numerous factors depending on the question under study. You could have N=10 be statistically powerful and N=1,000,000 be statistically meaningless. It depends ENTIRELY on the subject under study. More is not always better and in many cases completely unnecessary. https://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Power/BS704_Power_print.html https://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Power/B... Here is a decent guide. You may wish to read it so you can learn when you can dismiss a study based on N. You never mentioned, did you back out the study numbers and determine the N=17 number was outside the range of statistical power? If so, would you mind posting your calculations?
- johnday 4y ago
  This is clinical nutrition. There are so many confounding factors that it is not reasonable to draw any conclusions about how the body works (especially as a layman reader on HN) from a study that tested 17 people. Indeed, their own introduction claims that previous larger studies have found no correlation between meal plans and effects on the body, at least wrt caloric expenditure. > These comments are always so hilarious. It demonstrates a significant lack of understand of how statistics actually works. I do have a PhD, but thank you for your input anyway.
  - weakfortress 4y ago
    [dead]
- hahajk 4y ago
  Would this be an intuitive way of putting it? If I pull a random 17 phones off an assembly line, and 16 of them are defective, I can pretty confidently say there is something wrong with the whole product line even though n=17?
theteapot 4y ago
If it's clearly not worth HN readers' time, how did it get published in "The American Journal of Clinical Nutrition"? Or am I just naive in thinking published in a Journal still means something? Why did it pass peer review?
- WalterSear 4y ago
  It's a scientific journal - a place for the researchers to discuss and discover science. This is a lay forum - a place more suited to the discussion of decided science.
  - theteapot 4y ago
    Yes exactly. If your average HN user can see sample size is too small, Journal reviewers should be able to. Your not going to discover or decide any science if your sample size is too small.
- deleted 4y ago
  [deleted]
- johnday 4y ago
  > If it's clearly not worth HN readers' time, how did it get published in "The American Journal of Clinical Nutrition"? This comment is extremely funny. I heard a rumour that the American Journal of Clinical Nutrition might be aimed at scientists working on clinical nutrition, who could make meaningful use of this result - and not the readers of Hacker News.
  - theteapot 4y ago
    That would be a strange rumour hear. What I'm saying is, if some rando on HN can see see that the trial size is too small to be statistically significant how did it pass peer review?
    - johnday 4y ago
      The trial size is not too small to be statistically significant. It may warrant further investigation by scientists in the field of clinical nutrition. However, it is too small and insignificant for Hacker News readers to make meaningful use of, and it would be incorrect for them to read anything into the results presented. That is the difference.
amatic 4y ago
The problem is not N=17. If the effect (difference between skipping breakfast vs dinner) was large enough, it would be obvious and significant even in a small sample. The effect seems to be very small. I agree that what is missing is to measure skipping meals for a longer time period, like a month, at least.
- thechao 4y ago
  I have an identical twin brother; I began IF (eating only 1/3 of target calories on Mon/Fri) in 2012. We do functional similar levels of cardio, weight-training; we're both programmers, with similar hours. Right now, I'm 1/2 taller, ~70 lbs lighter, I have ~10 lbs more LBM, blood pressure and blood work is spot on. He's got high blood pressure, and is diabetes adjacent. I have a good friend who's an identical twin who did CFR (over 30 years, now); similar situation. Here's the summary: getting fat is bad; losing fat is bad; being fat is deadly.
  - amalgamated_inc 4y ago
    Why is losing fat bad?
    - thechao 4y ago
      Losing fat requires calorie restriction; calorie restriction will lead to loss of both fat and lean-body-mass. When you stop your calorie restriction, you gain back fat much more easily than lean-body-mass (this is a result of the Milwaukee starvation study). Lean-body-mass includes things like muscle, bone, and other things you'd rather keep. Furthermore, it can cause significant stress on other tissues. The healthiest thing to do is just never be fat in the first place.
carabiner 4y ago
But the entire field of medicine is based on small studies with high power to achieve statistical significance, that lead to sequentially larger studies and eventually regulator approval or rejection. Are you saying the entire field is a sham? I urge you to write to the FDA immediately them to inform them that their methodology is gravely flawed.
- johnday 4y ago
  I am saying that lay readers on Hacker News should not draw conclusions from this study on 17 people. Do you disagree?
  - deleted 4y ago
    [deleted]
  - themitigating 4y ago
    You shouldn't draw conclusions from this small study but it increases the probability of one view being correct
- amalgamated_inc 4y ago
  Yes. The entire field is a sham.
jcoq 4y ago
If we give substance X to 17 people and 100% of them die, I would recommend against taking substance X. Of course, it's not as black and white here, but sample size and effect size should be considered in relation to each other. Previous research and theoretical expectations should also play a leading role.
- johnday 4y ago
  In this case, previous research and theoretical expectations would indicate only a small effect, if any. This is covered in the introduction to the paper, which I'm sure you have read.
deleted 4y ago
[deleted]

n=17, so read nothing into this result. Also, what they tested is having no fixed eating schedule. The effects (if any) of skipping breakfast every day consistently are not measured.

jasonhansel 4y ago

I agree that we shouldn't read much into this. But I do suspect that, if an n=17 study came to the opposite conclusions, supporters of intermittent fasting would never stop citing it.

mrighele 4y ago
I doubt so, since the study is not about intermittent fasting, but intermittent meal skipping at best :-). I think that skipping a meal every once in a while, and regularly skipping meals according to a specific pattern for one, two or ten years, are completely different activities with very different outcomes

paulpauper 4y ago

i don't get why some studies have so few partipants. How hard is it to recruit people for something like this? We're not talking rare diseases.

deleted 4y ago
[deleted]
subharmonicon 4y ago
From what I have read in the past this is just how the funding tends to work. You write a grant proposal for a study with 20 people, get funding, get results, use that to get funding for another study with 200, then 2000, etc. Frequently they don’t manage to make it past the first couple levels because the results aren’t interesting.
- 3pt14159 4y ago
  Any study less than 200 people is essentially false advertising unless the population size is incredibly small (e.g., rare diseases) or the intervention has a crazy high prior (e.g., does getting shot in the chest lead to blood loss leading to passing out) so I consider these grants completely counter productive. There is a real benefit in having fewer pieces of junk to sort out and so many of these medical research studies are just noise that well meaning members of the public cite and adopt into practice because they don't know any better. It's the modern day version of leeches, essentially.
  - subharmonicon 4y ago
    Rather than “well meaning members of the public”, I think the larger problem is with health and science reporting which frequently reports findings of very small studies as if they are definitive fact, with very prominent and provocative headlines.
    - 3pt14159 4y ago
      > health and science reporting Is part of "well meaning members of the public" just as "YouTube commentary" is and "random celebrity that Googles something and repeats it on Twitter." And quite frankly, so are scientists themselves. I have met far, FAR too many scientists that do not know the very first thing about statistics and yet feel competent to reference other people's research. This is why I say it's a big pile of leeches for more than half this stuff. Sure the incentives are misaligned, sure its reporting, sure this and that. But what we really need here are people to say "no you idiot leeches don't do dittly squat to fix this problem."
belltaco 4y ago
All that screening and testing costs a lot of money.
daviddavis 4y ago
Nutritional studies are notoriously difficult to conduct. You either have to isolate subjects and monitor everything they eat and when. Or you have to rely on self reported data and people do an atrocious job of tracking food, estimating portion size, etc. This study did the former: > The duration of the study is 7 days including 3 days with controlled diet and 4 days (including 5 nights) in a metabolic chamber at the Institute of Nutritional Medicine at the University of Hohenheim. I imagine it was prohibitively expensive to have more participants.
bee_rider 4y ago
Try plinking around on a sample size calculator a bit, the results might be surprising. For example: https://www.surveymonkey.com/mp/sample-size-calculator/ https://www.surveymonkey.com/mp/sample-size-calculator/ If you are ok with a 25% margin of error and a 95% confidence level, 16 people is good enough. You have to make sure there isn’t some skew in your population, but if your method of sampling the population produces a skew, adding people won’t really help there.

weakfortress 4y ago

These comments are always so hilarious. It demonstrates a significant lack of understand of how statistics actually works. You see this on reddit all the time, you'd think it'd be better here. Power and sample size are determined by numerous factors depending on the question under study. You could have N=10 be statistically powerful and N=1,000,000 be statistically meaningless. It depends ENTIRELY on the subject under study. More is not always better and in many cases completely unnecessary. https://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Power/BS704_Power_print.html https://sphweb.bumc.bu.edu/otlt/MPH-Modules/BS/BS704_Power/B... Here is a decent guide. You may wish to read it so you can learn when you can dismiss a study based on N. You never mentioned, did you back out the study numbers and determine the N=17 number was outside the range of statistical power? If so, would you mind posting your calculations?

johnday 4y ago
This is clinical nutrition. There are so many confounding factors that it is not reasonable to draw any conclusions about how the body works (especially as a layman reader on HN) from a study that tested 17 people. Indeed, their own introduction claims that previous larger studies have found no correlation between meal plans and effects on the body, at least wrt caloric expenditure. > These comments are always so hilarious. It demonstrates a significant lack of understand of how statistics actually works. I do have a PhD, but thank you for your input anyway.
- weakfortress 4y ago
  [dead]
hahajk 4y ago
Would this be an intuitive way of putting it? If I pull a random 17 phones off an assembly line, and 16 of them are defective, I can pretty confidently say there is something wrong with the whole product line even though n=17?

theteapot 4y ago

If it's clearly not worth HN readers' time, how did it get published in "The American Journal of Clinical Nutrition"? Or am I just naive in thinking published in a Journal still means something? Why did it pass peer review?

WalterSear 4y ago
It's a scientific journal - a place for the researchers to discuss and discover science. This is a lay forum - a place more suited to the discussion of decided science.
- theteapot 4y ago
  Yes exactly. If your average HN user can see sample size is too small, Journal reviewers should be able to. Your not going to discover or decide any science if your sample size is too small.
deleted 4y ago
[deleted]
johnday 4y ago
> If it's clearly not worth HN readers' time, how did it get published in "The American Journal of Clinical Nutrition"? This comment is extremely funny. I heard a rumour that the American Journal of Clinical Nutrition might be aimed at scientists working on clinical nutrition, who could make meaningful use of this result - and not the readers of Hacker News.
- theteapot 4y ago
  That would be a strange rumour hear. What I'm saying is, if some rando on HN can see see that the trial size is too small to be statistically significant how did it pass peer review?
  - johnday 4y ago
    The trial size is not too small to be statistically significant. It may warrant further investigation by scientists in the field of clinical nutrition. However, it is too small and insignificant for Hacker News readers to make meaningful use of, and it would be incorrect for them to read anything into the results presented. That is the difference.

amatic 4y ago

The problem is not N=17. If the effect (difference between skipping breakfast vs dinner) was large enough, it would be obvious and significant even in a small sample. The effect seems to be very small. I agree that what is missing is to measure skipping meals for a longer time period, like a month, at least.

thechao 4y ago
I have an identical twin brother; I began IF (eating only 1/3 of target calories on Mon/Fri) in 2012. We do functional similar levels of cardio, weight-training; we're both programmers, with similar hours. Right now, I'm 1/2 taller, ~70 lbs lighter, I have ~10 lbs more LBM, blood pressure and blood work is spot on. He's got high blood pressure, and is diabetes adjacent. I have a good friend who's an identical twin who did CFR (over 30 years, now); similar situation. Here's the summary: getting fat is bad; losing fat is bad; being fat is deadly.
- amalgamated_inc 4y ago
  Why is losing fat bad?
  - thechao 4y ago
    Losing fat requires calorie restriction; calorie restriction will lead to loss of both fat and lean-body-mass. When you stop your calorie restriction, you gain back fat much more easily than lean-body-mass (this is a result of the Milwaukee starvation study). Lean-body-mass includes things like muscle, bone, and other things you'd rather keep. Furthermore, it can cause significant stress on other tissues. The healthiest thing to do is just never be fat in the first place.

carabiner 4y ago

But the entire field of medicine is based on small studies with high power to achieve statistical significance, that lead to sequentially larger studies and eventually regulator approval or rejection. Are you saying the entire field is a sham? I urge you to write to the FDA immediately them to inform them that their methodology is gravely flawed.

johnday 4y ago
I am saying that lay readers on Hacker News should not draw conclusions from this study on 17 people. Do you disagree?
- deleted 4y ago
  [deleted]
- themitigating 4y ago
  You shouldn't draw conclusions from this small study but it increases the probability of one view being correct
amalgamated_inc 4y ago
Yes. The entire field is a sham.

jcoq 4y ago

If we give substance X to 17 people and 100% of them die, I would recommend against taking substance X. Of course, it's not as black and white here, but sample size and effect size should be considered in relation to each other. Previous research and theoretical expectations should also play a leading role.

johnday 4y ago
In this case, previous research and theoretical expectations would indicate only a small effect, if any. This is covered in the introduction to the paper, which I'm sure you have read.

deleted 4y ago

[deleted]