7 ms·

Skeptical science remark: This whole "if cows eat seaweed that'll vastly reduce their carbon footprint" is based on a single study with 12 cows. If you take i

by hannob 7y ago

Skeptical science remark: This whole "if cows eat seaweed that'll vastly reduce their carbon footprint" is based on a single study with 12 cows. If you take into consideration that especially small, never replicated studies often don't hold up to scrunity the next thing to do is not to ask how to produce the seaweed. The next reasonable step would be to try to replicate this study in a larger sample (and probably also with a larger variety of cow breeds).

sieabahlpark 7y ago
But the confirmation bias is so easy! It's also green! Why do you hate the environment? /s
solveit 7y ago
You're absolutely right, but the effect size is pretty damn impressive and I expect this to at least partially replicate.
- Etheryte 7y ago
  Having a small sample and a large effect doesn't mean you're guaranteed at least some effect on a large sample. If anything, a very large effect is more likely to indicate a flaw in your method.
  - solveit 7y ago
    Yes, but effect sizes are relatively harder to hack/screw up (knowingly or otherwise) than p-values. My totally anecdotal impression is that findings showing large effect sizes seem to replicate better than findings showing highly significant effects.
    - haffi112 7y ago
      It could be an example of publication bias, from the wiki article on effect size [1]: An example of this is publication bias, which occurs when scientists report results only when the estimated effect sizes are large or are statistically significant. As a result, if many researchers carry out studies with low statistical power, the reported effect sizes will tend to be larger than the true (population) effects, if any. [1] https://en.wikipedia.org/wiki/Effect_size https://en.wikipedia.org/wiki/Effect_size
    - mumblemumble 7y ago
      Giant effect sizes are easy to hack in biological sciences. All you have to do is conduct an in vitro study. Like they did in this paper.
    - jfindley 7y ago
      Excessively large effect sizes are a warning sign in and of themselves. A good article on this is here: [0] -- simply if the effect is too huge, it becomes incumbent on the researcher to explain why no-one has noticed this before. Methane is a gas with a distinct and noticeable smell. Seaweed is plentiful in many places cows are kept. If the effect was really as dramatic as claimed you'd expect that it'd be well known that letting cows eat seaweed made them less smelly, and the authors of the original study make no attempt to explain why this might be the case. 0: http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html http://daniellakens.blogspot.com/2017/07/impossibly-hungry-j...
      - icxa 7y ago
        > "it becomes incumbent on the researcher to explain why no-one has noticed this before" Farmer: huh, we'll I'll be damned. Who in the hell woulda thought to feed them seaweed."
        VBprogrammer 7y ago
        If I'm not mistaken the original inspiration for this research was precisely that farmers noticed their cattle (or perhaps it was sheep) were happier when allowed to graze in fields which happened to be along the shore, allowing them to naturally eat quantities of sea weed.
      - random_savv 7y ago
        > Methane is a gas with a distinct and noticeable smell Methane is odorless https://en.wikipedia.org/wiki/Methane https://en.wikipedia.org/wiki/Methane
        jfindley 7y ago
        Interesting, thank you! I was surprised by this, as it's the major component of natural gas (as used widely for cooking here), and this has a distinct smell. I went looking however, and discovered that's actually because Ethyl Mercaptan is added specifically to make it detectable by humans. I never realised this, and does indeed explain why it's plausible no-one would have noticed such a large effect.
  - LifeLiverTransp 7y ago
    My prediction would be the same for humans who have a large salt intake. Cardiac problems for cows.
  - sjwright 7y ago
    Maybe? But an effect which relies on a link between diet and farting isn’t exactly remarkable or even surprising. My dog farts the most noxious gas if he eats apple or chicken.
- tmalsburg2 7y ago
  In a small sample, the observed effect needs to be large in order to cross the threshold for statistical significance. It could still be a fluke and the true effect zero or even negative.
  - joombaga 7y ago
    What's the tendency when a study with a small sample size that show a large observed effect get performed on larger samples? Has it been studied?
    - tmalsburg2 7y ago
      It depends. If the original study was published only because it had a positive result (significance filter, usually true), we expect the effect to be smaller in a larger replication. It's called Type M error, with M for magnitude. And, yes, large scale replications typically show smaller effects or no effects. See this paper for details: https://journals.sagepub.com/doi/full/10.1177/1745691614551642 https://journals.sagepub.com/doi/full/10.1177/17456916145516...
deleted 7y ago
[deleted]
avip 7y ago
Even more skeptical remark: Effects of nutrition on livestock methane production have been studied for over 20 years now. We have methane reduction results from both in-vivo and in-vitro research. What varies is the numbers (99%, 50%, 15%?). Replication is always welcome but I believe we have enough sound science to move into more industry/engineering efforts.
dsfyu404ed 7y ago
Yup. There's plenty of boutique dairy farms within easy driving distance of rich colleges in the northeast (some of whom already have existing relationships with said colleges). I'm sure anyone who seriously wants to replicate the study would be able to find a farmer that will work with then to get an n value of a few hundred. It's not like you have to go somewhere remote to study cow farts.
- giarc 7y ago
  Most agricultural colleges/universities have their own farms/herds as well.
  - dsfyu404ed 7y ago
    I went to a college like that. Most colleges with big ag programs aren't spending big resources on cow emissions. Their flagship researchers that get the money are going to be studying things that are of more immediate importance to the farming industry and farming in general. At present those things are usually along the lines of how to minimize negative impacts and/or leverage the current and future changes in climate.
- peterwwillis 7y ago
  > cow farts burps, fwiw
ianai 7y ago
Anecdotally, taking in a little chlorophyll per day dramatically reduces how much gas I produce. So in my experience, it’s at least possible the effect may be large. Worthy of more research.
- saalweachter 7y ago
  Don't cows already eat quite a bit of chlorophyll?
  - zaphod4prez 7y ago
    Grass fed cows but the vast majority of cows are fed primarily or exclusively grain, not grass/hay.
    - ianai 7y ago
      I’m talking about chlorophyll extract specifically so it’d be in addition to the normal dietary chlorophyll as well.
    - rywils21 7y ago
      > "It contains chemicals that reduce the microbes in the cows’ stomachs that cause them to burp when they eat grass." Apparently, they are comparing this to eating grass. I'm a little skeptical of that statement, though. Grass-fed vs. grain-fed have wildly different outcomes for the health of the animal.
strainer 7y ago
"is based on a single study with 12 cows." Its not - most often "skeptical science remarks" although popular, are poorly informed and misleading. A quick scholar search shows numerous studies broadly agreeing with the one featured here, including the seaweeds effects on sheep digestion and microbial and metabolic details: https://scholar.google.com/scholar?q=asparagopsis+methane https://scholar.google.com/scholar?q=asparagopsis+methane
- deleted 7y ago
  [deleted]
- MiroF 7y ago
  > most often "skeptical science remarks" although popular, are poorly informed and misleading. I thought I was the only one who noticed this! Seems very popular and common in forums like HN and Reddit, but less so elsewhere.
  - TravHatesMe 7y ago
    I would prefer being skeptical instead of blindly accepting things. This behavior will instigate another to provide evidence or a counter-argument. This makes for great discussion, in most cases I think it is healthy.
    - mikelyons 7y ago
      It's easy for what passes for a skeptical attitude to actually _be_ a blind acceptance. Whether of another implicit and unexamined paradigm that one is operating from, or just a dissonance-triggered knee-jerk. It's a very sneaky self-deception, false skepticism.
    - MiroF 7y ago
      Aren't you just blindly accepting the skeptical response? What did you learn from this great conversation: OP: This study has never been reproduced!!!! Response: Uh, yes it has.
dedestruction 7y ago
Further food for thought: the lower the statistical power of a study which meets a given significance threshold, the more exaggerated the effect size will be. So our default posture to evidence like this, even assuming we find it broadly credible, should be that the real-world effect is very likely to be smaller than this, and very unlikely to be larger.
SamuelAdams 7y ago
I agree, we need to slowly study other groups of cows to see if this is a viable solution. I'm reading Factfulness by Hans Rosling at the moment. He had a great point about "single numbers" - the jist was to always be skeptical of just a single number or metric. Here's an example [1]. During WWII, soliders were dying in their own vomit because they were laying prone on the battlefield, in sick beds, etc and left unattended. Doctors tried using the Recovery Position [2] and saw a decrease in those types of deaths by about 99% (this is in the book). Sounds like it should work for other types of people, right? First aid organizations quickly updated their "best recommendations" for all types of people: sick, drunk, young, old. This position was the best possible position for increased airway access, thereby decreasing preventable patient deaths. It wasn't until the 1980's that this idea was challenged. A few health organizations noticed an increase in infant deaths. Hong Kong's group was the first to investigate this [3]. Other groups joined in and realized that infants, when turned on their stomach, do not have the strength to tip their heads after they vomit. As a result, infant deaths increase when they are placed into the Recovery Position. The recommendation was quickly updated in the late 1980's / early 1990's. Now the recovery position is recommended for people ages 5 and up. Between 1940 - 1985, there was a larger number of infant deaths because one group of well-meaning people took some knowledge they gained for one group and quickly applied it to all other groups. I think this study (cows reducing methane based on diet) is a great idea. But we need to be cautious about applying this quickly to all cows, else we could kill half the cow population and cause the cost of beef to skyrocket. [1]: https://www.gapminder.org/factfulness-book/notes/ https://www.gapminder.org/factfulness-book/notes/, Page 163. [2]: https://en.wikipedia.org/wiki/Recovery_position https://en.wikipedia.org/wiki/Recovery_position [3]: https://www.ncbi.nlm.nih.gov/pubmed/2866397?dopt=Abstract https://www.ncbi.nlm.nih.gov/pubmed/2866397?dopt=Abstract
- halbritt 7y ago
  Good book, that.
- bjelkeman-again 7y ago
  My impression is that farmers generally areconservative about how they change their feed for their livestock. I don’t think there is any risk that they will “kill half the cow population” in a hurry.
mklarmann 7y ago
The study misses out to report on the increased production of N2O. Which really does not help.

by hannob 7y ago

sieabahlpark 7y ago

But the confirmation bias is so easy! It's also green! Why do you hate the environment? /s

solveit 7y ago

You're absolutely right, but the effect size is pretty damn impressive and I expect this to at least partially replicate.

Etheryte 7y ago
Having a small sample and a large effect doesn't mean you're guaranteed at least some effect on a large sample. If anything, a very large effect is more likely to indicate a flaw in your method.
- solveit 7y ago
  Yes, but effect sizes are relatively harder to hack/screw up (knowingly or otherwise) than p-values. My totally anecdotal impression is that findings showing large effect sizes seem to replicate better than findings showing highly significant effects.
  - haffi112 7y ago
    It could be an example of publication bias, from the wiki article on effect size [1]: An example of this is publication bias, which occurs when scientists report results only when the estimated effect sizes are large or are statistically significant. As a result, if many researchers carry out studies with low statistical power, the reported effect sizes will tend to be larger than the true (population) effects, if any. [1] https://en.wikipedia.org/wiki/Effect_size https://en.wikipedia.org/wiki/Effect_size
  - mumblemumble 7y ago
    Giant effect sizes are easy to hack in biological sciences. All you have to do is conduct an in vitro study. Like they did in this paper.
  - jfindley 7y ago
    Excessively large effect sizes are a warning sign in and of themselves. A good article on this is here: [0] -- simply if the effect is too huge, it becomes incumbent on the researcher to explain why no-one has noticed this before. Methane is a gas with a distinct and noticeable smell. Seaweed is plentiful in many places cows are kept. If the effect was really as dramatic as claimed you'd expect that it'd be well known that letting cows eat seaweed made them less smelly, and the authors of the original study make no attempt to explain why this might be the case. 0: http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html http://daniellakens.blogspot.com/2017/07/impossibly-hungry-j...
    - icxa 7y ago
      > "it becomes incumbent on the researcher to explain why no-one has noticed this before" Farmer: huh, we'll I'll be damned. Who in the hell woulda thought to feed them seaweed."
      - VBprogrammer 7y ago
        If I'm not mistaken the original inspiration for this research was precisely that farmers noticed their cattle (or perhaps it was sheep) were happier when allowed to graze in fields which happened to be along the shore, allowing them to naturally eat quantities of sea weed.
    - random_savv 7y ago
      > Methane is a gas with a distinct and noticeable smell Methane is odorless https://en.wikipedia.org/wiki/Methane https://en.wikipedia.org/wiki/Methane
      - jfindley 7y ago
        Interesting, thank you! I was surprised by this, as it's the major component of natural gas (as used widely for cooking here), and this has a distinct smell. I went looking however, and discovered that's actually because Ethyl Mercaptan is added specifically to make it detectable by humans. I never realised this, and does indeed explain why it's plausible no-one would have noticed such a large effect.
- LifeLiverTransp 7y ago
  My prediction would be the same for humans who have a large salt intake. Cardiac problems for cows.
- sjwright 7y ago
  Maybe? But an effect which relies on a link between diet and farting isn’t exactly remarkable or even surprising. My dog farts the most noxious gas if he eats apple or chicken.
tmalsburg2 7y ago
In a small sample, the observed effect needs to be large in order to cross the threshold for statistical significance. It could still be a fluke and the true effect zero or even negative.
- joombaga 7y ago
  What's the tendency when a study with a small sample size that show a large observed effect get performed on larger samples? Has it been studied?
  - tmalsburg2 7y ago
    It depends. If the original study was published only because it had a positive result (significance filter, usually true), we expect the effect to be smaller in a larger replication. It's called Type M error, with M for magnitude. And, yes, large scale replications typically show smaller effects or no effects. See this paper for details: https://journals.sagepub.com/doi/full/10.1177/1745691614551642 https://journals.sagepub.com/doi/full/10.1177/17456916145516...

deleted 7y ago

[deleted]

avip 7y ago

Even more skeptical remark: Effects of nutrition on livestock methane production have been studied for over 20 years now. We have methane reduction results from both in-vivo and in-vitro research. What varies is the numbers (99%, 50%, 15%?). Replication is always welcome but I believe we have enough sound science to move into more industry/engineering efforts.

dsfyu404ed 7y ago

Yup. There's plenty of boutique dairy farms within easy driving distance of rich colleges in the northeast (some of whom already have existing relationships with said colleges). I'm sure anyone who seriously wants to replicate the study would be able to find a farmer that will work with then to get an n value of a few hundred. It's not like you have to go somewhere remote to study cow farts.

giarc 7y ago
Most agricultural colleges/universities have their own farms/herds as well.
- dsfyu404ed 7y ago
  I went to a college like that. Most colleges with big ag programs aren't spending big resources on cow emissions. Their flagship researchers that get the money are going to be studying things that are of more immediate importance to the farming industry and farming in general. At present those things are usually along the lines of how to minimize negative impacts and/or leverage the current and future changes in climate.
peterwwillis 7y ago
> cow farts burps, fwiw

ianai 7y ago

Anecdotally, taking in a little chlorophyll per day dramatically reduces how much gas I produce. So in my experience, it’s at least possible the effect may be large. Worthy of more research.

saalweachter 7y ago
Don't cows already eat quite a bit of chlorophyll?
- zaphod4prez 7y ago
  Grass fed cows but the vast majority of cows are fed primarily or exclusively grain, not grass/hay.
  - ianai 7y ago
    I’m talking about chlorophyll extract specifically so it’d be in addition to the normal dietary chlorophyll as well.
  - rywils21 7y ago
    > "It contains chemicals that reduce the microbes in the cows’ stomachs that cause them to burp when they eat grass." Apparently, they are comparing this to eating grass. I'm a little skeptical of that statement, though. Grass-fed vs. grain-fed have wildly different outcomes for the health of the animal.

strainer 7y ago

"is based on a single study with 12 cows." Its not - most often "skeptical science remarks" although popular, are poorly informed and misleading. A quick scholar search shows numerous studies broadly agreeing with the one featured here, including the seaweeds effects on sheep digestion and microbial and metabolic details: https://scholar.google.com/scholar?q=asparagopsis+methane https://scholar.google.com/scholar?q=asparagopsis+methane

deleted 7y ago
[deleted]
MiroF 7y ago
> most often "skeptical science remarks" although popular, are poorly informed and misleading. I thought I was the only one who noticed this! Seems very popular and common in forums like HN and Reddit, but less so elsewhere.
- TravHatesMe 7y ago
  I would prefer being skeptical instead of blindly accepting things. This behavior will instigate another to provide evidence or a counter-argument. This makes for great discussion, in most cases I think it is healthy.
  - mikelyons 7y ago
    It's easy for what passes for a skeptical attitude to actually _be_ a blind acceptance. Whether of another implicit and unexamined paradigm that one is operating from, or just a dissonance-triggered knee-jerk. It's a very sneaky self-deception, false skepticism.
  - MiroF 7y ago
    Aren't you just blindly accepting the skeptical response? What did you learn from this great conversation: OP: This study has never been reproduced!!!! Response: Uh, yes it has.

dedestruction 7y ago

Further food for thought: the lower the statistical power of a study which meets a given significance threshold, the more exaggerated the effect size will be. So our default posture to evidence like this, even assuming we find it broadly credible, should be that the real-world effect is very likely to be smaller than this, and very unlikely to be larger.

SamuelAdams 7y ago

I agree, we need to slowly study other groups of cows to see if this is a viable solution. I'm reading Factfulness by Hans Rosling at the moment. He had a great point about "single numbers" - the jist was to always be skeptical of just a single number or metric. Here's an example [1]. During WWII, soliders were dying in their own vomit because they were laying prone on the battlefield, in sick beds, etc and left unattended. Doctors tried using the Recovery Position [2] and saw a decrease in those types of deaths by about 99% (this is in the book). Sounds like it should work for other types of people, right? First aid organizations quickly updated their "best recommendations" for all types of people: sick, drunk, young, old. This position was the best possible position for increased airway access, thereby decreasing preventable patient deaths. It wasn't until the 1980's that this idea was challenged. A few health organizations noticed an increase in infant deaths. Hong Kong's group was the first to investigate this [3]. Other groups joined in and realized that infants, when turned on their stomach, do not have the strength to tip their heads after they vomit. As a result, infant deaths increase when they are placed into the Recovery Position. The recommendation was quickly updated in the late 1980's / early 1990's. Now the recovery position is recommended for people ages 5 and up. Between 1940 - 1985, there was a larger number of infant deaths because one group of well-meaning people took some knowledge they gained for one group and quickly applied it to all other groups. I think this study (cows reducing methane based on diet) is a great idea. But we need to be cautious about applying this quickly to all cows, else we could kill half the cow population and cause the cost of beef to skyrocket. [1]: https://www.gapminder.org/factfulness-book/notes/ https://www.gapminder.org/factfulness-book/notes/, Page 163. [2]: https://en.wikipedia.org/wiki/Recovery_position https://en.wikipedia.org/wiki/Recovery_position [3]: https://www.ncbi.nlm.nih.gov/pubmed/2866397?dopt=Abstract https://www.ncbi.nlm.nih.gov/pubmed/2866397?dopt=Abstract

halbritt 7y ago
Good book, that.
bjelkeman-again 7y ago
My impression is that farmers generally areconservative about how they change their feed for their livestock. I don’t think there is any risk that they will “kill half the cow population” in a hurry.

mklarmann 7y ago

The study misses out to report on the increased production of N2O. Which really does not help.