5 ms·

Because self-experimentation is an excellent source of data. Obviously we'd prefer a series of double-blind, controlled experiments whose results have been rigo

by nhoven 12y ago

Because self-experimentation is an excellent source of data. Obviously we'd prefer a series of double-blind, controlled experiments whose results have been rigorously replicated, but that doesn't mean that less rigorous experiments have no value.

DanBC 12y ago
> Because self-experimentation is an excellent source of data. No it isn't. People quote Barry Marshall[1] all the time but most people don't perform their self-experimentation with any kind of rigour. How many people have access to endoscopy to take before / after tests? https://en.wikipedia.org/wiki/Barry_Marshall https://en.wikipedia.org/wiki/Barry_Marshall
- beagle3 12y ago
  > No it isn't. Self experimentation could be an excellent source of data. Quite a few "interesting" things cannot effectively be rigorously tested, either because we have no effective placebo, or because they are inhumane (e.g., the vast majority of so-called "rigorous" diet studies are actually based on self-reporting and willful compliance, because you can't keep people in controlled a lab for more than a few weeks). Also, as Ionnadis pointed out, quite a bit of (supposedly rigourous) published research is simply wrong. So for a lot of things, we have to do with "amateur hour" experiments, whether conducted in a university lab and published in Nature, or conducted at home and published in a blog. Self-experimentation can easily disprove a "general truth" but cannot really support one due to lack of controls ( and do recall the scientific method is statement about disproving things, not about proving them) However, if you do stumble on a local version of a "general truth", you can then look for the n->inf version more rigorously. You can look at e.g. the late Seth Roberts for examples; the ideas are interesting, some of his successes have been replicated by enough people; Among these - correlation between Vitamin D intake time and sleep quality (see gwern for a rigorous test of that experiment, that confirms it for n=|{gwern}|=1, and many other less rigorous; Rigorous correlation among flax seed (supposedly omega 3) intake, butter, bacon and response time for n=|{seth}|=1, which I have not published but have verified rigorously for n=|me|=1 about flax seed. And there are a few others. I think some of his ideas are crazy but almost all are interesting. (Self-reported) success on his Shangri-La diet is ~80%, which is very high for a diet, but obviously not a general truth since it is not 100%. It worked for me once, but not the 2nd time a couple of years later despite following exactly the same protocol. I'd say self-experimentation is much less rigorous, but way more time-and-captial efficient way of arriving at results.
  - mrfusion 12y ago
    I think most of our progress in life is self-experimentation in one sort or another. We try things and do what works and stop doing what doesn't. By the way, did anyone find out what happened to Seth?
    - beagle3 12y ago
      According to http://blog.sethroberts.net/2014/05/10/cause-of-death/ http://blog.sethroberts.net/2014/05/10/cause-of-death/ , his family should be getting the report around now, and they mentioned the intend to share the details.

by nhoven 12y ago

DanBC 12y ago

> Because self-experimentation is an excellent source of data. No it isn't. People quote Barry Marshall[1] all the time but most people don't perform their self-experimentation with any kind of rigour. How many people have access to endoscopy to take before / after tests? https://en.wikipedia.org/wiki/Barry_Marshall https://en.wikipedia.org/wiki/Barry_Marshall

beagle3 12y ago
> No it isn't. Self experimentation could be an excellent source of data. Quite a few "interesting" things cannot effectively be rigorously tested, either because we have no effective placebo, or because they are inhumane (e.g., the vast majority of so-called "rigorous" diet studies are actually based on self-reporting and willful compliance, because you can't keep people in controlled a lab for more than a few weeks). Also, as Ionnadis pointed out, quite a bit of (supposedly rigourous) published research is simply wrong. So for a lot of things, we have to do with "amateur hour" experiments, whether conducted in a university lab and published in Nature, or conducted at home and published in a blog. Self-experimentation can easily disprove a "general truth" but cannot really support one due to lack of controls ( and do recall the scientific method is statement about disproving things, not about proving them) However, if you do stumble on a local version of a "general truth", you can then look for the n->inf version more rigorously. You can look at e.g. the late Seth Roberts for examples; the ideas are interesting, some of his successes have been replicated by enough people; Among these - correlation between Vitamin D intake time and sleep quality (see gwern for a rigorous test of that experiment, that confirms it for n=|{gwern}|=1, and many other less rigorous; Rigorous correlation among flax seed (supposedly omega 3) intake, butter, bacon and response time for n=|{seth}|=1, which I have not published but have verified rigorously for n=|me|=1 about flax seed. And there are a few others. I think some of his ideas are crazy but almost all are interesting. (Self-reported) success on his Shangri-La diet is ~80%, which is very high for a diet, but obviously not a general truth since it is not 100%. It worked for me once, but not the 2nd time a couple of years later despite following exactly the same protocol. I'd say self-experimentation is much less rigorous, but way more time-and-captial efficient way of arriving at results.
- mrfusion 12y ago
  I think most of our progress in life is self-experimentation in one sort or another. We try things and do what works and stop doing what doesn't. By the way, did anyone find out what happened to Seth?
  - beagle3 12y ago
    According to http://blog.sethroberts.net/2014/05/10/cause-of-death/ http://blog.sethroberts.net/2014/05/10/cause-of-death/ , his family should be getting the report around now, and they mentioned the intend to share the details.