4 ms·

He gets to the "hypothesis forming" stage, and draws a conclusion. That's fine if you're trying to figure out how to live your life best. The negative consequen

He gets to the "hypothesis forming" stage, and draws a conclusion. That's fine if you're trying to figure out how to live your life best. The negative consequences of removing his mercury fillings are probably null, he can accept his hypothesis as true and move on. But this is bad science, without question. We don't know that his mental function has improved because of removing fillings... it's a potential hypothesis, sure, but to say "Mercury Damage Revealed by Brain Test" in a headline seems a bit bold for a sample size of n with no control and a bunch of other confounding variables at play.

npsimons 14y ago
Yes, he does seem to pull out the "jump to conclusions" mat a bit much. That being said, I can't help but think that this would only justify more data. Log more variables; have someone review your methodology; etc. I've always wondered, how do you test your own intelligence? If you're writing and scoring the test, doesn't that invalidate scores you get? Ditto for taking the same test more than once. Maybe I'm just not smart or creative enough to think of ways around this (is that a test? :)
- dminor 14y ago
  I'm guessing he has a program generating the test using random numbers.
- sinak 14y ago
  Check out http://quantified-mind.com http://quantified-mind.com for accurate, repeatable cognitive testing. The UI isn't great, but I'm working on that at the moment.
  - nyan_sandwich 14y ago
    It looks like I have to sign up to play. Trivial inconvenience, I know, but now I'm bored and doing other things. Just FYI.
    - sinak 14y ago
      Good to know - will work on that
- beagle3 14y ago
  > I've always wondered, how do you test your own intelligence? He is not measuring intelligence; He's measuring reaction times (how long he takes to solve a very simple arithmetic problem; or how long he takes to see which letter in a 4 letter string appears twice) and fine balance/motor function (how long he can keep his balance on an unstable platform he has built) He went over these things in his blog in great detail over the years. He keeps trying new tests, keeps using those that are statistically stable in a "stable" environment (without changing nutrition / sleep / etc, and after a learning period) - and after he has a statistically significant baseline, he measures how e.g. eating more butter or flax oil changes the results of these tests. Yes, this is drawing conclusions for n=1, but for himself, these conclusions are very scientific - and us humans are not unique enough for them to have wider applicability. No, you cannot deduce from n=1 to the entire human race. Yes, his conclusions are plausible and most of them are verified by others. Read e.g. gwern.net on vitamin D.
  - drostie 14y ago
    I'm sorry, but science is neither "we measure the things" nor is it "we look for the cause." Yes, those are things which scientists do, but no, that is not the essence of science. There is no attempt being made to come to any wholesome understanding. That is, a scientist is trying to model nature, and this consists in two parts: (a) develop a model, (b) test it against nature. Seth Roberts has perhaps hit upon a way to develop models, but he doesn't seem to test them. That is, cum hoc, ergo propter hoc -- "with this, therefore because of this" -- is not actually a test for causation, but merely a guess for causation. To do a proper test, you need variable isolation, and the stability of test results is no guarantee that it is an isolated variable. A good example of an isolated variable is a simple light switch. A terrible example of an isolated variable would be your time of waking up, because that time cannot be isolated from your own thoughts and beliefs -- that is, many people, especially if they're not sleep-deprived, can wake up earlier simply by telling themselves "I'm going to wake up early tomorrow." (I myself usually wake up before my alarm clock goes off.) Humans in general are not light switches. As has been documented repeatedly in medicine, dual effects of placebo and hypochondria plague us; things we expect to be medicinal often placate us even when they have no medicinal content and you can suddenly feel the symptoms of a malady soon after reading about it on Wikipedia. So even if you want to say "for himself, these conclusions are very scientific," you're going to have to account for the problem that he is probably going to confirm whatever he already expects. That is, any "follow up" tests after the first guess are already tainted by the fact that Seth knows what's being tested.

npsimons 14y ago

Yes, he does seem to pull out the "jump to conclusions" mat a bit much. That being said, I can't help but think that this would only justify more data. Log more variables; have someone review your methodology; etc. I've always wondered, how do you test your own intelligence? If you're writing and scoring the test, doesn't that invalidate scores you get? Ditto for taking the same test more than once. Maybe I'm just not smart or creative enough to think of ways around this (is that a test? :)

dminor 14y ago
I'm guessing he has a program generating the test using random numbers.
sinak 14y ago
Check out http://quantified-mind.com http://quantified-mind.com for accurate, repeatable cognitive testing. The UI isn't great, but I'm working on that at the moment.
- nyan_sandwich 14y ago
  It looks like I have to sign up to play. Trivial inconvenience, I know, but now I'm bored and doing other things. Just FYI.
  - sinak 14y ago
    Good to know - will work on that
beagle3 14y ago
> I've always wondered, how do you test your own intelligence? He is not measuring intelligence; He's measuring reaction times (how long he takes to solve a very simple arithmetic problem; or how long he takes to see which letter in a 4 letter string appears twice) and fine balance/motor function (how long he can keep his balance on an unstable platform he has built) He went over these things in his blog in great detail over the years. He keeps trying new tests, keeps using those that are statistically stable in a "stable" environment (without changing nutrition / sleep / etc, and after a learning period) - and after he has a statistically significant baseline, he measures how e.g. eating more butter or flax oil changes the results of these tests. Yes, this is drawing conclusions for n=1, but for himself, these conclusions are very scientific - and us humans are not unique enough for them to have wider applicability. No, you cannot deduce from n=1 to the entire human race. Yes, his conclusions are plausible and most of them are verified by others. Read e.g. gwern.net on vitamin D.
- drostie 14y ago
  I'm sorry, but science is neither "we measure the things" nor is it "we look for the cause." Yes, those are things which scientists do, but no, that is not the essence of science. There is no attempt being made to come to any wholesome understanding. That is, a scientist is trying to model nature, and this consists in two parts: (a) develop a model, (b) test it against nature. Seth Roberts has perhaps hit upon a way to develop models, but he doesn't seem to test them. That is, cum hoc, ergo propter hoc -- "with this, therefore because of this" -- is not actually a test for causation, but merely a guess for causation. To do a proper test, you need variable isolation, and the stability of test results is no guarantee that it is an isolated variable. A good example of an isolated variable is a simple light switch. A terrible example of an isolated variable would be your time of waking up, because that time cannot be isolated from your own thoughts and beliefs -- that is, many people, especially if they're not sleep-deprived, can wake up earlier simply by telling themselves "I'm going to wake up early tomorrow." (I myself usually wake up before my alarm clock goes off.) Humans in general are not light switches. As has been documented repeatedly in medicine, dual effects of placebo and hypochondria plague us; things we expect to be medicinal often placate us even when they have no medicinal content and you can suddenly feel the symptoms of a malady soon after reading about it on Wikipedia. So even if you want to say "for himself, these conclusions are very scientific," you're going to have to account for the problem that he is probably going to confirm whatever he already expects. That is, any "follow up" tests after the first guess are already tainted by the fact that Seth knows what's being tested.