4 ms·

25 people is not a "clinical study". It is a survey.

by pi18n 15y ago

nodata 15y ago
and 25 people is a small enough group that you can do just a few surveys, then pick the group that gives the results you wanted, all without spending very much money.
- sambeau 15y ago
  22/25 will not be caused by random error. That is either a significant find or a systematic error in method. It is also inconceivable that a University group like this would do that—If they did they would all be in serious trouble. Ten out of 13 patients (76.9%) achieved full remission Being that they used a sample of depression sufferers that is a pretty amazing result. It will be interesting to see what the double-blind survey finds.
  - DanBC 15y ago
    > It is also inconceivable that a University group like this would do that—If they did they would all be in serious trouble. What? It is trivially easy to find awful research from universities. I've been involved (as a lay member of a research review panel) with university students who want to do clinical research. They come to the RRP with a proposal, and we walk through it and point out where they might have problems. It is amazing how many groups ignore things like small sample sizes or double blinding or control groups etc. Those are all basics that any student should have been aware of. And then there are the more subtle things like anonymising data or getting proper consent. Having said all that, this news release link (http://www.prnewswire.com/news-releases/bright-light-into-the-ear-canal-treats-seasonal-depression-effectively-in-two-clinical-trials-presented-by-valkee-and-university-of-oulu-133420808.html http://www.prnewswire.com/news-releases/bright-light-into-th...) gives more information about methodology. Note that they're researching SAD, which is not depression, and is rarer. Most people with depression don't show great results from light boxes.
    - sambeau 15y ago
      I agree that there are many pieces of bad research—this may well be one of them. Without seeing their full methodology its difficult to judge. There are many ways a piece of research can be flawed. I disagree that a team of researchers from a University of this standing that is composed like this would run study multiple times and cherry pick the dataset with the result they desire and if they did they certainly would be risking serious professional (if not criminal) damage. The researchers are: Markku Timonen1 M.D. Ph.D, Juuso Nissilä1, MSc., Heidi Jurvelin1, MSc., Anu Liettu M.D., Jari Jokelainen1, M.Sc., Antti Aunio MSc. , Pirkko Räsänen3, M.D. Ph.D., Timo Takala M.D. Ph.D. That's a lot of doctors and doctorates, its not a bunch of inexperienced students.
      - DanBC 15y ago
        You're right. These are all people who should know how to conduct a sensible study. I haven't seen it, so I don't know if it's good or poor or complete rubbish. It's also trivially easy to find doctors who are idiots and wingnuts who believe obvious nonsense.
sambeau 15y ago
You'd be amazed by how many medical studies uses such small sample sizes. Often its for a good reason for instance when looking at the effectiveness of a surgical procedure in a sub-group or the treatment for a reasonably rare condition. Sometimes it is because of the need to use a random or carefully selected bias group. But, as a one-time criminologist's statistician used to samples in 1000s (or 100s at least), I have often concluded that it was just bad science.
FaceKicker 15y ago
The sample size has nothing to do with the distinction between a survey and a study, as far as I know. Either could have a small sample size or a large sample size. If they got a statistically significant result given some alpha value, then that's a valid result, assuming the experiment itself was set up in a valid way (random sampling, controlled, double blind, etc.). Statistical significance tests take the sample size into account. A large enough difference can make up for a small sample size, and vice versa.
regandersong 15y ago
Their most recent study presented at a conference in Budapest this week had 89 patients. There's a good amount of information on the Valkee website http://www.valkee.com/uk/science.html#navigation http://www.valkee.com/uk/science.html#navigation, as well as a more recent article here http://www.arcticstartup.com/2011/11/08/valkees-claimed-benefits-for-s-a-d-strengthened-after-clinical-trial-results http://www.arcticstartup.com/2011/11/08/valkees-claimed-bene...
exDM69 15y ago
I have said this before in HN, but I'll repeat myself: don't trust this study. The Valkee company is financially backed by ex-Nokia execs and other people with money and influence in Finland. They have the means to get a small University in Northern Finland to make a study in their favor. There have been interesting studies about light sensivity in the skin, so why not the brain. However, this study doesn't really prove anything. Also, don't underestimate the value of placebo in the treatment of mental ailments.
- JoeAltmaier 15y ago
  Right! So even as a $185 placebo, its good medicine?
  - exDM69 15y ago
    The $185 price probably contributes to the placebo effect. It was expensive so it _must_ work.

25 people is not a "clinical study". It is a survey.

by pi18n 15y ago

25 people is not a "clinical study". It is a survey.

and 25 people is a small enough group that you can do just a few surveys, then pick the group that gives the results you wanted, all without spending very much money.

sambeau 15y ago
22/25 will not be caused by random error. That is either a significant find or a systematic error in method. It is also inconceivable that a University group like this would do that—If they did they would all be in serious trouble. Ten out of 13 patients (76.9%) achieved full remission Being that they used a sample of depression sufferers that is a pretty amazing result. It will be interesting to see what the double-blind survey finds.
- DanBC 15y ago
  > It is also inconceivable that a University group like this would do that—If they did they would all be in serious trouble. What? It is trivially easy to find awful research from universities. I've been involved (as a lay member of a research review panel) with university students who want to do clinical research. They come to the RRP with a proposal, and we walk through it and point out where they might have problems. It is amazing how many groups ignore things like small sample sizes or double blinding or control groups etc. Those are all basics that any student should have been aware of. And then there are the more subtle things like anonymising data or getting proper consent. Having said all that, this news release link (http://www.prnewswire.com/news-releases/bright-light-into-the-ear-canal-treats-seasonal-depression-effectively-in-two-clinical-trials-presented-by-valkee-and-university-of-oulu-133420808.html http://www.prnewswire.com/news-releases/bright-light-into-th...) gives more information about methodology. Note that they're researching SAD, which is not depression, and is rarer. Most people with depression don't show great results from light boxes.
  - sambeau 15y ago
    I agree that there are many pieces of bad research—this may well be one of them. Without seeing their full methodology its difficult to judge. There are many ways a piece of research can be flawed. I disagree that a team of researchers from a University of this standing that is composed like this would run study multiple times and cherry pick the dataset with the result they desire and if they did they certainly would be risking serious professional (if not criminal) damage. The researchers are: Markku Timonen1 M.D. Ph.D, Juuso Nissilä1, MSc., Heidi Jurvelin1, MSc., Anu Liettu M.D., Jari Jokelainen1, M.Sc., Antti Aunio MSc. , Pirkko Räsänen3, M.D. Ph.D., Timo Takala M.D. Ph.D. That's a lot of doctors and doctorates, its not a bunch of inexperienced students.
    - DanBC 15y ago
      You're right. These are all people who should know how to conduct a sensible study. I haven't seen it, so I don't know if it's good or poor or complete rubbish. It's also trivially easy to find doctors who are idiots and wingnuts who believe obvious nonsense.

sambeau 15y ago

You'd be amazed by how many medical studies uses such small sample sizes. Often its for a good reason for instance when looking at the effectiveness of a surgical procedure in a sub-group or the treatment for a reasonably rare condition. Sometimes it is because of the need to use a random or carefully selected bias group. But, as a one-time criminologist's statistician used to samples in 1000s (or 100s at least), I have often concluded that it was just bad science.

FaceKicker 15y ago

The sample size has nothing to do with the distinction between a survey and a study, as far as I know. Either could have a small sample size or a large sample size. If they got a statistically significant result given some alpha value, then that's a valid result, assuming the experiment itself was set up in a valid way (random sampling, controlled, double blind, etc.). Statistical significance tests take the sample size into account. A large enough difference can make up for a small sample size, and vice versa.

regandersong 15y ago

Their most recent study presented at a conference in Budapest this week had 89 patients. There's a good amount of information on the Valkee website http://www.valkee.com/uk/science.html#navigation http://www.valkee.com/uk/science.html#navigation, as well as a more recent article here http://www.arcticstartup.com/2011/11/08/valkees-claimed-benefits-for-s-a-d-strengthened-after-clinical-trial-results http://www.arcticstartup.com/2011/11/08/valkees-claimed-bene...

exDM69 15y ago

I have said this before in HN, but I'll repeat myself: don't trust this study. The Valkee company is financially backed by ex-Nokia execs and other people with money and influence in Finland. They have the means to get a small University in Northern Finland to make a study in their favor. There have been interesting studies about light sensivity in the skin, so why not the brain. However, this study doesn't really prove anything. Also, don't underestimate the value of placebo in the treatment of mental ailments.

JoeAltmaier 15y ago
Right! So even as a $185 placebo, its good medicine?
- exDM69 15y ago
  The $185 price probably contributes to the placebo effect. It was expensive so it _must_ work.