3 ms·

Some success: 30% effectiveness in that 51 out of 8000 given the vaccine because HIV positive, and 74/8000 given the placebo did. I presume that really is stat

by redcap 17y ago

Some success: 30% effectiveness in that 51 out of 8000 given the vaccine because HIV positive, and 74/8000 given the placebo did. I presume that really is statistically significant despite the chance it could have just been random? They also don't know how it works...

timr 17y ago
"Although the difference was small, Dr. Kim said it was statistically significant and meant the vaccine was 31.2 percent effective." The confidence interval is determined by the number of people in the trial (N=16,400), not the number of people who get sick.
- sambrightman 17y ago
  If the rate of infection in the general population was even lower, say 1 in 100,000, and they actually trialled 200,000 people, finding that the vaccine perfectly prevented infection, should they conclude this with high confidence? Or have I misunderstood you? Besides, Dr Kim is quoted elsewhere as saying "the result could be due to chance" and the vaguely phrased "31.2 percent effective" is not encouraging. Add in the likely funding source of AIDS vaccine trials, the pressure to get results and the unknown mechanism and it doesn't look so great.
  - timr 17y ago
    The result can always be due to chance. Confidence intervals let you estimate how likely it is that your results are due to chance. In this case, the results are unlikely to be random -- but it's always possible. That's all he was saying. Maybe this study was the 1 in 10,000 that would randomly show a 30% difference in infection rates. Is that likely? No. Is it possible? Sure. Anything's possible. Confidence intervals get smaller as you include more people in an experiment, but they get smaller more slowly as more people are added. So it's fairly easy to go from a 2% to a 1% confidence interval by adding a couple thousand people, but it takes a lot more people to go from 1% to 0.5%. And of course, it's asymptotic -- you'll never get to zero. That said, in this case, the point is academic. This study was large enough that the confidence interval is going to be well below 30%.
  - earl 17y ago
    You didn't understand the parent. See my post below. Also, of course the result could be due to chance. You could flip a fair coin 25 times in a row and come up with heads each time -- it's just pretty damn unlikely. Similarly, I'd bet what he's measured is really this: you have two equally sized groups, one treated, one untreated. In group one you get a rate of x, and in group two, you get a rate of y. How big does |y-x| and the size of the two groups have to be to conclude it was not just chance? As below, a t-test can help you answer that.
  - psyklic 17y ago
    The key is not to find "who gets HIV" vs. "who does not get HIV." Instead, we are trying to measure "who gets exposed" vs. "who gets HIV." Although we do not know how many people are exposed in each group, we can make sure that the number of exposed people in each group is approximately the same by increasing the number of people vaccinated. If we know that the same number of people are exposed (which is less than the sample size), then the number of people who get sick is much more significant. By increasing the number of vaccinated people, the number of exposed people becomes more equal between the two groups since a few "more exposed/risky people were assigned to group A" will have a less significant impact.
cgherb911 17y ago
I have to agree that the numbers don't seem to differ significantly enough to draw any conclusive evidence. I agree with your math in saying that if you had the vaccine, your chance of infection is .6375% and without the chance of infection is .925%. From this perspective I don't see any hard numbers to back the claim. I'll wait for a more better controlled experiment before I believe this one.
- deleted 17y ago
  [deleted]
earl 17y ago
I presume that the investigators used something like the t-test for a difference of means. You treat 1/2 the group and don't treat the other; if you selected these two groups randomly, then they should a priori have equal likelihood of infection. You then can create a confidence interval for the hypothesis: are the two means / rates different. See the second use here: http://en.wikipedia.org/wiki/Student%27s_t-test http://en.wikipedia.org/wiki/Student%27s_t-test
tokenadult 17y ago
http://healthpolicy.stanford.edu/news/even_modestly_effective_hiv_vaccine_would_yield_substantial_benefits_researcher_finds_20040616/ http://healthpolicy.stanford.edu/news/even_modestly_effectiv...

by redcap 17y ago

timr 17y ago

"Although the difference was small, Dr. Kim said it was statistically significant and meant the vaccine was 31.2 percent effective." The confidence interval is determined by the number of people in the trial (N=16,400), not the number of people who get sick.

sambrightman 17y ago
If the rate of infection in the general population was even lower, say 1 in 100,000, and they actually trialled 200,000 people, finding that the vaccine perfectly prevented infection, should they conclude this with high confidence? Or have I misunderstood you? Besides, Dr Kim is quoted elsewhere as saying "the result could be due to chance" and the vaguely phrased "31.2 percent effective" is not encouraging. Add in the likely funding source of AIDS vaccine trials, the pressure to get results and the unknown mechanism and it doesn't look so great.
- timr 17y ago
  The result can always be due to chance. Confidence intervals let you estimate how likely it is that your results are due to chance. In this case, the results are unlikely to be random -- but it's always possible. That's all he was saying. Maybe this study was the 1 in 10,000 that would randomly show a 30% difference in infection rates. Is that likely? No. Is it possible? Sure. Anything's possible. Confidence intervals get smaller as you include more people in an experiment, but they get smaller more slowly as more people are added. So it's fairly easy to go from a 2% to a 1% confidence interval by adding a couple thousand people, but it takes a lot more people to go from 1% to 0.5%. And of course, it's asymptotic -- you'll never get to zero. That said, in this case, the point is academic. This study was large enough that the confidence interval is going to be well below 30%.
- earl 17y ago
  You didn't understand the parent. See my post below. Also, of course the result could be due to chance. You could flip a fair coin 25 times in a row and come up with heads each time -- it's just pretty damn unlikely. Similarly, I'd bet what he's measured is really this: you have two equally sized groups, one treated, one untreated. In group one you get a rate of x, and in group two, you get a rate of y. How big does |y-x| and the size of the two groups have to be to conclude it was not just chance? As below, a t-test can help you answer that.
- psyklic 17y ago
  The key is not to find "who gets HIV" vs. "who does not get HIV." Instead, we are trying to measure "who gets exposed" vs. "who gets HIV." Although we do not know how many people are exposed in each group, we can make sure that the number of exposed people in each group is approximately the same by increasing the number of people vaccinated. If we know that the same number of people are exposed (which is less than the sample size), then the number of people who get sick is much more significant. By increasing the number of vaccinated people, the number of exposed people becomes more equal between the two groups since a few "more exposed/risky people were assigned to group A" will have a less significant impact.

cgherb911 17y ago

I have to agree that the numbers don't seem to differ significantly enough to draw any conclusive evidence. I agree with your math in saying that if you had the vaccine, your chance of infection is .6375% and without the chance of infection is .925%. From this perspective I don't see any hard numbers to back the claim. I'll wait for a more better controlled experiment before I believe this one.

deleted 17y ago
[deleted]

earl 17y ago

I presume that the investigators used something like the t-test for a difference of means. You treat 1/2 the group and don't treat the other; if you selected these two groups randomly, then they should a priori have equal likelihood of infection. You then can create a confidence interval for the hypothesis: are the two means / rates different. See the second use here: http://en.wikipedia.org/wiki/Student%27s_t-test http://en.wikipedia.org/wiki/Student%27s_t-test

tokenadult 17y ago

http://healthpolicy.stanford.edu/news/even_modestly_effective_hiv_vaccine_would_yield_substantial_benefits_researcher_finds_20040616/ http://healthpolicy.stanford.edu/news/even_modestly_effectiv...