3 ms·

This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations. You can argue about the ter

This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations. You can argue about the terms "negative control" (which is appropriate here, as there is a positive control test in the paper), but there are only 4 non-cancer samples tested. That is not enough to be able to adequately know range of measurable values in the population of patients w/o cancer. But, that's not really the point. They aren't trying to diagnose cancer vs. healthy. Error bars here are absolutely necessary. Two reasons: First, you want to know the approximate ranges for each group in Figures 3 and 5. Not showing them is misleading. Secondly -- you actually also want error bars for each patient sample. I'd expect for there to be at least three replicates for each saliva sample to show that the strips are able to consistently measure a known value from each sample. I also mis-read part of the paper the first time. For the HER2 cases, there aren't 4 negative samples -- there are 20. There is only one positive sample. Part of the problem is really how they are presenting the data -- it is not all clear what they are testing. But, there is only one HER2+ sample in the mix. One... N=1. Samples include: * Non-cancer: 4 * In situ cancer: 3 * Invasive cancer, HER2-: 13 * Invasive cancer, HER2+: 1 What you'd really like to show is that the HER2+ patients could be differentiated from HER2- patients. Which, does look really good, but with only one HER2+ sample, you really can't tell much. (And the presence of so much signal in the HER2- samples raises some very interesting biological/mechanistic questions). Note: I'm not trying to say that the authors of the study are wrong or are trying to deliberately mislead people. There is so much here that could have been corrected to make this a much stronger paper. To me, this seems like a paper where the authors are likely engineers and not that well versed in biomedical statistics. The paper is published in a physics journal, so the journal itself is not a good place to make some of these arguments. Is the idea of a non-invasive test worthwhile? Yes! Absolutely. But they didn't show that it was a good test of clinical utility. They showed that it could measure differences in protein concentrations from saliva. That's not nothing, but that's it. Now, if that is an appropriate way to differentiate patients is a completely different question and requires substantially more testing (and orders of magnitude more patients).

jncfhnb 3y ago
> This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations. Complete Nonsense > That is not enough to be able to adequately know range of measurable values in the population of patients w/o cancer. True. But it is a promising initial signal that the distribution of non cancerous folks is probably very different from the invasive cancer folks. The effect size here is huge. “Range” is less interesting than “Distribution” > Error bars here are absolutely necessary. Two reasons: First, you want to know the approximate ranges for each group in Figures 3 and 5. Not showing them is misleading. You don’t really need error bars when you’re showing all of a small number of data points. But sure whatever > Secondly -- you actually also want error bars for each patient sample. I'd expect for there to be at least three replicates for each saliva sample to show that the strips are able to consistently measure a known value from each sample. Would be nice. Sounds like they did ten measurements per. > What you'd really like to show is that the HER2+ patients could be differentiated from HER2- patients. Which, does look really good, but with only one HER2+ sample, you really can't tell much. (And the presence of so much signal in the HER2- samples raises some very interesting biological/mechanistic questions). I’m not clear why you’re saying HER+ vs HER- is the important difference here.
- mbreese 3y ago
  Look, you obviously have your opinions here. I'm not sure just saying "Complete nonsense" at things is really all that helpful. What I said ("it doesn't tell you anything about the ability to differentiate between the three populations") is quite correct. This study shows that there is a difference between the groups of samples tested with their HER2 test strip, with a one-way p-value of ~0.002. I'm not convinced that the samples are representative of their populations. The number of non-cancer samples too low. >* You don’t really need error bars when you’re showing all of a small number of data points.* Error bars are visually helpful ways to show that the group values overlap. Which, in this case, they do (I did replot this data to confirm). >* Would be nice. Sounds like they did ten measurements per.* This is for one test. They sampled the test strip 10 times. I mean they should have tested each sample on at least 3 different test strips to get a mean value for the sample. This is a paper that is trying to say that their test strips are accurate, so it would make sense to test them multiple times. >I’m not clear why you’re saying HER+ vs HER- is the important difference here. I'm not sure what they are trying to claim in there paper... are they trying to say that they can diagnose breast cancer (which would requires many more biomarkers), or are they trying to say that they can differentiate between HER2+ and HER2- cancers (which would be more appropriate for a HER2 test). The other biomarker has even more overlap, so not sure how helpful that would be. Really, I think they are also missing an opportunity -- the bigger use for me would be in longitudinal testing. If they could show changes in signal over time for a particular patient that corresponded to treatment status -- that would be a great use for a cheap non-invasive test.

jncfhnb 3y ago

> This is an unbalanced study. As such, it doesn't tell you anything about the ability to differentiate between the three populations. Complete Nonsense > That is not enough to be able to adequately know range of measurable values in the population of patients w/o cancer. True. But it is a promising initial signal that the distribution of non cancerous folks is probably very different from the invasive cancer folks. The effect size here is huge. “Range” is less interesting than “Distribution” > Error bars here are absolutely necessary. Two reasons: First, you want to know the approximate ranges for each group in Figures 3 and 5. Not showing them is misleading. You don’t really need error bars when you’re showing all of a small number of data points. But sure whatever > Secondly -- you actually also want error bars for each patient sample. I'd expect for there to be at least three replicates for each saliva sample to show that the strips are able to consistently measure a known value from each sample. Would be nice. Sounds like they did ten measurements per. > What you'd really like to show is that the HER2+ patients could be differentiated from HER2- patients. Which, does look really good, but with only one HER2+ sample, you really can't tell much. (And the presence of so much signal in the HER2- samples raises some very interesting biological/mechanistic questions). I’m not clear why you’re saying HER+ vs HER- is the important difference here.

mbreese 3y ago
Look, you obviously have your opinions here. I'm not sure just saying "Complete nonsense" at things is really all that helpful. What I said ("it doesn't tell you anything about the ability to differentiate between the three populations") is quite correct. This study shows that there is a difference between the groups of samples tested with their HER2 test strip, with a one-way p-value of ~0.002. I'm not convinced that the samples are representative of their populations. The number of non-cancer samples too low. >* You don’t really need error bars when you’re showing all of a small number of data points.* Error bars are visually helpful ways to show that the group values overlap. Which, in this case, they do (I did replot this data to confirm). >* Would be nice. Sounds like they did ten measurements per.* This is for one test. They sampled the test strip 10 times. I mean they should have tested each sample on at least 3 different test strips to get a mean value for the sample. This is a paper that is trying to say that their test strips are accurate, so it would make sense to test them multiple times. >I’m not clear why you’re saying HER+ vs HER- is the important difference here. I'm not sure what they are trying to claim in there paper... are they trying to say that they can diagnose breast cancer (which would requires many more biomarkers), or are they trying to say that they can differentiate between HER2+ and HER2- cancers (which would be more appropriate for a HER2 test). The other biomarker has even more overlap, so not sure how helpful that would be. Really, I think they are also missing an opportunity -- the bigger use for me would be in longitudinal testing. If they could show changes in signal over time for a particular patient that corresponded to treatment status -- that would be a great use for a cheap non-invasive test.