3 ms·
Great project and really cool to see this. Question I have for you is that one of the biggest problems with cancer diagnoses is false positives: "Yes, there is
by hn_throwaway_99 5y ago
Great project and really cool to see this.
Question I have for you is that one of the biggest problems with cancer diagnoses is false positives: "Yes, there is something on your scan, we're not sure what it is, so we'll biopsy it." Biopsy is not a 0-risk procedure, and it can cause a lot of worry and pain, so it's not something to be taken lightly. Also, there are many cases of "OK, it's probably cancer, but the cure may be worse than the disease." This is the classic problem with detecting prostate cancer at an advanced age - it's very likely/probable something else will kill you before the cancer does.
How does your software deal with this issue? I'd be worried if, as you put it, people in a remote area with limited access to experienced radiologists, were given access to this and it came back with "Fairly decent chance of cancer" - what do they do then?
- coolwulf 5y agoGreat question. Definitely sensitivity / specificity balance is a crucial topic in AI-assisted diagnosis. I have to admit this model and website for mammography was done in 2018 and might not be the leading solution out there. At the moment, if I want to improve the results of my earlier work, I will add additional stage of radiomics model to do false positive reduction, and in the mean time lower the threshold in the first model to increase sensitivity. By doing this, combining the feature extractions of deep learning and the past 20 years of knowledge in medical imaging using Radiomics might give better performance in terms of sensitivity / specificity. Hao
- areoform 5y agoHow can we help? Is there some way for us to contact you regarding this?
- dekhn 5y agoYou're operating in a really serious area of medicine and I encourage you to takes the comments about false positives (and false negatives) more seriously. It's not really just a matter of making adjustments to a model; it has to be pervasive in the entire process of making reproducible modeels that are used for making decisiojns about humans.
- coolwulf 5y agoThe website is cleared marked as a breast health awareness tool and not for diagnosis. We are not doing any decision making for humans. However I would like to point out there are quite some FDA approved mammo AI products in the market at the moment.
- dekhn 5y agoSo, you think if you put up a website, people won't use it for diagnosis anyway? yes, I know mammo AI products are on the market- those that were approved followed a collection of regulations that I think you are not.
- endisneigh 5y agoEven if people use it for diagnosis, so what? Take the info and if it’s positive to confirm with doctor, if it’s negative but you have reservations go see doctor. It doesn’t really change anything.
- dekhn 5y agoso you're saying the test is worthless in terms of actionability? Why woudl you want to take it then?
- endisneigh 5y agoIt’s not that it’s worthless, it’s simply another data point to give to a professional.
- ska 5y agoI think this comment is perhaps a bit uncharitable. Even if you have done everything you are supposed to do in the process, at the end of the day you are looking at ROC curves or equivalent and trying to understand sensitivity vs. specificity trade offs, and often you have some sort of (indirect) parameters than can move to different points of that tradeoff. This is quite critical in deployment, if you are screening you usually want something different than in diagnosis; as alluded to elsewhere if you raise the work-up rate too much you definitely risk killing more people from biopsy complications than you help with higher sensitivity (it's more complicated than that in practice)
- billiam 5y agoYou will save even more lives if you over-communicate about the need to use your awareness tools (with better false positive reduction) to drive more effective diagnosis.
- deleted 5y ago[deleted]