4 ms·
You're operating in a really serious area of medicine and I encourage you to takes the comments about false positives (and false negatives) more seriously. It
by dekhn 5y ago
You're operating in a really serious area of medicine and I encourage you to takes the comments about false positives (and false negatives) more seriously. It's not really just a matter of making adjustments to a model; it has to be pervasive in the entire process of making reproducible modeels that are used for making decisiojns about humans.
- coolwulf 5y agoThe website is cleared marked as a breast health awareness tool and not for diagnosis. We are not doing any decision making for humans. However I would like to point out there are quite some FDA approved mammo AI products in the market at the moment.
- dekhn 5y agoSo, you think if you put up a website, people won't use it for diagnosis anyway? yes, I know mammo AI products are on the market- those that were approved followed a collection of regulations that I think you are not.
- endisneigh 5y agoEven if people use it for diagnosis, so what? Take the info and if it’s positive to confirm with doctor, if it’s negative but you have reservations go see doctor. It doesn’t really change anything.
- dekhn 5y agoso you're saying the test is worthless in terms of actionability? Why woudl you want to take it then?
- endisneigh 5y agoIt’s not that it’s worthless, it’s simply another data point to give to a professional.
- ska 5y agoI think this comment is perhaps a bit uncharitable. Even if you have done everything you are supposed to do in the process, at the end of the day you are looking at ROC curves or equivalent and trying to understand sensitivity vs. specificity trade offs, and often you have some sort of (indirect) parameters than can move to different points of that tradeoff. This is quite critical in deployment, if you are screening you usually want something different than in diagnosis; as alluded to elsewhere if you raise the work-up rate too much you definitely risk killing more people from biopsy complications than you help with higher sensitivity (it's more complicated than that in practice)
- dekhn 5y agodeploying ML medical diagnostics is like everything else in ML: the ML part is 1% of a much larger "thing", which involves business, legal, and many other concerns far beyond the data analytics. Nothing abotu what I'm saying is uncharitable- in a sense, it's charitable because I'm helping warn a person who is going down a dangerous path to consult more with experts in this field.
- ska 5y agoThe charitable response would have been to assume they do have some understanding the broader context, and perhaps raise specific concerns or points of interest. What you did was assume that they were ignorant in potentially dangerous ways, and assert they should do something different. I don't think a reading of the comments/responses (at least at your time of posting) really supported that assumption, especially considering the limitations of the medium. Hence my reply to you, while also detailing the trade offs a tiny bit.
- dekhn 5y agoI read the entire article and I didn't see anything in there that would convince me this author is anything other than an amateur programmer (I can't parse the section about bruker- is the "amateur programmer" also a director at Bruker who ddevelops medical devices full-time"?). Please be assured that I put a fair amount of thought into this - for example, I used to do due diligence for a VC firm evaluating proposals like this all the time and we had to reject most of them because the founders didn't understand the basic rules of deploying medical technology in highly regulated environments like the US. Based on my interactions with the author in the various parts this post, I continue to conclude this individual is lacking core knowledge and wisdom required to execute a project like this successfully at scale.