3 ms·

Just want to point out a few things that didn't get covered in the other thread: - The standard CV risk calculators that we actually use in medicine don't do a

Just want to point out a few things that didn't get covered in the other thread: - The standard CV risk calculators that we actually use in medicine don't do a tremendous job at discriminating risk. The AUC for the pooled risk equations tends to be around 0.74. As a refresher, AUC can be interpreted as the probability that someone with CVD has a risk score that is higher than someone without. - This deep learning model had an AUC of about 0.7, similar to the pooled risk equation which was about 0.72 - Adding this deep learning model to the pooled risk equation did not change the AUC (0.72 -> 0.72). There may have been recategorization, but we can't tell from the paper as published. - The pooled risk equations in this study did not have the second most important component: the lipid profile. (Age is the most important risk factor.) That absence hamstrings the baseline model compared to the deep learning model. We aren't seeing the standard pooled risk equations here. - As my PI argues on Twitter, you could see this deep learning model as a very good age predictor. - The fact that the model is paying "attention" to blood vessels and other structures that are relevant to humans is a promising sign.

epmaybe 9y ago
> paying "attention" to blood vessels and other structures that are relevant to humans Why is that necessarily a good thing? The point of machine learning is to unveil relationships that we may or may not have thought of. For all we know there's a structure we've never looked at that provides better correlates with disease, and if a model pays attention to that it would be an equally promising sign. I understand that in medical research it would be best to have a physiologically relevant correlate that is being recognized by the model, but it doesn't mean that we should completely disregard the other pixels that a model could find important either.
- carbocation 9y ago
  I agree with you. It's a double edged sword. It's good because if you want to put something like this into clinical use, physicians are going to be skeptical if it's not interpetable. It's bad because, if humans have already identified the relevant structural relationships, there is less likely to be a big breakthrough. Nevertheless, one could imagine a model identifying features that humans wouldn't consider even within the structures that we already know about.

epmaybe 9y ago

> paying "attention" to blood vessels and other structures that are relevant to humans Why is that necessarily a good thing? The point of machine learning is to unveil relationships that we may or may not have thought of. For all we know there's a structure we've never looked at that provides better correlates with disease, and if a model pays attention to that it would be an equally promising sign. I understand that in medical research it would be best to have a physiologically relevant correlate that is being recognized by the model, but it doesn't mean that we should completely disregard the other pixels that a model could find important either.

carbocation 9y ago
I agree with you. It's a double edged sword. It's good because if you want to put something like this into clinical use, physicians are going to be skeptical if it's not interpetable. It's bad because, if humans have already identified the relevant structural relationships, there is less likely to be a big breakthrough. Nevertheless, one could imagine a model identifying features that humans wouldn't consider even within the structures that we already know about.