5 ms·
So first clarification is that heart failure != heart attack. Heart failure is a chronic condition where the heart is unable to pump hard enough to keep blood f
by brilee 7y ago
So first clarification is that heart failure != heart attack. Heart failure is a chronic condition where the heart is unable to pump hard enough to keep blood flowing through the body. Typically results in blood pooling in the leg, shortness of breath, etc.
The study avoids the obvious pitfall, which is to put different slices of one patient's data into both training and test. The press also reports the training accuracy (100%) when the test accuracy/sensitivity/precision metrics are all at around 98%.
Another encouraging sign is that when you dig into the 2% error rate, a majority of those errors turned out to be mislabeled data.
The study also acknowledges the following:
"Our study must also be seen in light of its limitations... First the CHF subjects used in this study suffer from severe CHF only...could yield less accurate results for milder CHF."
I think this is a good proof of concept but that the severe CHF and tiny sample size (33 patients) means that we're a long ways away from clinical usage.
- RosanaAnaDana 7y agoWell, the issues are fundamental to the calculation of their error statistics. These models and their error rates are, well, crap. If any of the people in my group came back to me with this as an error assessment, they'd be re-doing their work.
- carbocation 7y agoThe study looks at 33 patients total, and the cases and controls come from entirely different data sets, with data coming from different devices that recorded signal at different frequencies. There is nothing to see here.
- RosanaAnaDana 7y agoYou realize the fact that the validation dataset is independent of the training dataset is a YUGE red flag right?
- carbocation 7y agoWhere do they say that the validation dataset is independent of the training dataset? The case dataset is independent of the control dataset.
- brilee 7y agoApologies - I didn't catch that the HF / healthy patients are from two different datasets. Agreed that this essentially invalidates the result. The missing experiment is to have a third dataset from yet another machine, with both positive/negative examples, and use it as the test dataset. Then transferability questions are at least somewhat addressed.
- carbocation 7y agoNo apologies needed, we're all in agreement on the broad strokes! Your proposal is good: I would be quite surprised if it generalized, but that is definitely the way to find out.