6 ms·
This sounds about right to me intuitively. I've read a ton of ML/AI papers in the area, and the vast majority don't get past the "maybe this is an interesting
by ska 7y ago
This sounds about right to me intuitively. I've read a ton of ML/AI papers in the area, and the vast majority don't get past the "maybe this is an interesting idea, hard to know until it's tested properly". Easy to recall interesting ones, hard to remember good one off the top of my head.
This is fine; of course the standards for "is this publishable" are vastly lower than "is this clinically applicable", but it unfortunate how often the lay science press fails to make this distinction.
The other thing is that raw performance numbers, even if properly validated, are only a small part of the story. We've had ML techniques that did better nominal sense/spec than average clinicians for very specific tasks for 20+ years now, but the workflow impact, liability, etc. combined has sometimes limited the implementation, and even if you do roll it out the actual impact can be minimal.
- davnn 7y agoHave a look at the Stanford Machine Learning Group projects [1]. What most articles miss in my opinion is that AI progress heavily depends in the data being available and that‘s where there are huge opportunities in the future. [1] https://stanfordmlgroup.github.io/ https://stanfordmlgroup.github.io/
- ska 7y agoThey have a good group and are thinking/heading in the right direction, mostly not there yet I think. I certainly don't disagree that there is potential for impactful use of AI in some areas of healthcare. Some of them just aren't going to practically happen without a) fundamentally reworking our approaches to data access and sharing and more importantly b) spending a lot of resources on quality labeling. Regardless of whether or not those preconditions are actually met, we'll get a lot of breathless claims on small retrospective sets, of course :)
- 0x8BADF00D 7y agoMedicine is one area where measurements are key in determining health outcomes. I don’t see any reason why an inference engine wouldn’t be at least as good as a human doctor. Although, self-supervised learning will need to become a reality, before these systems have any practical use IMO.
- ska 7y agoDon't misunderstand me - I think there is real potential in to make an impact in the area but there are systemic and cost issues to doing it well. That said, the problems are harder than appears to your typical ML grad student, and structurally we aren't there with data. There is no reason currently to think self supervised with "get there" in the near term, but even if you've assumed infinite labeling resources the data sets are for the most part still too disjoint and unavailable. Some strides have been made in some areas recently, but a long way to go. The algorithms part isn't necessarily that interesting really, building up the supporting infrastructure is the real problem.