4 ms·

Yes. Perhaps pushing sophisticated models that require validation, on users who most definitely won't (and the whole premise of the model itself is that it spar

Yes. Perhaps pushing sophisticated models that require validation, on users who most definitely won't (and the whole premise of the model itself is that it spares them work) was a bad idea...

bpodgursky 2mo ago
Validation will measure accuracy, and even if you require and get > 99.9% accuracy (and they should! don't get me wrong), it won't change that there will be anecdotes of patients getting wrong results given that millions of people go to the doctor a year. In fact, there will be MORE of those stories because it is shocking and strange, and clickworthy! The readers demand stories! But the reality is that there is no world where a story like this meaningfully informs the public about the accuracy of the services in question and the tradeoffs involved.
- NitpickLawyer 2mo ago
  > even if you require and get > 99.9% accuracy [...] there will be anecdotes of patients getting wrong results given that millions of people go to the doctor a year. This is known as the long tail problem in ML, and it's a reality in almost every field. It's also why we don't officially have self driving cars, despite there being thousands of videos out there with cars driving autonomously for hours without any errors. But every now and then, there will be cases where the system doesn't work. There are two interesting aspects here. One, we don't actually have quantitative data on how often this happens in the same field when human errors don't get caught. In a perfect world we'd have that, plus a long study for the "AI" systems, and we'd get to compare the two. Secondly, even if we'd have that data, people would still act out against "the machine" in the (ideally fewer) cases where it errors out, compared to a doctor. It's part human nature, part (manufactured) rage against the machines. I also agree with your second paragraph. Case in point, when a waymo hit a cat, we got a shit ton of articles, riled up communities, and so on. Or every time that other car hits something we get plenty of press, even if some of the "accidents" are fender benders that likely wouldn't get reported otherwise.
  - Terr_ 2mo ago
    Another aspect I like to harp on about is that the shape and kind of error is different. So even if a system has the same rates as humans, if we can't solve for the mismatch in our ability to predict and recover from errors, final outcomes will be worse.

Yes. Perhaps pushing sophisticated models that require validation, on users who most definitely won't (and the whole premise of the model itself is that it spares them work) was a bad idea...

bpodgursky 2mo ago

Validation will measure accuracy, and even if you require and get > 99.9% accuracy (and they should! don't get me wrong), it won't change that there will be anecdotes of patients getting wrong results given that millions of people go to the doctor a year. In fact, there will be MORE of those stories because it is shocking and strange, and clickworthy! The readers demand stories! But the reality is that there is no world where a story like this meaningfully informs the public about the accuracy of the services in question and the tradeoffs involved.

NitpickLawyer 2mo ago
> even if you require and get > 99.9% accuracy [...] there will be anecdotes of patients getting wrong results given that millions of people go to the doctor a year. This is known as the long tail problem in ML, and it's a reality in almost every field. It's also why we don't officially have self driving cars, despite there being thousands of videos out there with cars driving autonomously for hours without any errors. But every now and then, there will be cases where the system doesn't work. There are two interesting aspects here. One, we don't actually have quantitative data on how often this happens in the same field when human errors don't get caught. In a perfect world we'd have that, plus a long study for the "AI" systems, and we'd get to compare the two. Secondly, even if we'd have that data, people would still act out against "the machine" in the (ideally fewer) cases where it errors out, compared to a doctor. It's part human nature, part (manufactured) rage against the machines. I also agree with your second paragraph. Case in point, when a waymo hit a cat, we got a shit ton of articles, riled up communities, and so on. Or every time that other car hits something we get plenty of press, even if some of the "accidents" are fender benders that likely wouldn't get reported otherwise.
- Terr_ 2mo ago
  Another aspect I like to harp on about is that the shape and kind of error is different. So even if a system has the same rates as humans, if we can't solve for the mismatch in our ability to predict and recover from errors, final outcomes will be worse.