4 ms·
Reading and doing minor edits is much less of a cognitive load than writing. The article does not suggest that doctors should blindly trust the SOAP note creat
by williamcotton 2y ago
Reading and doing minor edits is much less of a cognitive load than writing.
The article does not suggest that doctors should blindly trust the SOAP note created by the tool in question.
- ceejayoz 2y ago> The article does not suggest that doctors should blindly trust the SOAP note created by the tool in question. But that's what will inevitably happen at some point, when they get to the point of only rarely making big dangerous mistakes.
- MichaelZuo 2y agoIsn't the vast majority of the world using computers without ECC memory already blindly trusting there are no bit flips causing silent corruption?
- snoman 2y ago> …when they get to the point of only rarely making big dangerous mistakes. Are you intentionally rubbing FUD on that or am I mis-reading you? I don’t think we need to wait to rely on technologies until they’ve achieved perfection - just when their mistakes are less frequent, less dangerous, or more predictable than human mistakes for the same task.
- ceejayoz 2y agoI'm saying getting close to perfection is truly dangerous territory, because everyone gets very complacent at that point. As a concrete example: https://www.cbsnews.com/news/pilots-fall-asleep-mid-flight-150-on-board-batik-air-indonesia/ https://www.cbsnews.com/news/pilots-fall-asleep-mid-flight-1...
- danielmarkbruce 2y agoWe are already there with humans. Most people take a doctor at their word and don't bother to get a second opinion.
- pixl97 2y agoExactly. If you have any kind of illness this is displaying atypical symptoms or otherwise may be rare, your life is in your own hands. Even something that is somewhat common like EDS can get you killed by doctors missing the signs. Keep a printout of all our own symptoms as they evolve over time, and immediately bring up anything that conflicts with what the doctor says.
- ceejayoz 2y agoAt that point, you've at least consulted with a medically trained professional who's licensed (which they have to regularly renew), has to complete annual CME, can be disciplined by a medical board, carries medical malpractice insurance, etc. There should be requirements for any AI tool provider in the medical space to go through something like an IRB (https://en.wikipedia.org/wiki/Institutional_review_board https://en.wikipedia.org/wiki/Institutional_review_board) given they're fundamentally conducting medical experimentation on patients, and patients should have to consent to its use.
- danielmarkbruce 2y agoIn the context described, it's acting as a tool for a doctor. AI scribes are not conducting experiments.
- ceejayoz 2y agoThe use of the AI to treat patients is a medical experiment.
- danielmarkbruce 2y agoany change to the practice is an experiment.
- williamcotton 2y agoI don’t think we’re yet in a position where we can make claims about how inevitable certain outcomes are. It is important to remind people that technology of any sort can be error prone and that human oversight should be relied on for any automated process, LLM based or not! I work in the legal industry and every lawyer is aware of the guy who used ChatGPT to spit out non-existing case law!
- nsagent 2y agoApparently not enough to prevent it from being repeated... One of Trump's former lawyers did it months after the first case made national news. https://text.npr.org/2023/12/30/1222273745/michael-cohen-ai-fake-legal-cases https://text.npr.org/2023/12/30/1222273745/michael-cohen-ai-...
- chollida1 2y ago> But that's what will inevitably happen at some point, when they get to the point of only rarely making big dangerous mistakes. So the same as doctors making occasional big dangerous mistakes that cause lives. Seems like it would be a win then as it takes some mental load off of doctors so they can focus on where they should, on the patients and not on note taking.
- ceejayoz 2y ago> So the same as doctors making occasional big dangerous mistakes that cause lives. Will it be? There are already unanswered questions on who's liable if Tesla's FSD runs you into someone.
- pixl97 2y agoI would assume you, the person behind the wheel of the car. Much the same the doctor/staff hitting the submit button on the validity of the records statement.
- ceejayoz 2y agoSo Boeing should not have any liability for the MCAS crashes, because the pilots were in control?
- williamcotton 2y agoThere's not much that a pilot can do when a plane is not working correctly. They can recognize the issue but they might not be in a position to do anything about it. If an auto-form filler is not working correctly the doctor can also recognize the issue and also be in a position to do something about it, namely, fix the error before they submit the form. That is to say that there's a world of a difference between a pilot flying a plane and a doctor filling out a form.
- deleted 2y ago[deleted]
- dv_dt 2y agothinking reading and doing edits as less work than entering it yourself is exactly what will cause critical errors to be made. It may not suggest that but just like there are Tesla drivers who are supposed to watch the road there are users who will not check. And in a medical record that can be deadly.