7 ms·

I'm a healthcare CIO of 12 years, and I've evaluated 4 and deployed 2 of these tools, one of which is currently deployed at my currently healthcare employer. I

• t-kalinowski 6mo ago
  Counterpoint from a doctor: https://substack.com/inbox/post/189714240 https://substack.com/inbox/post/189714240 Scribes _feel_ good in the short-term, but it's not clear if they're actually good on longer time horizons.
  • razingeden 6mo ago
    the two places they come in handy: 1) in the event you find yourself partially or totally disabled but the records don’t really make a good case for it and your provider has a dismissive attitude about filling out additional documentation to substantiate what they failed to in your records. You’re not necessarily going to get approved for FMLA, STD, LTD, SS etc based on a diagnosis or test results alone. They will nitpick over say, heart failure, as if that’s magically and spontaneously going to go away. If you’re telling your provider that you’re limited by things like oh I don’t know, “I’m only awake for 2-4 hours before I need to sleep again” or “some days I just can’t do it and sleep 20 hours” but it’s not in your chart… expect denials and clarifications and a huge burden on you to prove why it’s limiting. 2) continuity of care, so you don’t end up explaining everything from the top to a specialist or having them run all these tests and procedures from square one — when there’s months long backlogs , and we already did all this and you need treatment - but - there wasn’t much to work with in your referring chart. You might not appreciate the “intrusion” if you’re healthy and just worried about your privacy. If/When things go south and you find yourself fighting these entities for a year or two or three while they nitpick and delay and deny and drag their feet , you’ll be glad an “AI” kept up meticulous records because this is phenomenally stressful and an endless burden on you when they don’t. So, their AI slop can vomit out all this extra info on why insurance companies should pay them or why your condition is in fact disabling, and now their AI slop can comb through it looking for all that. Because they will try to avoid paying or approving any kind of leave or benefits if it’s not there And god forbid you hand them a form where they’re being asked to explain themselves. 50/50 on them being eager to help out or rolling their eyes and saying something really nasty about the imposition. And then even when they do that, they almost never file a copy in your chart so your chart STILL doesn’t substantiate your claims. I’m all for an “ai” doing the progress notes in a case where the facility or provider can’t be fucked to do so. Happily that’s not true of my current provider, who just, does that anyway (?) But I’ve been around enough to know they’re an exception. Even when providers are on your side and mean well, and want to bend over backwards to help you in any way they can — and I want to just acknowledge that’s the situation I’m in today — honestly , sometimes they just forget some of the details when they do their notes. That’s why some places make the provider do it in real time while they’re talking to you, so they didn’t forget something relevant thirty minutes later. The other side of the coin here may be that some providers find that distracting or off putting to be typing away like a stenographer while they’re examining you… I think it would be fair to say this can all be tedious and a burden for both patients and providers. There’s just a world of difference between a provider who wants to do this to provide excellency in care, and a provider who wants to do this because they resent it and think it’s beneath them.
  • bonsai_spool 6mo ago
    > Counterpoint from a doctor: https://substack.com/inbox/post/189714240 https://substack.com/inbox/post/189714240 That article is clearly LLM-assisted if not vibe-written, which is the height of irony given the context. Note that the CIO is talking about patient satisfaction, which is a distinct target. I agree about the long-run benefit being unclear.
  • sigmar 6mo ago
    "I am not saying ambient scribes are bad technology." is this a counterpoint? he just seems to be wary of the risk, without a firm position and decided to personally stop using it. people often overestimate their own skills and think their own charting is better than that of others, that doesn't mean the tech doesn't work.
  • jimbokun 6mo ago
    In an article critiquing over-use of AI assistants, the author confesses at the end this article itself was authored partly by Claude that introduced errors in the citations, lol. Nonetheless, I come away from this article with the sense the ambient devices automating documentation of an encounter are still a net win, with caveats about the need for the doctor to polish the note ti reflect his or her own narrative voice.
  • burnte 6mo ago
    I think every single provider should evaluate them for themselves. Some providers are absolutely better of without them and we don't make anyone use them.
• mmooss 6mo ago
  If I allow it, is the data from my meeting sent offsite at any stage, for example to an LLM service (e.g., Anthropic, OpenAI, etc.)? Or do the LLM vendors (or any others) have access to the internal data at any stage?
  • burnte 6mo ago
    > If I allow it, Which is your right, every patient can ask the provider to not use it. > is the data from my meeting sent offsite at any stage Yes, no one stores medical records on-prem any more. EMR systems are not like Quickbooks running on an 8 year old terminal server. > for example to an LLM service Yes, that's literally what an AI transcriber is, an LLM. > (e.g., Anthropic, OpenAI, etc.)? No. The recording goes (in realtime) to our vendor's infra where it is live transcribed, then summarized and returned. When complete only the finished note is saved, never the recording or transcript. > Or do the LLM vendors (or any others) have access to the internal data at any stage? Obviously, you can't pricess data you can't access, but the contractual and regulatory environment means that data can't be used for additional training without lots of consents. We do not participate in training activities at all. I won't allow it.
    • mmooss 6mo ago
      Most of your responses are uncharitable readings of the questions - as if you are looking for targets for your contempt, which shows up in all but one answer. If you are contemptuous of questions and questioners, it looks like you don't take these issues seriously. I didn't think that before your response but I do now.
      • burnte 6mo ago
        I have no idea what you're talking about, I found your questions to be excellent and worthy of answering at every level of detail. I have no contempt at all, I assure you. I'm genuinely confused by your reply here.
        • mmooss 6mo ago
• yding 6mo ago
  When you evaluated the tools, what stood out between which ones were better or worse?
  • burnte 6mo ago
    A few things. I'm price sensitive, so pricing was huge for me. The worst company also had the worst prices. I tried to ask them questions about how their backend works and they refused to answer. I spoke with the CEO and he said he couldn't reveal their "secret sauce". I said, "if you secret sauce is what infra providers you use and not your proprietary code, then you don't HAVE secret sauce and you're just reselling [Cloud Provider's Product]." Turns out that's exactly what they were doing. They were using Google Cloud for recording capture, and AWS for speech to text and then summary generation. I told them we would not ever be working with them. For me the big things are price, ease of use, and data protection policies. I need to know the data never leaves the US, and I need to know what processors will touch it. Then if it meets those needs we'll do clinical demos and tests to get provider feedback. That's where we learn if it is clinically accurate. About half of them suck in the accuracy department. What stands out to me the most is that the best companies have tended to be the small guys who have a strong grasp ion the entire stack and have somewhat simple apps. They focus on the tech, and have a minimal UI that just focuses on the main tasks and they don't spend engineering time on fancy pretty bells and whistles. If you see a simple UI, that's a good sign to me. Once you hit the big guys the quality goes down. Dragon Medical One is great for straight text to speech, but Dragon with Copilot for medical is really bad.
• cromka 6mo ago
  But WHY not do this on premises? WHY?
  • dsr_ 6mo ago
    Money.
    • reaperducer 6mo ago
      It's strange to me that it's not already on-prem. I work in healthcare, and we spend oodles of time and money making sure every technology that can possibly be on-prem is. Maybe it's just not technically possible yet?
      • dsr_ 6mo ago
        You had it 20 years ago: doctors spoke into recorders, transcriptionists turned that into notes, the docs reviewed them. The first study I cited replaces the "spoke into recorders" stage with non-AI voice recognition. The second study replaces the "spoke into recorders" stage with LLM voice recognition, and... crucially... also replaces the educated transcriptionist step with nothing. I imagine that the real problem is that the voice recognition can be classic or LLM and it just doesn't matter as much as having two humans in the loop instead of one. But that's not a story which gets you to replace cheap voicerec with expensive AI.
      • quantumwoke 6mo ago
        A pretty insightful viewpoint I heard recently from a doctor friend: doctors and hospitals believe that only a corporation could possibly implement this, so they fall into the SaaS trap and lose data sovereignty. Under the hood, a lot of the companies are Llama or Gemma wrappers connected to whisper.
  • 16bytes 6mo ago
    Why would you want to have anything on prem? Have you seen what that looks like in a hospital system?
  • 6mo ago
• jubilanti 6mo ago
  I still don't want a fucking audio recorder in my doctor's office or a fucking AI that sits in between me and my doctor. I am intentionally cursing to express my anger at this casual betrayal of medical trust.
  • kube-system 6mo ago
    It is standard practice to ask patients whether or not they want the scribe used, and in many cases required by law.
    • jubilanti 6mo ago
      For now. It always begins as voluntary. But then doctors will start to treat people who opt out the way TSA treats me when I opt out: a hostile adversary. I already get glares and sighs when I dare to actually read every word of a multipage form I am expected to sign without reading. Was told once I would lose my appointment if I took longer than a few minutes to read more than 10 pages because I could not be checked in until I signed. Other patients are waiting, your exercise of your human rights is inefficient. Then soon I'll have to pay a higher copay to opt out. Then I won't be able to opt out at all. All in the name of optimizing patient NPS scores and patient throughput.
      • ryandrake 6mo ago
        > Was told once I would lose my appointment if I took longer than a few minutes to read more than 10 pages. I'd be finding a new doctor at that point. Ridiculous. I love it how doctors can be 30 minutes late for their appointments because they're running late and all their appointment delays are cascading, but if the patient reads a document for 5 minutes, they're the problem!
      • kube-system 6mo ago
        I've never had this problem. IME every doctors office recommends showing up 15-20 minutes early to a new-patient appointment for the explicit reason of filling out paperwork.
• wl 6mo ago
  I got an erroneous Type II diabetes diagnosis dropped into the note by the AI scribe at my last appointment because my PCP discussed the A1C test he was ordering. Would not recommend. That isn't to say that manually typed notes or speech to text dictated notes are perfect (dot phrases have ended up "documenting" plenty of conversations that never happened), but a false diagnosis of a chronic disease seems like a really bad failure.
  • burnte 6mo ago
    > got an erroneous Type II diabetes diagnosis dropped into the note by the AI scribe at my last appointment because my PCP discussed the A1C test he was ordering. No, you got an inaccurate diagnoses because your doctor didn't do their job. It's the provider's job to check notes, and this would have gotten that provider a visit with their clinical director at my org.
    • stephenbez 6mo ago
      By that line of thinking, even if AI scribes are terrible, you can only blame the doctor because they didn’t check their notes. In this case as the patient, all you care is there was an inaccurate diagnosis in your notes. If the doctor were typing them up by hand, presumably that would not have happened. Similarly if Tesla Self Driving cars got into collisions at 3x the rate of non-self driving, would you defend Tesla because all issues are the drivers fault who are supposed to have their hands on the wheels and paying attention?
      • burnte 6mo ago
        > By that line of thinking, even if AI scribes are terrible, you can only blame the doctor because they didn’t check their notes. Same for any profession. If you use bad tools expect bad outcomes. Yes, I work in a company that expects employees to do their work well, and there are consequences to bad performance. > In this case as the patient, all you care is there was an inaccurate diagnosis in your notes. If the doctor were typing them up by hand, presumably that would not have happened. Doctors can absolutely mischart by hand. Human error is one of many reasons we moved to electronic charting. We have providers who love the tool and we see benefits from them having it, and some providers don't want it, so they don't have to use it. I've seen people say they feel it's slower, didn't like the output, and one provider simply enjoys charting. They're all good providers too. > Similarly if Tesla Self Driving cars got into collisions at 3x the rate of non-self driving, would you defend Tesla because all issues are the drivers fault who are supposed to have their hands on the wheels and paying attention? Bad analogy, since FSD is supposed to work without the human in the loop. This tool EXPLICITLY is to be operated and checked by a human. That said, I wouldn't defend Tesla at all, however yes I would state that if you are supposed to monitor a "self driving" car and get into an accident, you failed. In that case I'd say it's safer to just manually drive, and many of our providers choose to chart manually.
• dsr_ 6mo ago
  Pre-AI voice recognition (2018), followed by 2 human reviews https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2687052 https://jamanetwork.com/journals/jamanetworkopen/fullarticle... => the error rate was 7.4% in the version generated by speech recognition software, 0.4% after transcriptionist review, and 0.3% in the final version signed by physicians. Among the errors at each stage, 15.8%, 26.9%, and 25.9% involved clinical information, and 5.7%, 8.9%, and 6.4% were clinically significant, respectively. AI "scribes" in a perfectly replicable best-of-all-worlds scenario (2025): https://bmjdigitalhealth.bmj.com/content/1/1/e000092 https://bmjdigitalhealth.bmj.com/content/1/1/e000092 => Omissions dominated error counts (83.8%, p<<0.001), with CAISs varying widely in error frequency and severity, and a median of 1–6 omissions per consultation (depending on CAIS). Although less frequent, hallucinations and factual inaccuracies were more often clinically serious. No tested CAIS produced error-free summaries. On the gripping hand, people who work in the management end of the US healthcare industry can't be trusted with healthcare or information security to begin with.
  • joshstrange 6mo ago
    > On the gripping hand, It’s been a year or so since I last read The Mote In Gods Eye/The Gripping Hand but I randomly was thinking of this morning. Very funny that I would see a reference to it the same day.
  • kube-system 6mo ago
    Errors can be a significant problem in manual charting as well. I know a medical professional who does a similar evaluation process to what is outlined in your second link to human written charts. They then use that feedback to guide the department on how to improve their charting. So, don't presume that those error rates cited in those studies should be compared to a baseline rate of zero. If you review human-written charts, you will often also not have an error rate of zero.
    • fc417fc802 6mo ago
      Has anyone considered simply asking the patient to sign off on these things as well? I realize many wouldn't but at least some would.
• sonofhans 6mo ago
  I’ve been in tech and medicine too. Consider that any “HUGE” effect in this context is likely exaggerated, especially for something as prosaic as a note-taking assistant. As a patient sitting with a doctor, I don’t care how standardized the notes are. I don’t care about anyone’s NPS score. I do want the doctor to connect with me, but I also remember not too long ago when doctors did this anyway, without any assistance from robots.
  • reaperducer 6mo ago
    I also remember not too long ago when doctors did this anyway, without any assistance from robots. Or with assistance from other humans. The last time I had surgery, every time I met with the surgeon (about six times), he had an intern following him around with a Thinkpad, typing in everything said. The intern has the ability to understand context, idiomatic expressions, emotion, and a dozen other important and useful things that an AI transcription will never capture.
    • dpark 6mo ago
      That’s probably not an intern. Doctors with enough pull can get dedicated scribes like this, but they aren’t cheap, which is why most doctors don’t get them.
  • nitwit005 6mo ago
    If there is a large effect, I'd expect it's excitement about a new thing. Positive survey feedback certainly isn't a bad sign, but people can get very excited about cool new technologies, even ones that ultimately fail.
  • burnte 6mo ago
    > I’ve been in tech and medicine too. Consider that any “HUGE” effect in this context is likely exaggerated, especially for something as prosaic as a note-taking assistant. Imagine your doctor head down writing down everything you say. Now imagine your doctor looking you in the eye and listening intently. Which do you think feels better to the patient? That is "huge". Anything that helps improve patient care with little effort and cost IS HUGE to us. That feeling of the doctor being present and invested helps patient outcomes. THAT is also huge, even if it's a few percent. We're healing people, we're not looking for a unicorn startup, a few percent improvement IS HUGE to us. > As a patient sitting with a doctor, I don’t care how standardized the notes are. Yes you do, better notes mean better care because the next time your seen your records are clean, understandable, and compliant with regulations and best practices. Better notes mean doctors are following protocols. Better notes mean fewer claim rejections, and fewer claim rejections means less money wasted arguing with insurance companies. Better notes mean the data is more easily used for research, as well, which leads to new treatments and better outcomes. > I don’t care about anyone’s NPS score. Ever had a doctor with a bad bedside manner? Missed a diagnosis? Skips appointments on fridays? Tracking NPS scores can help with that. Every data point is useful, and patient satisfaction is massive. > I do want the doctor to connect with me, Ok, well, most people DO want this, most people DO want to have a good relationship with their doctor where they feel heard and cared about rather than just another widget on a conveyor belt. > but I also remember not too long ago when doctors did this anyway, without any assistance from robots. I also remember when doctors weren't constantly overruled by insurance companies. Ever heard of a Prior Auth? That's when your doctor writes a prescription or an order and then the insurance company makes the doctor call them back and say "yes, I did this on purpose, yes the patient really needs this." Then a bureaucrat at the insurance company will decide if the doctor is right or not. Usually those bureaucrats aren't even doctors. That's illegal, but happens every day. Anything I can do to help my doctors provide better care for our patients, I'll do. I've dealt with scribes for 12 years and I genuinely think these AI scribes are a genuinely amazing use of the technology. We don't have to hire human scribes, and our doctors are freed up to deal with the patient thanks to a documentation helper. I evaluated quite a number of these tools before we rolled any out. I've been researching these for two years. Dragon with Copilot is not a good tool, for example. There was another we evaluated, I just did a search on them and their story today is wildly different than it was 18 months ago when I discovered they were lying through their teeth about the tech. I see they claim to have secured a $70m round in 2024 (which I know is a lie) and more since, so maybe they can actually do what they say now but I couldn't trust them, so I kept evaluating. I'm not an AI truster, AI isn't a panacea, but it DOES have uses, and this is one I've seen make a positive difference. I'm not an insurer, I work for providers, my goal is helping my docs provide the best care, so I promise I'm not going to roll out bullshit tech or things that would endanger our patients. My reputation is on the line, and I take that incredibly seriously too.
• eclark 6mo ago
  Be careful with initial impressions of metrics. We as humans have a heavy tenancy to anchor to our first judgments or impression. We see a win and assume the win is long term, with no downsides, and dependent on the new information/change. So combine that with the Hawthorne effect and new business or health initiatives that can look great simply because participants notice change and notice the increased attention. However many human patterns have a tendency to regress to the mean. Personally I have seen this a lot with developer tools and DevOps. A new SEV/incident/disaster happens and everyone rushes to create or onboard to a tool that would help. Around the office everyone raves about it and is sure that it would fix all issues. And the number of commits goes up, or the number of SEV's in an area decreases for a while. People were paying attention, after a while the tool starts to slow down or not be as used. It's got rough edges that weren't seen or scenarios that were supposed to be supported never get fully integrated. Eventually the patterns regress, but with more tools and more complexity. - https://pmc.ncbi.nlm.nih.gov/articles/PMC1936999/ https://pmc.ncbi.nlm.nih.gov/articles/PMC1936999/ - https://arxiv.org/abs/2102.12893 https://arxiv.org/abs/2102.12893
  • burnte 6mo ago
    > We as humans have a heavy tenancy to anchor to our first judgments or impression. One of my lifelong guiding quotes: The first principle is that you must not fool yourself, and you are the easiest person to fool. - Richard P. Feynman > We see a win and assume the win is long term, with no downsides, and dependent on the new information/change. Not me. I've had a hard life and I've worked incredibly hard to get here. I'm a little more loss-averse and focus on what can go wrong, not what went right. It's far too easy for us to become complacent. All in all I'm not your average CIO at all. I'm extremely technical, got my experience as an IT consultant for years and learned business by doing. Since moving from consultancy to employed life, I took the time to get several certifications and even did an MBA about a decade ago.
• ryandrake 6mo ago
  > improvements in patient NPS scores, provider satisfaction, and note quality How are note quality improvements measured? Vibe-notes might be more verbose and better sounding (which would explain the NPS and satisfaction metrics), but still not actually match the doctor's actual words or intent. Are the AI-generated notes actually compared with ground truth to prove they are accurate?
  • burnte 6mo ago
    > How are note quality improvements measured? Every provider is under an Assistant Clinical Director, and they report to the Clinical Directors, who report to the CMO. ACDs see fewer patients than regular providers because they have more admin time. That admin time is used to check charts. We don't review every chart, but a pretty good sampling. I meet with them monthly to talk about tech issues, and that's where I helped them create templates for notes that we can have the system output in that same format. We'll tweak the formats as needed, or the ACDs will talk with a provider about changes in how they handle the patient. Also, we look at denial reasons. Any time a claim is rejected by a payor for note related reasons it gets a full review from clinical staff other than the original provider. > Vibe-notes might be more verbose and better sounding (which would explain the NPS and satisfaction metrics), but still not actually match the doctor's actual words or intent. That's the great thing about these, they listen to the entire visit, they hear everything that happens, make a full transcript, then create a summary. It's not a situation where the doc talks for 30 seconds into a mic then the AI fleshes it out, it's the exact opposite. We're using AI to distill the visit into the note, not expand a small note into a larger one. We're not generating data, we're condensing it. Doctors must read each note, and they are legally liable for the note quality. Doctors are highly competitive and image conscious, so they're actually a great backstop for accuracy. If they notice inaccuracies in their summaries, I ASSUME you I personally hear about each and every one. I'm ok with that, though, the buck stops at my desk. > Are the AI-generated notes actually compared with ground truth to prove they are accurate? Yes. A doctor could lose their license, so every provider checks their notes, and our CMO and clinical oversight staff take that extremely seriously.
• parliament32 6mo ago
  Good, I'm glad. Now find a way to do it in-house. Shipping our conversation to some random-ass fly-by-night SaaS who pinky-swear-promises they're HIPAA-compliant is a non-starter for a medical professional I'd actually want to give money to.
  • burnte 6mo ago
    I'd love to find a way to do it in house, but we're not large enough, and our core competency is healthcare, not prompt engineering. I'd rather pay a company I've evaluated and trust until I can bring a version of the tool in house. I expect in a couple of years we'll have on-prem options and I will absolutely do that if I can. I'm an on-prem-first guy. If you have good staff then it's generally cheaper and faster for many things.
  • quantumwoke 6mo ago
    It's crazy that the hospitals justify this approach. The major scribes are literally connecting Llama to Whisper and that's it.
• Jamesbeam 6mo ago
  This ad was brought to you by the AI scribe industry, Dr.Nicks favorite tool.
  • burnte 6mo ago
    Notice how I didn't state any names? I'm not here to be a free ad. I'm just saying they're actually good tools, if you vet them right. Microsoft Dragon Copilot is not good, for example. We piloted that and had an 11% retention rate after 3 months. Trash product.
    • Jamesbeam 6mo ago
      I am not saying you are lying. But how do I make sure you’re actually a healthcare CIO for 12 years and do not have any personal investment in the AI space or specifically in this kind of business, which means better ROI if you chill these kinds of products, right? I can’t, so every time people are overly enthusiastic about something and throw numbers purely based on anecdotal evidence without any scientific backing that this is a good and safe approach at scale, I am sceptical. Don’t take it personally, it’s just my approach and experience on the Internet that most of the people throwing anecdotal experience without anything to back up their claims are 95% of the time selling you something they directly profit from. In this case, you don’t need to mention any product names. It’s enough if you make it sound fancy and believable enough to make people invest money in the space you're already invested in so the value of your investment goes up.
• invalidptr 6mo ago
  How do you control for quality variation between patients? In my experience, AI note taking tools display a clear bias against participants who are {quieter, ESL, women, ...}. How can you evaluate whether these biases show up in a medical setting?
  • burnte 6mo ago
    Check some other replies I've given. Our clinical management team does that, quite in depth.
• zaptheimpaler 6mo ago
  Yep I would agree as a patient. My current doctor types so slow that 6 out of the 10 short minutes in an appointment just disappear while he types. Even with other docs who can touch type, it will free them up to focus completely on the appointment and reduce the hours they spend charting afterwards.
• kakacik 6mo ago
  As a husband of wife who is GP, I would add a general long term issue - cognitive overload. GPs have to be almost-expert in everything, my wife is doing everything from preliminary cancer diagnosis and treatment, heart attack diagnosis to psychiatric care and many other places in between which should be normally covered by specialists. But there are simply not enough of them here for many tasks here (Switzerland). Any mistake can be potentially fatal to the patient, easily, trivially. The amount of self-imposed stress and responsibility compared to puny insignificant software dev roles like mine is staggering. And its every single day, no easy day, ever. On top of that, 3-4 hours daily just doing paperwork for insurances, legal, judges etc. that has to be flawless. LLms can help massively here, but it would be great if they are opt-in for patient (and thus he can get better focus of doctor / longer time spent / lower meeting cost), and if they could be local-only. Absolutely nobody from anywhere in Europe wants to send any data to US nor any of their closer servers, that game is closed for good.
  • kelnos 6mo ago
    Even with how overworked GPs are, what makes you think that the LLMs will have a lower error rate? Or that errors the GP makes will be more severe than the LLM's errors?
• carefulfungi 6mo ago
  Once you've had your medical records used against you by a third party, you start being much more careful about what you share with your doctors about yourself. There is no trust in a Dr's office. What they record gets handed to companies who have interests adversarial to yours. Basically like talking to the police. If you, as a patient, think an automated recording is helping you long term, you are naive.
  • burnte 6mo ago
    You encountered bad providers, not bad tools. Don't blame the hammer if you hit your thumb, and don't blame the hammer if someone ELSE hits your thumb.
    • deleted 6mo ago
      [deleted]
    • carefulfungi 6mo ago
      Here's a simple example: two people have similar depression issues. One seeks treatment - is prescribed a drug and therapy. Two years later, they are better and back to drug free. The other doesn't and has more recurrences and longer depression bouts. Which gets a better life insurance rate?
      • burnte 6mo ago
        That's not being used against you. That's a situation where they have a genuine need to know your health in order to properly estimate your share for a policy. Yes, it's imperfect, but generally people don't volunteer the truth so we have to rely on documenation. As someone with depression, yes, I understand why I pay a little more for life insurance.
• ygjb 6mo ago
  Not to be antagonistic, but a healthcare CIO in which country? This is very relevant because outside of the US, I think it is probably fair that most people who are most active on HN are from countries with public health care, and stronger consumer protection and privacy laws. The healthcare outcomes are absolutely critical in evaluating the use and value of these tools, but there are second and third order effects from using the tools that need to be contextualized with the specific motivations of executives endorsing the tools.
  • burnte 6mo ago
    > Not to be antagonistic, but a healthcare CIO in which country? USA. I should have said that. > and stronger consumer protection and privacy laws. No, they may have stricter privacy laws outside of healthcare, but HIPAA is extremely strict and heavily enforced. In 2018 our legal team asked me if we were GDPR compliant if we accepted cash pay clients from Europe. I said from the healthcare side we're already adherent, and the department you'll have problems with is marketing because HIPAA already meets or exceeds GDPR rules. Same for CCPA in California. I've been the legal Data Security and Privacy Officer in 5 healthcare orgs, I'm more scared of OIG and HHS than I am of the EU. > specific motivations of executives endorsing the tools. My job doesn't include profit motives, and I'm extremely strict. Privacy and regulatory compliance trumps profit ideas. Yes, this tool absolutely helps us not have to pay for human scribes, but we weren't going to employ them anyway. Human scribes are EXPENSIVE. Usually the alternative was a microcassette recorder, or a digital recorder that produced digital files. Then we'd have to send those files, securely, to a licensed medical transcriptionist, then ensure the recording is destroyer and the transcript comes back, and then the doctor uses that to chart. These tools mean we skip most of that, so it's faster, cheaper, and more secure. It IS good for business, but frankly, so is good patient care.
    • onecommentman 6mo ago
      You touched on a pet idea of mine and, since you made the mistake of actually intelligently responding to online forum comments, now I get to make a pitch for it. Thesis: every student accepted into medical school must complete 9 months as a medical scribe (financially compensated at some reasonable level) assigned to various medical team(s) prior to their actual entrance into med school. They are formally trained on the latest and greatest scribing tech (which clinicians probably deprioritize). They get exposure to what it means to work as part of a medical team. A heads-up before they pursue a medical career. They get exposed to operational ethics, formality of ops, etc. in a role where they probably aren’t going to kill anyone. They learn useful operational jargon and the lore of clinical practice to motivate the unending hours they will spend memorizing metabolic pathways and general trivia in med school. They provide a friendlier, more humane “UI” for clinicians who loathe automated scribing systems, but love the fact they get to actually go home at a reasonable hour instead of charting til the wee hours. They should be actually, visibly and directly making the clinician’s job easier and more pleasant, so will be more likely to be treated with respect, perhaps even be coveted, and ultimately view the experience as a life-affirming one. They make some decent money, less than a permanent professional scribe but more than flipping burgers, enough to secure decent med school student housing, maybe even pay for their books. The program fits nicely into the concept of interning already part of medical training, being a sort of “data intern” with no access to the more physically impactful elements of medical practice.
• tengbretson 6mo ago
  Were those surveys performed before or after the patient received the bill? Getting billed for a "dietary consult" because your doctor may have asked you what you had for lunch due to the coding intensity of these scribes is asinine.
  • burnte 6mo ago
    > Were those surveys performed before or after the patient received the bill? In America this doesn't matter, everyone's bills are insane. > Getting billed for a "dietary consult" because your doctor may have asked you what you had for lunch due to the coding intensity of these scribes is asinine. For what we do it's also illegal. We can only charge for services the patient consents to, and we're obligated through federal and state regulations to provide transparent pricing and estimates, so we couldn't do surprise billing if we wanted to. not that we do! We actually find it better to avoid trying to capture every single procedure code like that because it drives up rejections and thus collection costs. We'd rather bill and collect the straight procedure with no bullshit. No, the transcript will never result in a bill that is different than the service the provider rendered.
• kelnos 6mo ago
  How have you evaluated the error rate? It's unreasonable to expect that these systems will not commit any errors at all. Have any errors resulted in adverse patient outcomes? Also consider that these aren't usually just transcription services. They also interpret what the doctor and patient are saying. Presumably they also offer summaries as well. Unless the doctor immediately reviews the transcript, interpretation, and summary after each visit, and manually corrects any inconsistencies, these sorts of things will just go unnoticed, with incorrect things being a part of a person's permanent medical history. See a comment below[0] where a joke made by the patient about "doing coke" (as in coca-cola) was interpreted by the AI as "the patient used cocaine recently". That sort of error has horrifying implications. If the doctor didn't catch that, I imagine that note could have all sorts of negative consequences for the patient, including insurance rejections and possible legal action if any of this data leaks. And it's funny that you say that patients feel more comfortable and like the doctor connects with them more: after people (both patients and doctors) figure out this weakness of these systems, they will have to start self-censoring and speaking in an impersonal, neutral way in order to avoid mistakes like the above. [0] https://news.ycombinator.com/item?id=47893185 https://news.ycombinator.com/item?id=47893185
  • burnte 6mo ago
    I have, it's a metric I check in on ever month with my providers. It's a few percent, and the exact reason our official policy requires all users (including providers) to check AI output for accuracy. It's heavily enforced by our CMO. We teach our people to think of it like a scribe, and just like with a scribe you need to check it because you're legally on the hook.
    • kelnos 6mo ago
      Great, glad to hear that. I'm still concerned, though. I absolutely believe that's indeed your official policy, but people get tired, and people get overworked, and sometimes they'll succumb to the temptation to instead just give it a "quick skim" or not even really review it at all. And the more and more we rely on these systems, the more people will be lulled into a false sense of security about their accuracy. I'm not really sure what the solution is. Policy and process aren't always followed. Sure, tired providers can make mistakes themselves when manually taking notes and updating a chart, but I'm much more comfortable accepting a provider making an honest mistake, over an AI system hallucinating something, or misinterpreting a joke as something serious. One thing I can think of is to give patients direct access to these notes. Not just a printout, but actual access to the system that holds them, so that they can make their own notes to correct any issues, that the provider can incorporate, and if the provider doesn't incorporate them, then the notes remain for anyone to see in the future. But, frankly, I think it is way too early for adoption of AI systems in this sort of critical context. These systems are just not good enough. Even if they're right 99% of the time, that's still not good enough. And they absolutely are not right 99% of the time. (Also just wanted to note here that you replied before I edited my comment to add a bunch of extra stuff, just in case others see this and get the incorrect impression that you've ignored the rest of my comment.)
• m463 6mo ago
  One question I have: can they sell or share your transcripts?
  • burnte 6mo ago
    No, they're not even stored. We wouldn't allow it, we don't even allow the data to be used for training.
• CyberDildonics 6mo ago
  Saying something is good over and over doesn't mean much without saying why it accomplishes that.
• AbstractH24 6mo ago
  Is there a risk this will simply shorten appointment times and increase caseloads in the long term? Leading to doctors being more burnt out and less engaged and making these improvements transient at best? Feels like historically that’s where every other optimization that reduced a MD’s “busy work” led. Particularly adding more paraprofessionals of various kinds into the patient’s interaction.