I would be curious as to which (kind of) model they've used there, how the tooling and prompts and all look like, etc. I bet that to make the business case via

I would be curious as to which (kind of) model they've used there, how the tooling and prompts and all look like, etc. I bet that to make the business case viable (or rather profitable), it's probably something small and cheap. Bigger and better models don't come with any guarantees as to correctness either, but they do push down the probability of something as wrong as this happening by orders of magnitude. That and processes. Even a smol dumb model can throw a report at both parties in the end where both need to sign off on it. Which should also scale better if both do, because the patient doesn't get fatigued because it's not happening many times per day. Or you maybe mirror what humans do and ask for active confirmation the less plausible something sounds. Hmm. __ Point being that I wouldn't necessarily blame it on the tech itself, but rather the (probably) startup, the culture and the fact that no one is going to jail here. Your honor, the floats are innocent. They were simply forced to do this by the evil startup founder.