4 ms·

Not all businesses are likely to jump on this bandwagon. For example, take health insurance claim denials (in the US). Among issuers who reported the highest

by vivegi 4y ago

Not all businesses are likely to jump on this bandwagon. For example, take health insurance claim denials (in the US). Among issuers who reported the highest volume of in-network claims in 2021, receiving over 5 million claims, denial rates ranged from 5.7%. to 41.9% (Source: https://www.kff.org/private-insurance/issue-brief/claims-denials-and-appeals-in-aca-marketplace-plans/ https://www.kff.org/private-insurance/issue-brief/claims-den...) In every one of these cases, the insured receives an "Explanation of Benefit" that lists the claims and a "Denial Reason" along with the "Paid, Denied" amounts. On paper, the LLM approach you outlined above looks like it would fit this claims processing workflow. But, it would be another 50 years, if that, before the health insurance industry takes that approach. Healthcare claims processing is already messed up. Good luck pitching an LLM based system for this. ps: The simple reason for the resistance/inertia is not technical, but regulatory/legal risk. If the insured sues the insurance company, the company can bring their engineers as witnesses to explain how the denial logic has been coded. There is no way (at least currently) to explain why an LLM model took a certain decision. As long as there is the risk of the LLM hallucinating on the witness stand when prompted by the suing party's lawyers, there is no way the business/risk teams would sign off on that.

sjy 4y ago
> If the insured sues the insurance company, the company can bring their engineers as witnesses to explain how the denial logic has been coded. Sometimes I wonder how useful this option is in practice. The British Post Office scandal [1] is a rare example where engineers actually were called as witnesses to explain the logic of a complex enterprise software system with legally significant consequences. The cost of just getting to the 313-page judgment addressing the technical issues [2] was astronomical. Some of the hundreds of wrongfully convicted small business people had gone to trial and tried, but failed, to discredit the prosecution's expert evidence that the software was reliable. If the decisions had come out of an inscrutable LLM, it might have saved a lot of trouble. [1]: https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal [2]: https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-post-office-judgment.pdf https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-...
- vivegi 4y ago
  That may be true with the human providing the explanation. However, there is at least some hope of being able to reason with the explanation (about the adequacy or lack thereof). With a hallucinating LLM, most bets are off. However, if the technology progresses to a level where the LLM output works like a proof of a theorem, verifiable step-by-step, there is a chance we could trust those systems. We are not there yet, I think.

by vivegi 4y ago

sjy 4y ago

> If the insured sues the insurance company, the company can bring their engineers as witnesses to explain how the denial logic has been coded. Sometimes I wonder how useful this option is in practice. The British Post Office scandal [1] is a rare example where engineers actually were called as witnesses to explain the logic of a complex enterprise software system with legally significant consequences. The cost of just getting to the 313-page judgment addressing the technical issues [2] was astronomical. Some of the hundreds of wrongfully convicted small business people had gone to trial and tried, but failed, to discredit the prosecution's expert evidence that the software was reliable. If the decisions had come out of an inscrutable LLM, it might have saved a lot of trouble. [1]: https://en.wikipedia.org/wiki/British_Post_Office_scandal https://en.wikipedia.org/wiki/British_Post_Office_scandal [2]: https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-post-office-judgment.pdf https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-...

vivegi 4y ago
That may be true with the human providing the explanation. However, there is at least some hope of being able to reason with the explanation (about the adequacy or lack thereof). With a hallucinating LLM, most bets are off. However, if the technology progresses to a level where the LLM output works like a proof of a theorem, verifiable step-by-step, there is a chance we could trust those systems. We are not there yet, I think.