4 ms·

That's the dystopian future I've been scared of. LLMs (Large Language Models i.e. generators of most probable text) will be used to DECIDE if a person gets the

That's the dystopian future I've been scared of. LLMs (Large Language Models i.e. generators of most probable text) will be used to DECIDE if a person gets the money back or not. People in command (managers) showing the lack of understanding of what an LLM is and misusing it in the worst way (another is to write to people why their insurance claims have been rejected)

intelVISA 2y ago
They don't lack understanding: un unaccountable magic box that can be 'accidentally' tuned to favor current business goals is a manager's dream.
- DwnVoteHoneyPot 2y ago
  or else pay another $15 (another manager's dream).
- p0w3n3d 2y ago
  > A computer can never be held accountable. > Therefore a computer must never make a management decision https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/ https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
FireBeyond 2y ago
> another is to write to people why their insurance claims have been rejected Health insurers are already using it to "assist" their claims reviewers decide to reject a claim. One system I saw basically summarizes a claim, shows why it should be rejected, and asks the user to decide if these should be overridden.
- jalapenos 2y ago
  Probably more like: the claim is rejected by default, then the AI is asked to come up with reasoning why, then review decide if that reasoning is just too silly to fly.
- p0w3n3d 2y ago
  I recently have put my CV into ChatGPT and tell it to assess my experience according to the position in the job offer, and give the answer in percentage. It said 80%. With small adjustments I could make it 100%. But if they use another tool (another LLM or whatever), I might fail. Just these are those s/interesting/hard/g times everybody was speaking of.
- p0w3n3d 2y ago
  rejecting people's claims when a doctor says it is required is a great example of a conflict of interest. The company is a judge in their own case against the patient. And there is an impartial doctor whose word is just ignored in this "court" case

intelVISA 2y ago

They don't lack understanding: un unaccountable magic box that can be 'accidentally' tuned to favor current business goals is a manager's dream.

DwnVoteHoneyPot 2y ago
or else pay another $15 (another manager's dream).
p0w3n3d 2y ago
> A computer can never be held accountable. > Therefore a computer must never make a management decision https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/ https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...

FireBeyond 2y ago

> another is to write to people why their insurance claims have been rejected Health insurers are already using it to "assist" their claims reviewers decide to reject a claim. One system I saw basically summarizes a claim, shows why it should be rejected, and asks the user to decide if these should be overridden.

jalapenos 2y ago
Probably more like: the claim is rejected by default, then the AI is asked to come up with reasoning why, then review decide if that reasoning is just too silly to fly.
p0w3n3d 2y ago
I recently have put my CV into ChatGPT and tell it to assess my experience according to the position in the job offer, and give the answer in percentage. It said 80%. With small adjustments I could make it 100%. But if they use another tool (another LLM or whatever), I might fail. Just these are those s/interesting/hard/g times everybody was speaking of.
p0w3n3d 2y ago
rejecting people's claims when a doctor says it is required is a great example of a conflict of interest. The company is a judge in their own case against the patient. And there is an impartial doctor whose word is just ignored in this "court" case