4 ms·

Ooph, now the AI will be wrong and the human won't know it. And that's running our... let me see... life saving care. Cool. cool. cool.

decimalenough 2mo ago
"Computer says no" has been a trope of terrible customer service long before AI.
- kombookcha 2mo ago
  "Computer says no" is about unhelpful bureaucracy. Being knowingly locked to a specific procedure and unwilling to divert from it is a different thing from mistakes being made that you are blissfully unaware of. The problem is now invisible and the patient is dying for no reason anyone can discern.
  - gfrecvh 2mo ago
    The people who say "computer says no" are also unaware that's a problem. It's invisible to them. To them the process is the point. I think that two arent as different as you believe.
    - kombookcha 2mo ago
      Well, possibly. To paraphrase the great American philosopher Donald Rumsfeld - there are known knowns, known unknowns, unknown knowns and unknown unknowns. And making the actual process a black box for the people managing it will create a lot more Unknown Unknowns.
  - TeMPOraL 2mo ago
    The "unhelpful bureaucracy" arises precisely because the process is structured so there's no point in the chain that can actually consider and solve your problem directly. The process is the point, outsourcing work to the opposite end of the globe has long been a perfect way to guarantee no it's robust against errors like "compassion" and "helpfulness", and that's as true of LLMs performing those functions tomorrow as it was of people doing it yesterday.
seanmcdirmid 2mo ago
Has anything really changed for transparency?
dyauspitr 2mo ago
Your average devs are wrong more than the AI is at this point.
- eru 2mo ago
  At a former job for a regional bank in South East Asia my boss used to joke that outsourcing is really great for the company, because now they can fail to deliver their software projects for cheaper.
- watwut 2mo ago
  This is not about development. This is about rubber stamping insurance denial by an unaccountable system.
  - lotsofpulp 2mo ago
    If you sell something in the US, and a customer sues you, going to court and saying you are not accountable because the work was performed by someone in another jurisdiction is not going to go well for you. The only time that works if you have no assets or presence in the US, or I guess you have an in with the US president.
    - watwut 2mo ago
      It is about a.) avoiding criminal liabilities for frauds by the insurance company b.) facilitating desirable fraud by employees while keeping plausible deniality. It is super hard to build class cast action and suing companies for denied insurance claims is also quite hard - and super expensive. Besides, the current management dont need it to go on forever, they just need it to go on for now. If company gets to pay fine 20 years later and you already pocketed the bonuses and salary, why would you care?

decimalenough 2mo ago

"Computer says no" has been a trope of terrible customer service long before AI.

kombookcha 2mo ago
"Computer says no" is about unhelpful bureaucracy. Being knowingly locked to a specific procedure and unwilling to divert from it is a different thing from mistakes being made that you are blissfully unaware of. The problem is now invisible and the patient is dying for no reason anyone can discern.
- gfrecvh 2mo ago
  The people who say "computer says no" are also unaware that's a problem. It's invisible to them. To them the process is the point. I think that two arent as different as you believe.
  - kombookcha 2mo ago
    Well, possibly. To paraphrase the great American philosopher Donald Rumsfeld - there are known knowns, known unknowns, unknown knowns and unknown unknowns. And making the actual process a black box for the people managing it will create a lot more Unknown Unknowns.
- TeMPOraL 2mo ago
  The "unhelpful bureaucracy" arises precisely because the process is structured so there's no point in the chain that can actually consider and solve your problem directly. The process is the point, outsourcing work to the opposite end of the globe has long been a perfect way to guarantee no it's robust against errors like "compassion" and "helpfulness", and that's as true of LLMs performing those functions tomorrow as it was of people doing it yesterday.

seanmcdirmid 2mo ago

Has anything really changed for transparency?

dyauspitr 2mo ago

Your average devs are wrong more than the AI is at this point.

eru 2mo ago
At a former job for a regional bank in South East Asia my boss used to joke that outsourcing is really great for the company, because now they can fail to deliver their software projects for cheaper.
watwut 2mo ago
This is not about development. This is about rubber stamping insurance denial by an unaccountable system.
- lotsofpulp 2mo ago
  If you sell something in the US, and a customer sues you, going to court and saying you are not accountable because the work was performed by someone in another jurisdiction is not going to go well for you. The only time that works if you have no assets or presence in the US, or I guess you have an in with the US president.
  - watwut 2mo ago
    It is about a.) avoiding criminal liabilities for frauds by the insurance company b.) facilitating desirable fraud by employees while keeping plausible deniality. It is super hard to build class cast action and suing companies for denied insurance claims is also quite hard - and super expensive. Besides, the current management dont need it to go on forever, they just need it to go on for now. If company gets to pay fine 20 years later and you already pocketed the bonuses and salary, why would you care?