5 ms·

AI is kind of the ultimate expression of "Deferred responsibility". Kind of like "I was protecting shareholder interests" or "I was just following orders".

johnnyanmac 2y ago
"A computer can never be held accountable, therefore a computer must never make a management decision". How did we stray so far?
- moffkalast 2y ago
  What would the definition of accountability there be though? I can't think of anything that one couldn't apply to both. If a person does something mildly wrong, we can explain it to them and they can avoid making that mistake in the future. If a person commits murder, we lock them away forever for the safety of society. If a program produces an error, we can "explain" to the code editor what's wrong and fix the problem. If a program kills someone, we can delete it. Ultimately a Nuremberg defense doesn't really get you off the hook anyway, and you have a moral obligation to object to orders that you perceive as wrong, so there's no difference if the orders come from man or machine - you are liable either way.
  - johnnyanmac 2y ago
    Well the reality if/when a death by AI occurs is that lawsuits will hit everyone. So the doctor working on the patient, the hospital and its owners, and the LLM tech company will all try to be hit. The precedent from that will legally settle that issue. Morally, it's completely reckless to use 2024 LLMS in any mission/safety critical factor, and to be honest LLMS should redirect all medical and legal inquiries to a doctor/lawyer. Maybe in 2044 that can change, but in 2024 companies are explicitly marketing to try and claim these are ready for those areas. >If a program produces an error, we can "explain" to the code editor what's wrong and fix the problem. Yes. And that's the crux of the issue. LLMS aren't marketed to supplement professionals who become more productive. In marketed to replace labor. To say "you don't need a doctor for everything, ask GPT". Even to the hospitals themselves. If you're not a professional, these are black boxes, and the onus falls solely on the box maker in that case. Now if we were talking about medical experts leveraging computing to help come to a decision, and not blindly just listening to a simple yes/no, we'd come to a properly nuanced issue worth discussing. But medicine shouldn't be a black box catch all.
    - moffkalast 2y ago
      Yeah that will probably happen, but I feel like it has no basis to really make any sense. I mean this is just the latest shiniest way of getting knowledge, and people don't really sue Google when they get wrong results. If you read something in a book as a doctor and a patient dies because it was wrong, it's also not the book that's really getting the blame. It's gonna be you for not doing the due diligence of double checking. There's zero precedence for it. The marketing department could catch a lawsuit or two for false advertising though and they'd probably deserve it.
      - johnnyanmac 2y ago
        Firstly, Google was indeed sued many times early on over search results. Those were settled when Google simply claimed to be a middleman between queries and results. I'm not sure of modern lawsuits, but you can argue they started to become more and more of a curator as the algorithms shifted and they accepted more SEO optimization of what we now call slop. Gemini itself is first and foremost for many results, so we've more or less come full circle on that idea. I agree there's no precedence for this next iteration, and that's what the inevitable lawsuits will determine over the years, decades. I think the main difference is that these tech companies are taking more "ownership" of the results with their black boxes. And it seems like the last thing they'd do is open that box.
- shreezus 2y ago
  >"A computer can never be held accountable, therefore a computer must never make a management decision". Ultimately it's algorithmic diffusion of responsibility that leads to unintended consequences.
Cthulhu_ 2y ago
I don't think it's the ulimate expression per se, just the next step. Software, any kind of predictive model, has been used to make decisions for a long time now, some for good, some for bad.
Treesrule14 2y ago
https://www.bloomberg.com/news/articles/2024-07-01/dan-davies-explains-why-accountability-sinks-are-everywhere-now https://www.bloomberg.com/news/articles/2024-07-01/dan-davie... Dan davies did a great interview on odd lots about this he called it accountability sinks
- hinkley 2y ago
  I think about a third of the reason I get lead positions is because I'm willing to be an 'accountability sink', or the much more colorful description: a sin-eater. You just gotta be careful about what decisions you're willing to own. There's a long list of decisions I won't be held responsible for and that sometimes creates... problems. Some of that is on me, but a lot is being taken for granted. I'm not a scapegoat I'm a facilitator, and being able to say, "I believe in this idea enough that if it blows up you can tell people to come yell at me instead of at you." unblocks a lot of design and triage meetings.
photochemsyn 2y ago
It all depends on how you use it. Tell the AI to generate text in support of option A, that's what you mostly get (unless you hit the built-in 'safety' mechanisms). Do the same for options B, C, etc and then ask the AI to compare and contrast each viewpoint (get the AI to argue with itself). This is time-consuming but a failure to converge on a single answer using this approach does at least indicate that more research is needed. Now, if the overall population has been indoctrinated with 'trust the authority' thinking since childhood, then a study like this one might be used to assess the prevalence of critical thinking skills in the population under study. Whether or not various interests have been working overtime for some decades now to create a population that's highly susceptible to corporate advertising and government propaganda is also an interesting question, though I doubt much federal funding would be made available to researchers for investigating it.
hinkley 2y ago
I wonder how much of the bureaucratic mess of medicine is caused by this. Oh your insurance doesn't cover this or let me prescribe this to you off-label. Sorry!

johnnyanmac 2y ago

"A computer can never be held accountable, therefore a computer must never make a management decision". How did we stray so far?

moffkalast 2y ago
What would the definition of accountability there be though? I can't think of anything that one couldn't apply to both. If a person does something mildly wrong, we can explain it to them and they can avoid making that mistake in the future. If a person commits murder, we lock them away forever for the safety of society. If a program produces an error, we can "explain" to the code editor what's wrong and fix the problem. If a program kills someone, we can delete it. Ultimately a Nuremberg defense doesn't really get you off the hook anyway, and you have a moral obligation to object to orders that you perceive as wrong, so there's no difference if the orders come from man or machine - you are liable either way.
- johnnyanmac 2y ago
  Well the reality if/when a death by AI occurs is that lawsuits will hit everyone. So the doctor working on the patient, the hospital and its owners, and the LLM tech company will all try to be hit. The precedent from that will legally settle that issue. Morally, it's completely reckless to use 2024 LLMS in any mission/safety critical factor, and to be honest LLMS should redirect all medical and legal inquiries to a doctor/lawyer. Maybe in 2044 that can change, but in 2024 companies are explicitly marketing to try and claim these are ready for those areas. >If a program produces an error, we can "explain" to the code editor what's wrong and fix the problem. Yes. And that's the crux of the issue. LLMS aren't marketed to supplement professionals who become more productive. In marketed to replace labor. To say "you don't need a doctor for everything, ask GPT". Even to the hospitals themselves. If you're not a professional, these are black boxes, and the onus falls solely on the box maker in that case. Now if we were talking about medical experts leveraging computing to help come to a decision, and not blindly just listening to a simple yes/no, we'd come to a properly nuanced issue worth discussing. But medicine shouldn't be a black box catch all.
  - moffkalast 2y ago
    Yeah that will probably happen, but I feel like it has no basis to really make any sense. I mean this is just the latest shiniest way of getting knowledge, and people don't really sue Google when they get wrong results. If you read something in a book as a doctor and a patient dies because it was wrong, it's also not the book that's really getting the blame. It's gonna be you for not doing the due diligence of double checking. There's zero precedence for it. The marketing department could catch a lawsuit or two for false advertising though and they'd probably deserve it.
    - johnnyanmac 2y ago
      Firstly, Google was indeed sued many times early on over search results. Those were settled when Google simply claimed to be a middleman between queries and results. I'm not sure of modern lawsuits, but you can argue they started to become more and more of a curator as the algorithms shifted and they accepted more SEO optimization of what we now call slop. Gemini itself is first and foremost for many results, so we've more or less come full circle on that idea. I agree there's no precedence for this next iteration, and that's what the inevitable lawsuits will determine over the years, decades. I think the main difference is that these tech companies are taking more "ownership" of the results with their black boxes. And it seems like the last thing they'd do is open that box.
shreezus 2y ago
>"A computer can never be held accountable, therefore a computer must never make a management decision". Ultimately it's algorithmic diffusion of responsibility that leads to unintended consequences.

Cthulhu_ 2y ago

I don't think it's the ulimate expression per se, just the next step. Software, any kind of predictive model, has been used to make decisions for a long time now, some for good, some for bad.

Treesrule14 2y ago

https://www.bloomberg.com/news/articles/2024-07-01/dan-davies-explains-why-accountability-sinks-are-everywhere-now https://www.bloomberg.com/news/articles/2024-07-01/dan-davie... Dan davies did a great interview on odd lots about this he called it accountability sinks

hinkley 2y ago
I think about a third of the reason I get lead positions is because I'm willing to be an 'accountability sink', or the much more colorful description: a sin-eater. You just gotta be careful about what decisions you're willing to own. There's a long list of decisions I won't be held responsible for and that sometimes creates... problems. Some of that is on me, but a lot is being taken for granted. I'm not a scapegoat I'm a facilitator, and being able to say, "I believe in this idea enough that if it blows up you can tell people to come yell at me instead of at you." unblocks a lot of design and triage meetings.

photochemsyn 2y ago

It all depends on how you use it. Tell the AI to generate text in support of option A, that's what you mostly get (unless you hit the built-in 'safety' mechanisms). Do the same for options B, C, etc and then ask the AI to compare and contrast each viewpoint (get the AI to argue with itself). This is time-consuming but a failure to converge on a single answer using this approach does at least indicate that more research is needed. Now, if the overall population has been indoctrinated with 'trust the authority' thinking since childhood, then a study like this one might be used to assess the prevalence of critical thinking skills in the population under study. Whether or not various interests have been working overtime for some decades now to create a population that's highly susceptible to corporate advertising and government propaganda is also an interesting question, though I doubt much federal funding would be made available to researchers for investigating it.

hinkley 2y ago

I wonder how much of the bureaucratic mess of medicine is caused by this. Oh your insurance doesn't cover this or let me prescribe this to you off-label. Sorry!