5 ms·

a computer can never be held accountable — therefore a computer must never make a management decision

ericb 2y ago
That's a really fascinating, but horrible, point. Every AI decision becomes a way to shirk responsibility, even more than just automated ones (because then its "your" rule). Welcome to the brave new world of AI-decision laundering.
- epgui 2y ago
  That makes it a great point, not a horrible one.
  - ulbu 2y ago
    I think they mean its reality and implications are horrible.
    - ericb 2y ago
      Yes, thank-you.
  - ericb 2y ago
    You mistook my meaning. Great thought from op. Horrible implications.
- Nasrudith 2y ago
  The same can be said of any bureaucracy's function. It isn't your fault that you made an abhorrently stupid decision, you were just following the directive. Not to say it isn't a problem, but that it isn't new.
loloquwowndueo 2y ago
Unless of course you don’t care about accountability. This gives google a lot of “oops, the system did it” leeway.
- rwmj 2y ago
  a.k.a "The Unaccountability Machine" - https://www.amazon.co.uk/Unaccountability-Machine-Systems-Terrible-Decisions-ebook/dp/B0CGFWBFD6 https://www.amazon.co.uk/Unaccountability-Machine-Systems-Te...
  - tivert 2y ago
    And the market is the ultimate unaccountability machine. Just pay attention to how people use it as a justification sometimes.
simonw 2y ago
For anyone who hasn't seen this before, it's from an IBM internal training document in 1979. Sadly the original source was lost in a flood and IBM archives do not have a copy: https://twitter.com/jonty/status/1798170111058264280 https://twitter.com/jonty/status/1798170111058264280 Edit: here's a better link https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/ https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
- alistairSH 2y ago
  Seen what, all I see is a tweet that there isn't an archive? Or is that tweet about the parent post?
  - simonw 2y ago
    I just pulled together a bunch of notes here, since linking to Twitter sucks now (logged out users can't navigate conversations): https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/ https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
    - myrmidon 2y ago
      Thanks for the source of the quote. It's very interesting: Seeing "a computer can never be held accountable" is something where the cynic in me immediately expected "we just solved accountability, boys!", and not "this must never be allowed to happen". Especially from IBM... On topic: I think our whole current Ad-economy is cancerous-- the decoupling of incentives between platforms and consumers/customers is IMO a bad trend for all of us long term (both sides), and having public attention/sentiment basically permanently on sale is yet another can of worms. I really hope we can learn to deal with this before it ruins our society. On the other hand, there are so many possibilities opening up, and generally just things improving without anyone much noticing, I'll still take unaccountable, capricious corporate overlords any day of the week in exchange. I would not want to live in any other time in the past if I had to make an honest choice. Maybe the "customer support via HN/twitter" model can save the day here at least for this (as ugly as that whole concept is).
afh1 2y ago
People who sign-off on the programs to do the task can be held accountable, they just aren't. It's not about computers, it's about the legal system.
Ukv 2y ago
While not the case here, I feel like if some system really does have a lower error rate than humans on some task, then it'd be wrong to rule it out just because it can't be held accountable or punished. To me those are primarily means to reduce mistakes, opposed to requirements or ends onto themselves.

That's a really fascinating, but horrible, point. Every AI decision becomes a way to shirk responsibility, even more than just automated ones (because then its "your" rule). Welcome to the brave new world of AI-decision laundering.

epgui 2y ago
That makes it a great point, not a horrible one.
- ulbu 2y ago
  I think they mean its reality and implications are horrible.
  - ericb 2y ago
    Yes, thank-you.
- ericb 2y ago
  You mistook my meaning. Great thought from op. Horrible implications.
Nasrudith 2y ago
The same can be said of any bureaucracy's function. It isn't your fault that you made an abhorrently stupid decision, you were just following the directive. Not to say it isn't a problem, but that it isn't new.

loloquwowndueo 2y ago

Unless of course you don’t care about accountability. This gives google a lot of “oops, the system did it” leeway.

rwmj 2y ago
a.k.a "The Unaccountability Machine" - https://www.amazon.co.uk/Unaccountability-Machine-Systems-Terrible-Decisions-ebook/dp/B0CGFWBFD6 https://www.amazon.co.uk/Unaccountability-Machine-Systems-Te...
- tivert 2y ago
  And the market is the ultimate unaccountability machine. Just pay attention to how people use it as a justification sometimes.

simonw 2y ago

For anyone who hasn't seen this before, it's from an IBM internal training document in 1979. Sadly the original source was lost in a flood and IBM archives do not have a copy: https://twitter.com/jonty/status/1798170111058264280 https://twitter.com/jonty/status/1798170111058264280 Edit: here's a better link https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/ https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...

alistairSH 2y ago
Seen what, all I see is a tweet that there isn't an archive? Or is that tweet about the parent post?
- simonw 2y ago
  I just pulled together a bunch of notes here, since linking to Twitter sucks now (logged out users can't navigate conversations): https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/ https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
  - myrmidon 2y ago
    Thanks for the source of the quote. It's very interesting: Seeing "a computer can never be held accountable" is something where the cynic in me immediately expected "we just solved accountability, boys!", and not "this must never be allowed to happen". Especially from IBM... On topic: I think our whole current Ad-economy is cancerous-- the decoupling of incentives between platforms and consumers/customers is IMO a bad trend for all of us long term (both sides), and having public attention/sentiment basically permanently on sale is yet another can of worms. I really hope we can learn to deal with this before it ruins our society. On the other hand, there are so many possibilities opening up, and generally just things improving without anyone much noticing, I'll still take unaccountable, capricious corporate overlords any day of the week in exchange. I would not want to live in any other time in the past if I had to make an honest choice. Maybe the "customer support via HN/twitter" model can save the day here at least for this (as ugly as that whole concept is).

afh1 2y ago

People who sign-off on the programs to do the task can be held accountable, they just aren't. It's not about computers, it's about the legal system.

Ukv 2y ago

While not the case here, I feel like if some system really does have a lower error rate than humans on some task, then it'd be wrong to rule it out just because it can't be held accountable or punished. To me those are primarily means to reduce mistakes, opposed to requirements or ends onto themselves.