3 ms·

We need more of this. Companies should not be able excuse their actions because of 'software mistake' while the tool works exactly as designed until someone po

We need more of this. Companies should not be able excuse their actions because of 'software mistake' while the tool works exactly as designed until someone points out a shortcoming or bias.

mabub24 5y ago
There is very annoying double-speak that goes on when a company's algo comes under fire. They often present the algos as if they are real AI, something set loose and allowed to run free, like a wild animal, rather than something designed to match business purposes. Algos aren't mysterious black boxes, but marketing them as such sure makes the people that make and sell them look like "wizards" to oblivious execs/c-suite people. It's about removing and hiding accountability while they fiddle under the hood behind a curtain like the Wizard of Oz.
- jjk166 5y ago
  At the end of the day, algorithms are industrial equipment, like a bulldozer. If a bulldozer runs over a bunch of people, saying "the operator doesn't know how to correctly operate a bulldozer" doesn't get you off the hook, if anything it makes it worse.
  - mabub24 5y ago
    I agree. When people act with disbelief at the results of an algo they've made, they're either sincere, in which case they don't actually understand the thing they have built (which would suggest a lack of competence, which many are terrified to admit); or, they're lying, and attempting to absolve themselves of any responsibility for those outcomes. As soon it comes in contact with a human, you don't get to wipe your hands of the outcome if you dislike it. You have to own the successes and the failures.
  - Spivak 5y ago
    It is really bonkers to me that we don't design systems to be introspectable and replay-able. If you have a system relies on sampling from a random distribution to make decisions then you really ought to make the whole system deterministic and provide it with a single logged entropy source so when something goes south you can replay exactly what happened.

We need more of this. Companies should not be able excuse their actions because of 'software mistake' while the tool works exactly as designed until someone points out a shortcoming or bias.

mabub24 5y ago

There is very annoying double-speak that goes on when a company's algo comes under fire. They often present the algos as if they are real AI, something set loose and allowed to run free, like a wild animal, rather than something designed to match business purposes. Algos aren't mysterious black boxes, but marketing them as such sure makes the people that make and sell them look like "wizards" to oblivious execs/c-suite people. It's about removing and hiding accountability while they fiddle under the hood behind a curtain like the Wizard of Oz.

jjk166 5y ago
At the end of the day, algorithms are industrial equipment, like a bulldozer. If a bulldozer runs over a bunch of people, saying "the operator doesn't know how to correctly operate a bulldozer" doesn't get you off the hook, if anything it makes it worse.
- mabub24 5y ago
  I agree. When people act with disbelief at the results of an algo they've made, they're either sincere, in which case they don't actually understand the thing they have built (which would suggest a lack of competence, which many are terrified to admit); or, they're lying, and attempting to absolve themselves of any responsibility for those outcomes. As soon it comes in contact with a human, you don't get to wipe your hands of the outcome if you dislike it. You have to own the successes and the failures.
- Spivak 5y ago
  It is really bonkers to me that we don't design systems to be introspectable and replay-able. If you have a system relies on sampling from a random distribution to make decisions then you really ought to make the whole system deterministic and provide it with a single logged entropy source so when something goes south you can replay exactly what happened.