5 ms·

The article is a lot better than the headline lets on. In particular: > Crucially, she said, we should place blame on the human individuals behind the creation

by Tarq0n 6y ago

The article is a lot better than the headline lets on. In particular: > Crucially, she said, we should place blame on the human individuals behind the creation and use of the algorithm rather than reify the technology as being the main driver. > “Systems and algorithms are human-made, and do exactly what they’ve been instructed to do,” she said. “They can act as an easy and sort of cowardly way to take the blame off yourself. What’s important to understand is that often algorithms can use historical data without any proper interpretation of the context surrounding that data. With that pattern, you can only expect social problems like institutional racism to increase. The result is a sort of feedback loop that increases social problems that already exist.” As someone in an adjacent institution this affair has been frustrating to watch. In my opinion black-box algorithms have no place in government, and for this reason it's critical for the government to develop the expertise to develop and understand models themselves, so they aren't reliant on the trade secrets of contractors. Interestingly though the administration resigned over this affair, it does not look like they'll suffer much over it in the upcoming election this wednesday. The responsible prime minister is likely to get a fifth term if things go as the polls are predicting.

tchalla 6y ago
> In my opinion black-box algorithms have no place in government, and for this reason it's critical for the government to develop the expertise to develop and understand models themselves, so they aren't reliant on the trade secrets of contractors. We learned from 2000 years of human history - war, disease etc. - to create a transparent system of checks and balances. These days we are trying to pretty much remove all of it (digitally) for "convenience" and "efficiency". I can not understand how any semi-decent societal system can accept a situation of opaqueness and no accountability.
- Rule35 6y ago
  > These days we are trying to pretty much remove all of it (digitally) for "convenience" and "efficiency". Well, the same 'we' that push voting machines and other crap like locking government records up in proprietary software. I push for algorithms specifically because they're (as a written document) transparent. If a human has to judge a historical context you're never going to get repeatable results. But I want the system to be able to diagram, with weights, all the inputs that went into something before I'll trust an automated version.
- reaperducer 6y ago
  These days we are trying to pretty much remove all of it (digitally) for "convenience" and "efficiency". And "cost." As in, "I'd rather get crap news online for free than pay for a newspaper or broadcaster to keep the government honest."
- crispyambulance 6y ago
  > I can not understand how any semi-decent societal system can accept a situation of opaqueness and no accountability. Part of it comes down to an effect that's been known for a long time with human-computer interaction. Whenever you have a machine in the decision loop of a job, many workers WILL DEFER to the machine whenever there's a question of whether something is wrong or right. By deferring to the machine, the workers absolve themselves of accountability and shift all responsibility to "the system". I think this is often deliberately "baked in" by management to customer service of all kinds. You can see this in a small way if you've ever had to call an internet service provider in the USA to deal with a billing mistake.
- 6gvONxR4sf7o 6y ago
  Or even 5000 years if you start the human governments clock with sumer/egypt/indus valley/yellow river[0], but it doesn't detract from the point. [0] https://en.wikipedia.org/wiki/Government#History https://en.wikipedia.org/wiki/Government#History
- reinkaos 6y ago
  I don't think "black-box" algorithms are the problem per se. In this case the system was doomed from the get-go by using biased and flawed input attributes. What I find very problematic in the use of automated systems for fraud detection is taking the system decision as final and not offering an easy and clear way of appealing decisions.
im3w1l 6y ago
> you can only expect social problems like institutional racism to increase. The result is a sort of feedback loop I dislike how people just carelessly make these claims without justifying them. For instance, another (far more likely imo) possibility is that it converges into certain state.

by Tarq0n 6y ago

tchalla 6y ago

> In my opinion black-box algorithms have no place in government, and for this reason it's critical for the government to develop the expertise to develop and understand models themselves, so they aren't reliant on the trade secrets of contractors. We learned from 2000 years of human history - war, disease etc. - to create a transparent system of checks and balances. These days we are trying to pretty much remove all of it (digitally) for "convenience" and "efficiency". I can not understand how any semi-decent societal system can accept a situation of opaqueness and no accountability.

Rule35 6y ago
> These days we are trying to pretty much remove all of it (digitally) for "convenience" and "efficiency". Well, the same 'we' that push voting machines and other crap like locking government records up in proprietary software. I push for algorithms specifically because they're (as a written document) transparent. If a human has to judge a historical context you're never going to get repeatable results. But I want the system to be able to diagram, with weights, all the inputs that went into something before I'll trust an automated version.
reaperducer 6y ago
These days we are trying to pretty much remove all of it (digitally) for "convenience" and "efficiency". And "cost." As in, "I'd rather get crap news online for free than pay for a newspaper or broadcaster to keep the government honest."
crispyambulance 6y ago
> I can not understand how any semi-decent societal system can accept a situation of opaqueness and no accountability. Part of it comes down to an effect that's been known for a long time with human-computer interaction. Whenever you have a machine in the decision loop of a job, many workers WILL DEFER to the machine whenever there's a question of whether something is wrong or right. By deferring to the machine, the workers absolve themselves of accountability and shift all responsibility to "the system". I think this is often deliberately "baked in" by management to customer service of all kinds. You can see this in a small way if you've ever had to call an internet service provider in the USA to deal with a billing mistake.
6gvONxR4sf7o 6y ago
Or even 5000 years if you start the human governments clock with sumer/egypt/indus valley/yellow river[0], but it doesn't detract from the point. [0] https://en.wikipedia.org/wiki/Government#History https://en.wikipedia.org/wiki/Government#History
reinkaos 6y ago
I don't think "black-box" algorithms are the problem per se. In this case the system was doomed from the get-go by using biased and flawed input attributes. What I find very problematic in the use of automated systems for fraud detection is taking the system decision as final and not offering an easy and clear way of appealing decisions.

im3w1l 6y ago

> you can only expect social problems like institutional racism to increase. The result is a sort of feedback loop I dislike how people just carelessly make these claims without justifying them. For instance, another (far more likely imo) possibility is that it converges into certain state.