5 ms·

On the other hand, with algorithms you can actually study how the decisions are made. You can investigate what was given as input and how it affected the result

On the other hand, with algorithms you can actually study how the decisions are made. You can investigate what was given as input and how it affected the results. With complicated algorithms it may not be obvious, but then you can try to poke the black box with thousands of sample cases to see how it reacts. Compare this with humans, who often don’t even know themselves what their decisions are really based on.

giantg2 5y ago
"On the other hand, with algorithms you can actually study how the decisions are made. You can investigate what was given as input and how it affected the results." You'd have to have access to it. This is highly unlikely. If testing as a black box, could they identify a test and change its behavior, like VW emissions tests?
the_snooze 5y ago
Machines don't have shame, reputation, values, empathy. There's little hope of self-correction. An algorithm will never ask "Are we the baddies?"
- hotpotamus 5y ago
  I haven't met many humans who would ever ask the baddies question either - I don't think it ever occurs to people to ask, which is why it makes for a decent comedy skit. Shame and empathy are built in because we really do work better in at least small groups of up to perhaps a few hundred people.
dehrmann 5y ago
Yeah, it's easier to investigate a racist algorithm than a racist manager. The article makes the following point, but I'm not sure if it's necessarily worse than the status quo: > All the while, the potential for discrimination based on race, gender, or disability, especially in hiring and firing software, remains obscured behind the black box of AI.
advisedwang 5y ago
Who is the "you" here? Workers certainly do not get to probe the internals of the algorithm that fired them. For liability reasons I'm sure human resource management systems are not exposed to researchers. Maybe a court could but in practice I don't think that's happening and the bar to get that kind of review is incredibly high. Only the company itself is actually in practice going to be testing their system but they only do so with a view to their own interests.
- jpalomaki 5y ago
  That should be defined in the legislation. Maybe something where the employee can for example file a complaint of unfair practises and then the issue is investigated. This is pretty much the same case when humans are accused of unfair or discriminatory decions. I think the GDPR legislation already takes small steps to this direction.
  - contravariant 5y ago
    This is nothing to do with GDPR, this is about workers' right against unfair dismissal. Being dismissed without warning, because of rules you cannot know, without explanation or right to appeal is ridiculous.
    - PeterisP 5y ago
      It has a lot to do with GDPR, because GDPR article 22 explicitly prohibits all kinds of severe decisions (including but not limited to dismissal) from being done by an automated unappealable system. Like, even if the dismissal was after a warning, because of rules well known in advance and with an accurate explanation, GDPR would still prohibit making this decision solely through automated processing, there has to be a place for human intervention.
      - contravariant 5y ago
        Trying to protect basic workers' rights through legislation that protects personal information is downright weird. You'd think workers should be better protected than just random people who share their personal information. Article 22 even acknowledges this to some extent, making an explicit exception for people who have some other kind of contract (including employment) and for which the law has defined suitable safeguards. U.S. employment legislation is indeed not a sufficient safeguard, but the fact that it insufficiently protects privacy is somewhat of an understatement.
telxosser 5y ago
I am sure people's thoughts on this reflect their current boss/leadership. I would have disagreed 2 years ago but got a new boss last year that seems to make huge business decisions based on a random number generator.

giantg2 5y ago

"On the other hand, with algorithms you can actually study how the decisions are made. You can investigate what was given as input and how it affected the results." You'd have to have access to it. This is highly unlikely. If testing as a black box, could they identify a test and change its behavior, like VW emissions tests?

the_snooze 5y ago

Machines don't have shame, reputation, values, empathy. There's little hope of self-correction. An algorithm will never ask "Are we the baddies?"

hotpotamus 5y ago
I haven't met many humans who would ever ask the baddies question either - I don't think it ever occurs to people to ask, which is why it makes for a decent comedy skit. Shame and empathy are built in because we really do work better in at least small groups of up to perhaps a few hundred people.

dehrmann 5y ago

Yeah, it's easier to investigate a racist algorithm than a racist manager. The article makes the following point, but I'm not sure if it's necessarily worse than the status quo: > All the while, the potential for discrimination based on race, gender, or disability, especially in hiring and firing software, remains obscured behind the black box of AI.

advisedwang 5y ago

Who is the "you" here? Workers certainly do not get to probe the internals of the algorithm that fired them. For liability reasons I'm sure human resource management systems are not exposed to researchers. Maybe a court could but in practice I don't think that's happening and the bar to get that kind of review is incredibly high. Only the company itself is actually in practice going to be testing their system but they only do so with a view to their own interests.

jpalomaki 5y ago
That should be defined in the legislation. Maybe something where the employee can for example file a complaint of unfair practises and then the issue is investigated. This is pretty much the same case when humans are accused of unfair or discriminatory decions. I think the GDPR legislation already takes small steps to this direction.
- contravariant 5y ago
  This is nothing to do with GDPR, this is about workers' right against unfair dismissal. Being dismissed without warning, because of rules you cannot know, without explanation or right to appeal is ridiculous.
  - PeterisP 5y ago
    It has a lot to do with GDPR, because GDPR article 22 explicitly prohibits all kinds of severe decisions (including but not limited to dismissal) from being done by an automated unappealable system. Like, even if the dismissal was after a warning, because of rules well known in advance and with an accurate explanation, GDPR would still prohibit making this decision solely through automated processing, there has to be a place for human intervention.
    - contravariant 5y ago
      Trying to protect basic workers' rights through legislation that protects personal information is downright weird. You'd think workers should be better protected than just random people who share their personal information. Article 22 even acknowledges this to some extent, making an explicit exception for people who have some other kind of contract (including employment) and for which the law has defined suitable safeguards. U.S. employment legislation is indeed not a sufficient safeguard, but the fact that it insufficiently protects privacy is somewhat of an understatement.

telxosser 5y ago

I am sure people's thoughts on this reflect their current boss/leadership. I would have disagreed 2 years ago but got a new boss last year that seems to make huge business decisions based on a random number generator.