5 ms·

I don't really see that it is that bad compared to what happens now. You have people doing it currently with all their subjective biases (such as race and class

I don't really see that it is that bad compared to what happens now. You have people doing it currently with all their subjective biases (such as race and class). Maybe this just makes it cheaper and maybe more effective as the tech matures. Surely accuracy concerns can be addressed by combining multiple data sources (e.g. looks like someone dodgy, is somewhere strange, doing something strange, just the way a police officer would rationalize investigating further).

jdc 8y ago
Because we don't want people who can track the entire population at once. It's too much power.
dreamfactored 8y ago
The fundamental principle of presumption of innocence means you shouldn't go around spying on everyone looking for signs of criminality. And the idea that the tech is more dispassionate and therefore fairer than humans is a step on the road to hell. I guess you never had an anomalous credit or insurance rating yet. (Notably GDPR has a clause for precisely the ability to review automated decision making where it can have a material effect.)
zeth___ 8y ago
And you think the algorithms made by racists aren't racist? This isn't a sorting algorithm where there is a simple provable solution, this is putting heuristics in a computer.
- godelski 8y ago
  > And you think the algorithms made by racists aren't racist? I'd hedge that a little. Because I think there are a lot of people who are just naive and think that using data that is historically racist will result in racist outcomes. Not saying that the person creating the algo is racist, but for such a harsh accusation you need enough substantial evidence. That, or we dilute the term.
  - serioussecurity 8y ago
    Hm? At this point there's really nothing controversial about "racist data makes racist models" (I'm being flippant in my phrasing, but, more rigorously: "the model will reflect and reify the biases of the person training the model and the data used to train it, and the people who annotate the training data") I like this: https://arxiv.org/abs/1706.09847 https://arxiv.org/abs/1706.09847
    - salty_biscuits 8y ago
      But the system in question is scanning a crowd looking for known faces in a database, race only enters it when you are comparing it to the human system of a police officer looking for terrorists at a football game. I get that people object to persistent surveillance but that doesn't seem to be what this system is about. It is looking for certain people at certain events. The way your face gets on the list in the first place is where you should put your concerns...
      - godelski 8y ago
        Well what I was saying, and why I suggested hedging the accusation, is that if someone does, say an AI that determines court sentences then that might cause the AI to be racist even though race was never set as a parameter. Such an AI might have racist behavior because of the naivety of designer[1], not that the designer was themselves racist. Obviously the designer could make this data set racist on purpose, I was just saying that an AI that judges on race might be due to stupidity and not malice. Hanlon's Razor. [1]Not realizing that there are other parameters that correlate with race

jdc 8y ago

Because we don't want people who can track the entire population at once. It's too much power.

dreamfactored 8y ago

The fundamental principle of presumption of innocence means you shouldn't go around spying on everyone looking for signs of criminality. And the idea that the tech is more dispassionate and therefore fairer than humans is a step on the road to hell. I guess you never had an anomalous credit or insurance rating yet. (Notably GDPR has a clause for precisely the ability to review automated decision making where it can have a material effect.)

zeth___ 8y ago

And you think the algorithms made by racists aren't racist? This isn't a sorting algorithm where there is a simple provable solution, this is putting heuristics in a computer.

godelski 8y ago
> And you think the algorithms made by racists aren't racist? I'd hedge that a little. Because I think there are a lot of people who are just naive and think that using data that is historically racist will result in racist outcomes. Not saying that the person creating the algo is racist, but for such a harsh accusation you need enough substantial evidence. That, or we dilute the term.
- serioussecurity 8y ago
  Hm? At this point there's really nothing controversial about "racist data makes racist models" (I'm being flippant in my phrasing, but, more rigorously: "the model will reflect and reify the biases of the person training the model and the data used to train it, and the people who annotate the training data") I like this: https://arxiv.org/abs/1706.09847 https://arxiv.org/abs/1706.09847
  - salty_biscuits 8y ago
    But the system in question is scanning a crowd looking for known faces in a database, race only enters it when you are comparing it to the human system of a police officer looking for terrorists at a football game. I get that people object to persistent surveillance but that doesn't seem to be what this system is about. It is looking for certain people at certain events. The way your face gets on the list in the first place is where you should put your concerns...
    - godelski 8y ago
      Well what I was saying, and why I suggested hedging the accusation, is that if someone does, say an AI that determines court sentences then that might cause the AI to be racist even though race was never set as a parameter. Such an AI might have racist behavior because of the naivety of designer[1], not that the designer was themselves racist. Obviously the designer could make this data set racist on purpose, I was just saying that an AI that judges on race might be due to stupidity and not malice. Hanlon's Razor. [1]Not realizing that there are other parameters that correlate with race