4 ms·

I'd like to throw a thought out there: It is a problem if the government uses algorithms or ML systems that are secret and/or cannot be independently vetted by

I'd like to throw a thought out there: It is a problem if the government uses algorithms or ML systems that are secret and/or cannot be independently vetted by the citizenry. Judges cannot just sign off on warrants and sentencing just because "the computer says so", for all we know the computer may be programmed to say "poor and non-comformist = guilty" or some other nonsense, and/or reprogrammed after every election. We need to be able to trust this stuff.

emteycz 6y ago
Is it what's happening? I thought "computer says so" triggers an investigation, which in turns produces reasoning for the judge, based on which the judge decides? Is there any source that confirms the belief about judges deciding based on computer output in the past or in the future - relevant cases, law, etc?
- mikem170 6y ago
  The worst I've heard of along these lines currently in practice is predictive policing [0]: First the Sheriff’s Office generates lists of people it considers likely to break the law, based on arrest histories, unspecified intelligence and arbitrary decisions by police analysts. Then it sends deputies to find and interrogate anyone whose name appears, often without probable cause, a search warrant or evidence of a specific crime. They swarm homes in the middle of the night, waking families and embarrassing people in front of their neighbors. They write tickets for missing mailbox numbers and overgrown grass, saddling residents with court dates and fines. They come again and again, making arrests for any reason they can. One former deputy described the directive like this: “Make their lives miserable until they move or sue.” It seems that police departments are happy to buy tools for stuff like this, and private companies are happy to make a buck. But are these systems vetted by anyone on behalf of the public? Some like sting rays and breathalyzers have been hidden behind non-disclosure agreements, kept out of open court, etc. Other than the above I've also heard of computer systems in NJ advising judges whether a suspect is a risk of not appearing in court (as they are reforming the bail system). I wondered if the criteria is published, how it reviewed and modified, etc. I'm just concerned about the increasing influence of hidden algorithms on our society, and very concerned in general that the government is going to hook all of its databases together and do more of this, a la Chinese social scores, etc. [0] https://www.techdirt.com/articles/20200907/12212945257/florida-sheriffs-predictive-policing-program-is-protecting-residents-unkempt-lawns-missing-mailbox-numbers.shtml https://www.techdirt.com/articles/20200907/12212945257/flori...

emteycz 6y ago

Is it what's happening? I thought "computer says so" triggers an investigation, which in turns produces reasoning for the judge, based on which the judge decides? Is there any source that confirms the belief about judges deciding based on computer output in the past or in the future - relevant cases, law, etc?

mikem170 6y ago
The worst I've heard of along these lines currently in practice is predictive policing [0]: First the Sheriff’s Office generates lists of people it considers likely to break the law, based on arrest histories, unspecified intelligence and arbitrary decisions by police analysts. Then it sends deputies to find and interrogate anyone whose name appears, often without probable cause, a search warrant or evidence of a specific crime. They swarm homes in the middle of the night, waking families and embarrassing people in front of their neighbors. They write tickets for missing mailbox numbers and overgrown grass, saddling residents with court dates and fines. They come again and again, making arrests for any reason they can. One former deputy described the directive like this: “Make their lives miserable until they move or sue.” It seems that police departments are happy to buy tools for stuff like this, and private companies are happy to make a buck. But are these systems vetted by anyone on behalf of the public? Some like sting rays and breathalyzers have been hidden behind non-disclosure agreements, kept out of open court, etc. Other than the above I've also heard of computer systems in NJ advising judges whether a suspect is a risk of not appearing in court (as they are reforming the bail system). I wondered if the criteria is published, how it reviewed and modified, etc. I'm just concerned about the increasing influence of hidden algorithms on our society, and very concerned in general that the government is going to hook all of its databases together and do more of this, a la Chinese social scores, etc. [0] https://www.techdirt.com/articles/20200907/12212945257/florida-sheriffs-predictive-policing-program-is-protecting-residents-unkempt-lawns-missing-mailbox-numbers.shtml https://www.techdirt.com/articles/20200907/12212945257/flori...