5 ms·

By reading, seems that the algorithm is just a scapegoat used in the title. This is racial discriminations dating way back, purposely introduced in the algorith

By reading, seems that the algorithm is just a scapegoat used in the title. This is racial discriminations dating way back, purposely introduced in the algorithm, and supported for 6 years. > Having dual nationality was marked as a big risk indicator, as was a low income. >In 2020, Trouw and another Dutch news outlet, RTL Nieuws revealed that the tax authorities also kept secret blacklists of people for two decades, which tracked both credible and unsubstantiated “signals” of potential fraud. Citizens had no way of finding out why they were on the list or defending themselves. >An audit showed that the tax authorities focused on people with “a non-Western appearance,” while having Turkish or Moroccan nationality was a particular focus. Being on the blacklist also led to a higher risk score in the child care benefits system.

ClumsyPilot 4y ago
No fly lists too, no algorythms involved
- droopyEyelids 4y ago
  The only reason it isn't an "algorithm" is because no one thought to use that language when it was created. All "algorithm" means is "a process or set of rules to be followed in calculations"- computers don't need to be involved, nor does "AI". The no-fly list is a process of looking up people in a list of names when calculating whether they can fly. That fits the definition as much as anything else the media calls an algorithm.
  - samhw 4y ago
    Honestly, as someone most of whose life is spent writing algorithms, when I hear any non-technical person say the word "algorithm" that's my cue to tune out. "Algorithms" seems to have a meaning that's approximately "super-intelligent runaway AI robots ... oh, and they're also racist". Meanwhile, neural nets are accomplishing the great, hitherto-unreachable task of [checks notes] grade school math: https://www.tensorflow.org/datasets/catalog/math_dataset https://www.tensorflow.org/datasets/catalog/math_dataset
  - raxxorraxor 4y ago
    Very true that this just obscures the issue with processing personal data. That of course happens already on large scales for years.
neaden 4y ago
Yeah based on the article I think the racism was a feature, not a bug.
photochemsyn 4y ago
People across the government-corporate spectrum are now fully aware that if they train 'self-learning algorithms' on biased data sets then those algorithms will adopt those biases. Such people seem to believe they can then get away with blaming the algorithm for the outcome while avoiding any legal responsibility themselves. On the positive side more and more people are generally aware of how this works. Criminal investigators need to look at the people who selected the datasets that were used to train the algorithms, they're the ones who bear responsibilty.
- masklinn 4y ago
  > People across the government-corporate spectrum are now fully aware that if they train 'self-learning algorithms' on biased data sets then those algorithms will adopt those biases. "Bias laundering" is how it's usually called. Take biased training data, put it in "the algorithm", and presto it's all legal because it comes from "the algorithm".
legulere 4y ago
Algorithms calculating the probability of some event of a person will always be discriminatory, because the probability depends on which groups you belong to. The problem is that algorithms are an easy way to hide behind. Think of an algorithm to check wether to do a police check for a person. Statistically black people are more likely to commit crimes. The police could easily hide behind an algorithm and say that they are not doing checks only on black people but only on persons flagged by the algorithm. The problem with that is that most people are not criminal, but will be discriminated just because of which group they belong to.
- pessimizer 4y ago
  > Statistically black people are more likely to commit crimes Statistically black people are more likely to be arrested and convicted for crimes. So if you use an algorithm to determine who should be arrested and convicted, how long they should be sentenced, and likelihood of recidivism, and seed that algorithm with historical information about arrests and convictions tagged by race (even inadvertently through names or addresses), you end up permanently encoding untrue information like "black people are more likely to commit crimes." an example from a particular class of crimes: Study: Whites More Likely to Abuse Drugs Than Blacks https://healthland.time.com/2011/11/07/study-whites-more-likely-to-abuse-drugs-than-blacks/ https://healthland.time.com/2011/11/07/study-whites-more-lik... Black and white Americans use drugs at similar rates. One group is punished more for it. https://www.vox.com/2015/3/17/8227569/war-on-drugs-racism https://www.vox.com/2015/3/17/8227569/war-on-drugs-racism
  - skrbjc 4y ago
    For the first article: "The study, which was published Monday in the Archives of General Psychiatry, controlled for variables like socioeconomic status because rates of severe drug problems tend to be greater amongst the poor. Despite this, Native American youth fared worst, with 15% having a substance use disorder, compared to 9.2% for people of mixed racial heritage, 9.0% for whites, 7.7% for Hispanics, 5% for African Americans and 3.5% for Asians and Pacific Islanders." You can get into a debate about the merits of controlling for factors, but the title is somewhat misleading. They are suggesting that whites have higher rates of drug use when you control for factors like socioeconomic status, not that the hard numbers show that whites use more drugs than blacks. The second article is literally two paragraphs with no actual data other than an infographic. I think you need some better sources if you want to make the claims you are trying to make.
puszczyk 4y ago
What I don't understand looking at the article is how > “signals” of potential fraud turned into 100k EUR fines. Shouldn't the "signal" only flag suspect transfers and start in-person investigation? Or did they just slap the fines without any human due-process? I guess this is only possible because it's an administrative fine, and not a criminal case, right?

ClumsyPilot 4y ago

No fly lists too, no algorythms involved

droopyEyelids 4y ago
The only reason it isn't an "algorithm" is because no one thought to use that language when it was created. All "algorithm" means is "a process or set of rules to be followed in calculations"- computers don't need to be involved, nor does "AI". The no-fly list is a process of looking up people in a list of names when calculating whether they can fly. That fits the definition as much as anything else the media calls an algorithm.
- samhw 4y ago
  Honestly, as someone most of whose life is spent writing algorithms, when I hear any non-technical person say the word "algorithm" that's my cue to tune out. "Algorithms" seems to have a meaning that's approximately "super-intelligent runaway AI robots ... oh, and they're also racist". Meanwhile, neural nets are accomplishing the great, hitherto-unreachable task of [checks notes] grade school math: https://www.tensorflow.org/datasets/catalog/math_dataset https://www.tensorflow.org/datasets/catalog/math_dataset
- raxxorraxor 4y ago
  Very true that this just obscures the issue with processing personal data. That of course happens already on large scales for years.

neaden 4y ago

Yeah based on the article I think the racism was a feature, not a bug.

photochemsyn 4y ago

People across the government-corporate spectrum are now fully aware that if they train 'self-learning algorithms' on biased data sets then those algorithms will adopt those biases. Such people seem to believe they can then get away with blaming the algorithm for the outcome while avoiding any legal responsibility themselves. On the positive side more and more people are generally aware of how this works. Criminal investigators need to look at the people who selected the datasets that were used to train the algorithms, they're the ones who bear responsibilty.

masklinn 4y ago
> People across the government-corporate spectrum are now fully aware that if they train 'self-learning algorithms' on biased data sets then those algorithms will adopt those biases. "Bias laundering" is how it's usually called. Take biased training data, put it in "the algorithm", and presto it's all legal because it comes from "the algorithm".

legulere 4y ago

Algorithms calculating the probability of some event of a person will always be discriminatory, because the probability depends on which groups you belong to. The problem is that algorithms are an easy way to hide behind. Think of an algorithm to check wether to do a police check for a person. Statistically black people are more likely to commit crimes. The police could easily hide behind an algorithm and say that they are not doing checks only on black people but only on persons flagged by the algorithm. The problem with that is that most people are not criminal, but will be discriminated just because of which group they belong to.

pessimizer 4y ago
> Statistically black people are more likely to commit crimes Statistically black people are more likely to be arrested and convicted for crimes. So if you use an algorithm to determine who should be arrested and convicted, how long they should be sentenced, and likelihood of recidivism, and seed that algorithm with historical information about arrests and convictions tagged by race (even inadvertently through names or addresses), you end up permanently encoding untrue information like "black people are more likely to commit crimes." an example from a particular class of crimes: Study: Whites More Likely to Abuse Drugs Than Blacks https://healthland.time.com/2011/11/07/study-whites-more-likely-to-abuse-drugs-than-blacks/ https://healthland.time.com/2011/11/07/study-whites-more-lik... Black and white Americans use drugs at similar rates. One group is punished more for it. https://www.vox.com/2015/3/17/8227569/war-on-drugs-racism https://www.vox.com/2015/3/17/8227569/war-on-drugs-racism
- skrbjc 4y ago
  For the first article: "The study, which was published Monday in the Archives of General Psychiatry, controlled for variables like socioeconomic status because rates of severe drug problems tend to be greater amongst the poor. Despite this, Native American youth fared worst, with 15% having a substance use disorder, compared to 9.2% for people of mixed racial heritage, 9.0% for whites, 7.7% for Hispanics, 5% for African Americans and 3.5% for Asians and Pacific Islanders." You can get into a debate about the merits of controlling for factors, but the title is somewhat misleading. They are suggesting that whites have higher rates of drug use when you control for factors like socioeconomic status, not that the hard numbers show that whites use more drugs than blacks. The second article is literally two paragraphs with no actual data other than an infographic. I think you need some better sources if you want to make the claims you are trying to make.

puszczyk 4y ago

What I don't understand looking at the article is how > “signals” of potential fraud turned into 100k EUR fines. Shouldn't the "signal" only flag suspect transfers and start in-person investigation? Or did they just slap the fines without any human due-process? I guess this is only possible because it's an administrative fine, and not a criminal case, right?