3 ms·

For anyone who is jumping to the comments to complain about how more rules from the EU is going to make innovation difficult, I highly recommend you to read the

For anyone who is jumping to the comments to complain about how more rules from the EU is going to make innovation difficult, I highly recommend you to read the summary presentation: https://www.ceps.eu/wp-content/uploads/2021/04/AI-Presentation-CEPS-Webinar-L.-Sioli-23.4.21.pdf https://www.ceps.eu/wp-content/uploads/2021/04/AI-Presentati... Basically, as I understand it, it divides AI systems (in the broadest sense Machine Learning sense) into risk categories: unacceptable risk (prohibited), high risk, medium/other risk, and low risk. Applications in the high risk category include medical devices, law enforcement, recruiting/employment and others. AI systems in this category will be subject to the requirements mentioned by most people here (oversight, clean and correct training data, etc). Medium risk applications seem to revolve around the risk of tricking people, for example via chatbots, deepfakes etc. In this case they require to “notify” people that they are interacting with an AI or that the content was generated by AI. How this can be enforced in practice remains to be seen. And the low risk category is basically everything else, from marketing applications to ChatGPT (as I understand it). Applications in this category would have no mandatory obligations. If you ask me, that’s a quite sensible approach.

kortilla 3y ago
How is recruiting higher risk than deepfakes?
- alwayseasy 3y ago
  As in a deepfake customer service rep. Not a deepfake President of the United States declaring he's leaving NATO.
- mrtksn 3y ago
  Systematic discrimination in employment because the AI said so because the data was biased can ghettos and riots. Deepfakes are a storm in a teacup. Remember how OpenAI's Dall-E lost to Stable Diffusion because they were too afraid that someone will make an image of a politician or something? Well we had some fun with Pope wearing Balenciaga and Trump being arrested but nothing bad happened.
kpw94 3y ago
Not going to comment on how determining which risk category a given system falls into. > clean and correct training data If there's a country where majority of poor people have blue skin, and majority of rich people have green skin, and poorer people commit more crimes in general. Then if one of the features (explicitly defined or somehow inferred by model) of your ML model is skin color, it will correctly correlate blue skin as high risk for stealing. So a supermarket chain AI face detection would always flag those more often. And so you'd have AI-scale systemic racism. Even if that application was categorized as "high risk", since it is "clean and correct training data". How do you legislate that an AI system can't fall for "correlation != causation" and how can you audit that if there's no clearly define ML model feature, like the simple neural networks studied in ML 101.
- jdiez17 3y ago
  Indeed, you’ve highlighted one of the potential problems with applying ML (a.k.a. a big pile of statistics/linear algebra/whathaveyou) in ways that threaten fundamental rights. In this case by creating a product that would discriminate people based on protected attributes like skin color. That’s illegal, even if your ML system picks up on that correlation. So you agree that we shouldn’t allow companies to ML-wash discrimination?
- deserialized 3y ago
  Maybe do a better job on the AI so it correlates poverty to crime instead of skin color to crime, or is able to spot laws created specifically to target and oppress the poor. If the punishment is a fine, then it's legal for the rich
- neel_k 3y ago
  Luckily, lawyers in the 1970s already figured this out, by anticipating the research on causality that computer scientists and mathematicians like Judea Pearl and Peter Spirtes did in the 1990s. Really! In the first instance, you just can't use race as a feature, since it is a protected characteristic. But, you might also be worried that protected characteristics can generally be easily identified by looking for innocuous traits that correlate (since people tend to cluster into communities). For example, if you know an American's ZIP code and their three favorite musicians, you can determine their race with an accuracy in the high 90s. (Basically, the US is still just as segregated now as it was a hundred years ago, and black and white Americans tend to listen to different music.) So when the US Civil Rights Act was passed, the courts came up with the idea of "disparate impact" -- when doing something like hiring, you are not allowed to base the decision on features that disproportionately affect one group rather than another, even if they are formally neutral, unless the feature directly impacts the ability to do the job. In other words, you have to show that the features you are basing the decision on _causally influence_ the decision you are making, exactly like you see in Pearl's causal influence diagrams or structural equation models or whatever. Eg, if you want to hire a math professor, you can base the decision on the articles they published in math journals, but you can't base it on whether they like old school Goa trance. So, what about black box neural networks, where you don't know which features are being used? In this case, it's pretty clear that you shouldn't use them directly when making a home loan, because the law wants to know what features are in use, and you can't answer the question of whether you're redlining when you have a black box. However, using black box techniques to learn (eg) the best random forest model to use is fine, because it lets you easily see which factors are going into the decision before deploying it. FWIW, people have been doing this for decades already. (I did stuff like this back in the 1990s.)

kortilla 3y ago

How is recruiting higher risk than deepfakes?

alwayseasy 3y ago
As in a deepfake customer service rep. Not a deepfake President of the United States declaring he's leaving NATO.
mrtksn 3y ago
Systematic discrimination in employment because the AI said so because the data was biased can ghettos and riots. Deepfakes are a storm in a teacup. Remember how OpenAI's Dall-E lost to Stable Diffusion because they were too afraid that someone will make an image of a politician or something? Well we had some fun with Pope wearing Balenciaga and Trump being arrested but nothing bad happened.

kpw94 3y ago

Not going to comment on how determining which risk category a given system falls into. > clean and correct training data If there's a country where majority of poor people have blue skin, and majority of rich people have green skin, and poorer people commit more crimes in general. Then if one of the features (explicitly defined or somehow inferred by model) of your ML model is skin color, it will correctly correlate blue skin as high risk for stealing. So a supermarket chain AI face detection would always flag those more often. And so you'd have AI-scale systemic racism. Even if that application was categorized as "high risk", since it is "clean and correct training data". How do you legislate that an AI system can't fall for "correlation != causation" and how can you audit that if there's no clearly define ML model feature, like the simple neural networks studied in ML 101.

jdiez17 3y ago
Indeed, you’ve highlighted one of the potential problems with applying ML (a.k.a. a big pile of statistics/linear algebra/whathaveyou) in ways that threaten fundamental rights. In this case by creating a product that would discriminate people based on protected attributes like skin color. That’s illegal, even if your ML system picks up on that correlation. So you agree that we shouldn’t allow companies to ML-wash discrimination?
deserialized 3y ago
Maybe do a better job on the AI so it correlates poverty to crime instead of skin color to crime, or is able to spot laws created specifically to target and oppress the poor. If the punishment is a fine, then it's legal for the rich
neel_k 3y ago
Luckily, lawyers in the 1970s already figured this out, by anticipating the research on causality that computer scientists and mathematicians like Judea Pearl and Peter Spirtes did in the 1990s. Really! In the first instance, you just can't use race as a feature, since it is a protected characteristic. But, you might also be worried that protected characteristics can generally be easily identified by looking for innocuous traits that correlate (since people tend to cluster into communities). For example, if you know an American's ZIP code and their three favorite musicians, you can determine their race with an accuracy in the high 90s. (Basically, the US is still just as segregated now as it was a hundred years ago, and black and white Americans tend to listen to different music.) So when the US Civil Rights Act was passed, the courts came up with the idea of "disparate impact" -- when doing something like hiring, you are not allowed to base the decision on features that disproportionately affect one group rather than another, even if they are formally neutral, unless the feature directly impacts the ability to do the job. In other words, you have to show that the features you are basing the decision on _causally influence_ the decision you are making, exactly like you see in Pearl's causal influence diagrams or structural equation models or whatever. Eg, if you want to hire a math professor, you can base the decision on the articles they published in math journals, but you can't base it on whether they like old school Goa trance. So, what about black box neural networks, where you don't know which features are being used? In this case, it's pretty clear that you shouldn't use them directly when making a home loan, because the law wants to know what features are in use, and you can't answer the question of whether you're redlining when you have a black box. However, using black box techniques to learn (eg) the best random forest model to use is fine, because it lets you easily see which factors are going into the decision before deploying it. FWIW, people have been doing this for decades already. (I did stuff like this back in the 1990s.)