5 ms·

Obviously, this is a very crude example of AI acting in a racist manner -- basically, just parroting back phrases -- but it's worth thinking about how AI might

• jxy 11y ago
  There is nothing complex about processing some numbers and make inferences. Simple algorithms do that. What a complex AI should be able to do is infer from human social and emotional behaviors.
• wjnc 11y ago
  Several letters / positions in ones name are pretty highly correlated with several ethnic groups. So many automated systems knowing names, would use this information, and act in an illegal way. So totally true and a reason why unmanaged data mining / machine learning will get many organisations towards substantial regulatory problems.
• rtpg 11y ago
  This kind of gets solved through the courts "Oh, you used an AI to establish probable cause?" "Yes, your honor" "Can you actually explain what happened?" "We give it a bunch of data and that makes a neural network and then it gives us answers" "OK... so basically you do not know the source of PC? This is the same has having no probable cause. Illegal search" There's something fascinating about how most AIs work. It's almost impossible (for the learning, neural network-y type) to breakdown what the "thought process" was for a result. Seemingly unimportant details because the main thought process for the AI (even if the correlation is spurious). I'm almost surprised that Google can implement any court ruling, considering how their search must be based so much on data fed to a big system.
  • dietrichepp 11y ago
    "We do not know probable cause" could easily become "neural nets are a statistical technique, here's the case law supporting it." This is why AI, as a field, is known as machine learning these days. Not because of substantive changes in content, but primarily due to the complicated and poorly-understood implications of the word "intelligence".
  • gaur 11y ago
    There are plenty of pro-police-state judges who would trip over themselves to issue AI-generated warrants. It will not get resolved through the courts.
• bilbo0s 11y ago
  ??? Why would you create an AI bot for policing that said, "... "Hey, here's a dataset. Cluster the people in the set." ..." ? (That doesn't really strike me as policing. Policing is enforcing the current laws. And there is nothing about the current law in a dataset that simply clusters people.) Not trying to be snarky, that was a serious question. If you were making a bot to police laws, would it not make sense to feed the bot a learning set of laws, the punishments for those laws, and then as input feed it what the defendant did ? Why would it need any information about who the defendant was at all ? (Actually, maybe it might need a list of the defendant's prior convictions for instance... but little else.) I guess my question is, why is an AI for enforcing laws, instead being fed information to identify groups of people ?
  • askldfhjkasfhd 11y ago
    Courts enforce the laws. Policing (detective work, particularly) is to identify people to feed into the court system.
• golergka 11y ago
  > And I think most of us agree that it is good that it is illegal, because we know that it is unfair to show bias against a person based on the color of their skin. That's not the reason. The reason is that correlation is weak. Think about it: people with down syndrome are genetically different others. So are people of different races. But when people with down syndrome are supposed to have lower then average intelligence, no one is offended by it, because the correlation is there, and it's strong. Meanwhile, all statistical comparisons of different races shown results with very low significance.
• IkmoIkmo 11y ago
  > For instance, at least here in the U.S., it is illegal for police to profile people based on race It must also be mentioned, sadly, that data shows that shows that police profile racially despite it being illegal, including, sadly, judges, who seem to punish minorities about whom there are negative stereotypes more harshly for similar crimes. > in the aggregate, have some predictive value. This data does have predictive value, for sure, studies have shown. But it's important to recognise that it's not the ethnicity that is a causal factor, which studies have shown over and over again, too. (at least in the countries I'm familiar with, not sure how it is in the US but I wouldn't be surprised if the results are similar). i.e., it may well be true that a minority has a higher probability of engaging in crime, such that racial profiling has value for the police. But studies show that this is because this particular minority is: 1) more likely to be less educated, live in a poorer neighbourhood, have fewer well-connected friends, be of a younger age, more likely to be unemployed, more likely to have parents who are unemployed or illiterate etc etc. Once you adjust for all these things, i.e. compare a minority person of age x, employment y, neighbourhood z, education t etc, to a majority person with similar parameters, the probability of crime is not significantly different. Such that it's not the ethnicity or culture, but rather the socioeconomic standing of a person. And that standing is often a function of the system of government, the institutions, the (migration) history etc of a group of people. For example, Italians in their first years were socioeconomically worse off, explained a lot by their recent migration history (poor migrants in a new country), and they tended to crime more than the average person. Today that's no longer the case. None of that had much to do with their skin color, ethnicity or culture. So while ethnicity has predictive values, it's not a factor itself. Rather it's a proxy for recognising a person's probable socioeconomic status, which is a key factor in crime. If you can establish that by looking at a person's ethnicity, that's useful, but it's also racist by definition. But that does mean the following, if a bot is supplied ample data, it may not need to use ethnicity as a proxy for key factors (e.g. socioeconomic status, broadly) that can predict crime. We can skip racial profiling all together. And then we'll be stuck with a new problem, not racism, but some form of classism that we've seen before but in a new way. I felt that China recently made its first steps towards such a future with the scoring of citizens on a whole range of parameters. It's hugely valuable, but it reduces people to being scored on a number of parameters, e.g. income, education of parents, favorite sport etc, just like skin color, for which you can run all kinds of regressions to figure out which factors are 'good' and 'bad' for any given scoring... and then a priori judge people, rather than allow for the possibility that someone who has parameters that are statistically likely to be bad, may be perfectly fine and do nothing wrong (like skin color, where statistically being 'black' is 'worse' on a whole range of topics, from which it does not follow that any black individual is worse.) That's the story of racism, making judgements before the fact on the basis of a parameter that reduces a person to a society's perception of that parameter, skin color. It's bad, but I'm afraid it may become worse. Racism will continue to be 'not done', but it may be replaced by proxies that are just as bad, which are considered perfectly fine. One example is where in my country, it's not incredibly popular to demand X amount of income for a home. The idea is that, if you don't have X, you can't afford it. The truth is, that X is way higher than the amount you need to reasonable afford that home, and so it's essentially creating a neighbourhood of certain affluence that weeds out socioeconomic groups that could afford to live there, but are discriminated against on the basis of their income, which, surprise, happens to create segregated communities because ethnicity are income are still pretty closely linked. But because it's done under the guise of 'we're just protecting ourselves and renters by renting only to those who can afford it', it doesn't create a single word of discussion. It's a tricky situation because I tend to appreciate the concept of minimum income for homes, but the level they set it at made me suspicious, and I'm seeing neighbourhoods segregate in my city. Further, OF COURSE it must be allowed to judge people on certain parameters, like say a degree when getting a job as a proxy for how qualified someone is, even though a non-degree holder may be more qualified, it's a reasonable form of discrimination. But at some point, we're also reducing a person to a small set of data and its associated probability set, much like skin color. I find this a tricky issue, although fortunately it seems that non-racial discrimination is limited to only a few things, like gender and age. You hear stories of some companies rejecting a person on the basis of his post code or address (reject poor neighbourhoods), looks or favorite sport (ski vacations?), but it's not widespread. Anyway so much for my rambling. In short, I think it's reasonably easy to not give a bot racial data, but I think the bot will recreate the probabilities on the basis of other data that proxies socieconomic status, and that this is concerning in a way that is similar to racial profiling, because it reduces a person to the parameters he has and how those parameters happen to correlate with e.g. crime in a large group of people that he may behave differently to. This is how the race-parameter works, you're black? well that correlates to crime in a group of people, so we'll treat you like you're likely to be a criminal. That's bad, but if you ignore that parameter, there are others that are similarly bad, that a bot will find. Like oh you're poor, or oh you love in that neighborhood, or oh your parents are illiterate? Well that correlates to crime, so we'll treat you like you're likely to be a criminal. Non-racial parameter profiling, bots will be champs in that.
• mikestew 11y ago
  But what about a bot, particularly one that is powered by the type of AI that is complex enough to make its own inferences and form its own conclusions based on the data presented, rather than being fed a bunch of rules? That bot then figures out that, in aggregate, we're all a bunch of assholes, then something something Skynet.
  • smileysteve 11y ago
    Had a discussion the other day, there are two likely outcomes as AIs process humanity, Skynet or Ignorance. The Matrix, Terminator etc being Skynet - War Games being ignorance (stop playing the game)
    • mikestew 11y ago
      Hmm, interesting thought. Where I take this is that the War Games scenario might be more likely. Given that AI will not just wake up one day fully self-aware and aware of its externalities, I'll assume it learns about things as it needs to know. Needs to know where the opponents nuclear missiles are stationed. Needs to know the major population centers, which are just ints stored in database rows. Needs to nothing about "people" because those aren't parameters worth knowing for the task. So, yeah, I've always been a Terminator/Matrix kind of guy, but your comment moved me to wonder if I've considered all possibilities.
      • mattkevan 11y ago
        Have you read Rule 34 by Charles Stross? In it, a machine learning spam filter decides that the best way to stop spam is to bump off the spammers themselves. No-one realises its found such a drastic solution until people start dying in mysterious ways. I'm not sure the robots will take over Matrix or Terminator style - instead, as algorithms get more complex and are able to take greater leaps of logic, we'll see more and more strange behaviour until suddenly things get really weird. And then it'll be too late. Too many of our systems will be reliant on them to reverse course. The digital domain will have become quite literally another domain of life, and we'll just have to deal with the messiness and unpredictability that entails.
• patio11 11y ago
  People have made this argument against FICO, for what it's worth. (FICO is likely the most consequential AI employed in the United States outside of the Googleplex. It is not called AI because AI as a field demotes anything that ships into mathematics or engineering.) FICO operates off of credit reports, which have no explicit way to encode race in them. It is fundamentally a clustering algorithm. It is very good at clustering. (There's a good counterargument to be made, which is that FICO actually clusters people based off observable behavior and the system which FICO supplements/replaces -- human underwriters -- cluster people based on "Tom is an upstanding Christian who goes to my church; of course he is good for the money.")
• Houshalter 11y ago
  The purpose of these rules is to keep stupid humans from being biased against race. Humans are awful and treat people very differently if they are just ugly or have different political opinions, etc. An algorithm only cares what is true. It's not biased. It does its best to make the most accurate predictions possible with whatever data it can get. So it's not the same as a jerk that decides he hates black people and won't give them a loan. It's not even remotely the same as a chatbot trained on 4chan trolls. The alternative is not using predictive algorithms at all. Humans count too btw. Humans are the worst. You can't just remove features that correlate with race, since everything correlates with everything. At best you can give it race as a variable, and then control for that, so it gives the average result either way. Rather than trying to infer race from data. But why? If men really are more likely to get in accidents than women, they should pay higher insurance. The whole point of insurance is to predict your risk. Wood houses should pay more for fire insurance than a new brick house, etc.
• mizzao 11y ago
  Sendhil Mullainathan (econ prof at Harvard) is doing some work on using machine learning to predict whether people should be granted bail. The machine discrimination (no pun intended) issues here could be quite terrifying. Not published yet, but example abstract: http://inequality.hks.harvard.edu/event/sendhil-mullainathan-title-tba http://inequality.hks.harvard.edu/event/sendhil-mullainathan... On the other hand, there's starting to be some research on how to codify and prevent this in ML. For example: http://arxiv.org/abs/1412.3756 http://arxiv.org/abs/1412.3756
• Blarbenfad 11y ago
  The FAT ML (Fairness, Accountability, and Transparency in Machine Learning) conference proceedings would definitely be of interest to you, if you're interested in learning more about these kinds of questions. http://www.fatml.org/index.html#scope http://www.fatml.org/index.html#scope