5 ms·

Maybe it's just me, but it seems to me that the author doesn't have a clue of what he is writing about. For instance, when he mentions the Amazon hiring AI emba

Maybe it's just me, but it seems to me that the author doesn't have a clue of what he is writing about. For instance, when he mentions the Amazon hiring AI embarrassment, it seem to me like he mixes the very different concepts of bias in machine learning with the everyday concept of bias as prejudice. If the sample data is representative of the population, then it is unbiased in machine learning terms, even if the population as a whole is prejudiced, or biased, which will be duly reflected in the learned model, which shouldn't surprise anyone with at least a rudimentary knowledge of machine learning.

piokoch 8y ago
I have the same impression. If AI system figured out that the risk of the car being stolen when someone lives in a certain place is bigger and adjusted fee to be higher, then it means AI worked correctly. One might claim (I presume author does and this is perfectly ok) that this is a bad thing that there are "bad neighborhoods" and we should fight it, but what does it have to do with proper calculation of insurance fee? Should we expect that AI will apply some "social justice" concepts while calculating insurance price. I am not sure what author proposes.
- charlysl 8y ago
  Or maybe I am wrong and the author is right. After all, he claims that the problem is not AI but the humans, and maybe we shouldn't blame systems trained on human data, which reflects all human's prejudices. Then again, if AI's aim is to perform activities until now restricted to humans, just like humans, how could it not be prejudiced? And who decides how AI should make decisions that are politically loaded?
- sgt101 8y ago
  Remember that the prediction is from the rear view window - on data that has been gathered in the past, and that now you are fixing risk in the future.
- skybrian 8y ago
  I'm not impressed with the article, but the idea here is that some forms of bias are okay, others are not, and it's up to the people working on the system to build it properly. There are laws about these things and you can't expect some algorithm to know the law automatically and avoid prohibited discrimination without going through the effort to build it in. It's kind of odd to think this would happen automatically. There is some kind of wishful thinking going on that technology can't have bad effects if you don't have bad intentions.
scottlocklin 8y ago
It's absolutely not just you: the author is completely clueless and is basically making shit up. For example: "The reason is not only because race is used as a factor in assigning risk (not unlike Amazon’s system picking “female” as a proxy for “inadequate”), but also because lenders are obviously incentivized to find new ways to make money on loans and insurance rates. " This is complete horse shit. I've done this sort of risk assessment for banks. Race is absolutely not used as a feature in assigning risk, and you have to work REALLY HARD to remove things such as zip codes which are simple proxies for race, AND you have to have an explainable model (GLM, decision tree, 2-3 layer neurons, etc) which you can defend as not being racist. This is a legal requirement, and I assume it is possible that banks have done overtly racist things in the past, but they certainly don't do this now: it's something that could cost them money. The over all point is legit: AI isn't real and anyone who thinks it is should have their head examined, machine learning is gratuitously overhyped, and its use by powerful groups is what you should fear. How he got there is not legit.
- sgt101 8y ago
  >AI isn't real and anyone who thinks it is should have their head examined, machine learning is gratuitously overhyped Whooohaaa! ARTIFICIAL INTELLIGENCE is not real, but there is a lot of AI TECHNOLOGY that is real and useful - for example the knowledge graph that Google uses, Bayesian modelling, economic and social simulation and so on. Both AI and ML are overhyped at the moment, data quality and knowledge engineering are killer problems, but there will be incremental improvements in both of these continually from now on, and this will enable both AI and ML tech to be much more easily applied than now.
  - scottlocklin 8y ago
    > the knowledge graph that Google uses, Bayesian modelling, economic and social simulation and so on... Those are not "AI TECHNOLOGY" -they're just math (the examples you list are basically all linear regression). Treating "AI" anything as if it were an actual subject outside of very, very obscure research projects is about 90% of the hype problem here. The use of the term is very much history repeating itself: people actually thought "AI" was a real thing back in the 1980s -the hype back then was even more ridiculous, and it was based on stuff which amounted to "we will use interpreters and parse trees."
noelsusman 8y ago
I don't think he was mixing his terms. He was clearly talking about the everyday concept of bias, not statistical bias. I don't think this author laid it out very well, but the problem of bias in machine learning is real and difficult to handle. The Amazon resume fiasco is a textbook example of what can go wrong with practices I see all the time. In my experience ML practitioners rarely stop and ask themselves the hard questions about this. They either don't think of it at all or they don't think we should be asking those questions. "The model is accurate" shouldn't be sufficient.
- charlysl 8y ago
  Well, maybe I misinterpreted what the author meant. In fact, I was a bit casual myself: statistical bias, which is what I meant, would have been more accurate than what I wrote, machine learning bias (which might be interpreted as the bias in "bias and variance", which has nothing to do with the discussion). The thing is, if the training data was generated by human behaviour, warts and all, then it is going to represent our prejudices. If this is a concern, then it seems to me that ML might not be the right technology to make decisions that have a political dimension. If we want the system's predictions to be PC, then a rules based system might be more appropriate, blast from the past, unless we start labeling samples, with their output as an extra feature, as "PC" and "Not PC".

piokoch 8y ago

I have the same impression. If AI system figured out that the risk of the car being stolen when someone lives in a certain place is bigger and adjusted fee to be higher, then it means AI worked correctly. One might claim (I presume author does and this is perfectly ok) that this is a bad thing that there are "bad neighborhoods" and we should fight it, but what does it have to do with proper calculation of insurance fee? Should we expect that AI will apply some "social justice" concepts while calculating insurance price. I am not sure what author proposes.

charlysl 8y ago
Or maybe I am wrong and the author is right. After all, he claims that the problem is not AI but the humans, and maybe we shouldn't blame systems trained on human data, which reflects all human's prejudices. Then again, if AI's aim is to perform activities until now restricted to humans, just like humans, how could it not be prejudiced? And who decides how AI should make decisions that are politically loaded?
sgt101 8y ago
Remember that the prediction is from the rear view window - on data that has been gathered in the past, and that now you are fixing risk in the future.
skybrian 8y ago
I'm not impressed with the article, but the idea here is that some forms of bias are okay, others are not, and it's up to the people working on the system to build it properly. There are laws about these things and you can't expect some algorithm to know the law automatically and avoid prohibited discrimination without going through the effort to build it in. It's kind of odd to think this would happen automatically. There is some kind of wishful thinking going on that technology can't have bad effects if you don't have bad intentions.

scottlocklin 8y ago

It's absolutely not just you: the author is completely clueless and is basically making shit up. For example: "The reason is not only because race is used as a factor in assigning risk (not unlike Amazon’s system picking “female” as a proxy for “inadequate”), but also because lenders are obviously incentivized to find new ways to make money on loans and insurance rates. " This is complete horse shit. I've done this sort of risk assessment for banks. Race is absolutely not used as a feature in assigning risk, and you have to work REALLY HARD to remove things such as zip codes which are simple proxies for race, AND you have to have an explainable model (GLM, decision tree, 2-3 layer neurons, etc) which you can defend as not being racist. This is a legal requirement, and I assume it is possible that banks have done overtly racist things in the past, but they certainly don't do this now: it's something that could cost them money. The over all point is legit: AI isn't real and anyone who thinks it is should have their head examined, machine learning is gratuitously overhyped, and its use by powerful groups is what you should fear. How he got there is not legit.

sgt101 8y ago
>AI isn't real and anyone who thinks it is should have their head examined, machine learning is gratuitously overhyped Whooohaaa! ARTIFICIAL INTELLIGENCE is not real, but there is a lot of AI TECHNOLOGY that is real and useful - for example the knowledge graph that Google uses, Bayesian modelling, economic and social simulation and so on. Both AI and ML are overhyped at the moment, data quality and knowledge engineering are killer problems, but there will be incremental improvements in both of these continually from now on, and this will enable both AI and ML tech to be much more easily applied than now.
- scottlocklin 8y ago
  > the knowledge graph that Google uses, Bayesian modelling, economic and social simulation and so on... Those are not "AI TECHNOLOGY" -they're just math (the examples you list are basically all linear regression). Treating "AI" anything as if it were an actual subject outside of very, very obscure research projects is about 90% of the hype problem here. The use of the term is very much history repeating itself: people actually thought "AI" was a real thing back in the 1980s -the hype back then was even more ridiculous, and it was based on stuff which amounted to "we will use interpreters and parse trees."

noelsusman 8y ago

I don't think he was mixing his terms. He was clearly talking about the everyday concept of bias, not statistical bias. I don't think this author laid it out very well, but the problem of bias in machine learning is real and difficult to handle. The Amazon resume fiasco is a textbook example of what can go wrong with practices I see all the time. In my experience ML practitioners rarely stop and ask themselves the hard questions about this. They either don't think of it at all or they don't think we should be asking those questions. "The model is accurate" shouldn't be sufficient.

charlysl 8y ago
Well, maybe I misinterpreted what the author meant. In fact, I was a bit casual myself: statistical bias, which is what I meant, would have been more accurate than what I wrote, machine learning bias (which might be interpreted as the bias in "bias and variance", which has nothing to do with the discussion). The thing is, if the training data was generated by human behaviour, warts and all, then it is going to represent our prejudices. If this is a concern, then it seems to me that ML might not be the right technology to make decisions that have a political dimension. If we want the system's predictions to be PC, then a rules based system might be more appropriate, blast from the past, unless we start labeling samples, with their output as an extra feature, as "PC" and "Not PC".