6 ms·

Why not change the training data to reflect an equal amount of the population? Officially the US has six “races” according to the Census Bureau. So, another opt

Why not change the training data to reflect an equal amount of the population? Officially the US has six “races” according to the Census Bureau. So, another option is to make each race ~16% of the training data. Some might object to the decreased accuracy this gives whites — but if we’re going to have any hope of preventing existing systemic inequalities from being encoded in AI systems, we’ll have to socially engineer the data sets.

zo1 7y ago
>"Some might object to the decreased accuracy this gives whites — but if we’re going to have any hope of preventing existing systemic inequalities from being encoded in AI systems, we’ll have to socially engineer the data sets. " So to fix systemic inequalities for some groups, we're going to increase systemic inequality for certain other groups? Definitely a step in the wrong direction as it's just going to swing hatred back the other way. And N-levels later of this "fiddling" you'll end up in a spot where neither side can reconcile with the other because each side has legitimate reasons to feel slighted and no one can look past that "hatred". Just look at the mess that is currently in the middle-east after people "fiddled", and then "fiddled" again to try fix it.
- pessimizer 7y ago
  > So to fix systemic inequalities for some groups, we're going to increase systemic inequality for certain other groups? No; in order to accurately identify people through appearances, we'll try to sample evenly throughout the range of appearances and features instead of sampling based on proportions within the population of the people setting up the system. You're can't be seriously making the case that white people are such an overwhelming portion of the world's population that not concentrating on them in your training data constitutes racial discrimination. Not only racial discrimination even, but is a attack that will inspire a "legitimate" counter-retaliation.
  - zo1 7y ago
    No that's not what I'm saying. If you look at the quote I mentioned, the person was suggesting we "socially engineer" data sets in order to "preventing existing systemic inequalities from being encoded in AI systems". This has wider implications and tacit-meanings that I think I picked up on and definitely implies doing more than just having an equal sampling of each racial group. How that would be enacted, I don't know. The point is to not swing the pendulum the other way in order to "fix things" as it'll cause problems down the line, but to rather just "fix things" in the most non-intrusive and fair way.
- igorkraw 7y ago
  I'm glad you feel this way. So do you agree that it is therefore unacceptable to use such a system even if one accepts the surveillance state that comes with them since it demonstrably disadvantages minorities and making them "fair" would only make things worse? Or is this concern more selective?
  - zo1 7y ago
    I'm pretty libertarian, so I lean to simple solutions and my view on "fairness" is very non-complicated. I.e. Yes there may be bias in this technology we use, however, I would say that the important part we should really be focusing on is "how" that tech gets used, not the fact that it isn't perfect in some "unfair" kind of way. All sorts of things around us are unfair to all sorts of sub-groups of people that I don't think we should be going down this path and mixing government intervention in order to resolve it. Yes, it's kinda unfair on some level that the algorithms used don't work so well on women/blacks/children/elderly as well as "white middle-aged men", but it's not unfair in the "government should do something about it" kind of way just yet. If someone gets misidentified, that doesn't mean police should just go out and "arrest" them. Rather we add a person into the mix and have them talk to that person, make a final call. That is actionable counter-case data that we can use to resolve the bias in the training sets. A lot of facial-recognition systems include human components in order to resolve errors. E.g. I've worked with facial-recognition software here in South Africa that was developed by Asian companies where the majority of their data-sets and tests were using Asian individuals, so one has to manage error risk using humans in the loop.
    - beardedwizard 7y ago
      How are you managing human bias in this model?
      - zo1 7y ago
        Depends on what kind of bias you want/need to manage? Corruption/bribery is a big risk factor in SA so that is usually the one most people worry about, rather than racial bias. One of the things you have to look at is "how" that bias may manifest and if bias can even be applied in any negative sense in the realm of control you give to the user. If you give someone a task where they are to compare two faces, and they don't like the demographic of that individual, I suppose they might pick incorrectly (because you only gave them true/false type options). But you can build in review processes, reporting, performance feedback, etc to mitigate that. You need not treat it as bias at all, merely bad performance/mistakes that need correcting.
dcow 7y ago
Because now the system is only going to perform at peak accuracy in some fantasy world split up into 6 races of equal part man, woman, child, and elderly. If you’re seeking a realistic model, you randomly sample actual data. There are many problems with socially engineering data sets. For one we would need to achieve consensus on what constitutes an appropriately representative (i.e. socially just) data set. And then we need a feedback loop to make sure that our dynamically changing inputs don’t knock our meticulously tuned output out of whack. And we’d need to continually adjust our model based on our changing definition of social justice. Finally we need to find social engineers with credentials in building perfect societies (I most certainly don’t trust Joe PM) to perform all this work. Sociologists have been arguing forever that humans are inherently racist. Maybe these “AI systems” are just proving their point. Perhaps this bias is more a reflection of our inability to maintain global context that is also maximally relevant within our local tribes. Maybe we should focus on acknowledging biases and working through them rather than trying to transcend into some unicorn world where everybody has all the context all the time and bias is impossible. Anyway, I see your census and raise you an FBI: white people commit way more crimes in the US [1]. So if your goal is to reduce total harm then social engineering is not your solution. Perhaps more accurately identifying the most frequent perpetrators (in the case of the US, white people) is exactly how you’d end up tuning your system... 1: https://ucr.fbi.gov/crime-in-the-u.s/2010/crime-in-the-u.s.-2010/tables/table-43/10tbl43a.xls https://ucr.fbi.gov/crime-in-the-u.s/2010/crime-in-the-u.s.-...
- steveeq1 7y ago
  > white people commit way more crimes in the US How is this even possible? Aren't black responsible for more than 50% of the murders despite being only 13% of the population? These stats don't add up.
  - basseq 7y ago
    Did you click on the link? It literally shows you how these stats add up.
    - steveeq1 7y ago
_-o-_ 7y ago
This attitude is one of the many reasons why we can't have nice things. Instead of trying to elevate everyone to best possible standing you want to reduce all to the worst one.
AnthonyMouse 7y ago
> Why not change the training data to reflect an equal amount of the population? Officially the US has six “races” according to the Census Bureau. So, another option is to make each race ~16% of the training data. The difficulty with this is that you commonly don't have any control over the training data. Someone else collected it and you can use it but you can't choose what's in it. And in some cases you already have census-level data, which means you couldn't even increase the number of people with lower representation in the data set by collecting more data, because there are no such people not already in the data set. The only way to have equal representation would be to throw out data you do have on the groups with higher representation. Which is obviously objectionable because we need to be improving accuracy for the group with lower accuracy (if we can), not reducing accuracy for the group with greater accuracy just to make the numbers the same.

zo1 7y ago

>"Some might object to the decreased accuracy this gives whites — but if we’re going to have any hope of preventing existing systemic inequalities from being encoded in AI systems, we’ll have to socially engineer the data sets. " So to fix systemic inequalities for some groups, we're going to increase systemic inequality for certain other groups? Definitely a step in the wrong direction as it's just going to swing hatred back the other way. And N-levels later of this "fiddling" you'll end up in a spot where neither side can reconcile with the other because each side has legitimate reasons to feel slighted and no one can look past that "hatred". Just look at the mess that is currently in the middle-east after people "fiddled", and then "fiddled" again to try fix it.

pessimizer 7y ago
> So to fix systemic inequalities for some groups, we're going to increase systemic inequality for certain other groups? No; in order to accurately identify people through appearances, we'll try to sample evenly throughout the range of appearances and features instead of sampling based on proportions within the population of the people setting up the system. You're can't be seriously making the case that white people are such an overwhelming portion of the world's population that not concentrating on them in your training data constitutes racial discrimination. Not only racial discrimination even, but is a attack that will inspire a "legitimate" counter-retaliation.
- zo1 7y ago
  No that's not what I'm saying. If you look at the quote I mentioned, the person was suggesting we "socially engineer" data sets in order to "preventing existing systemic inequalities from being encoded in AI systems". This has wider implications and tacit-meanings that I think I picked up on and definitely implies doing more than just having an equal sampling of each racial group. How that would be enacted, I don't know. The point is to not swing the pendulum the other way in order to "fix things" as it'll cause problems down the line, but to rather just "fix things" in the most non-intrusive and fair way.
igorkraw 7y ago
I'm glad you feel this way. So do you agree that it is therefore unacceptable to use such a system even if one accepts the surveillance state that comes with them since it demonstrably disadvantages minorities and making them "fair" would only make things worse? Or is this concern more selective?
- zo1 7y ago
  I'm pretty libertarian, so I lean to simple solutions and my view on "fairness" is very non-complicated. I.e. Yes there may be bias in this technology we use, however, I would say that the important part we should really be focusing on is "how" that tech gets used, not the fact that it isn't perfect in some "unfair" kind of way. All sorts of things around us are unfair to all sorts of sub-groups of people that I don't think we should be going down this path and mixing government intervention in order to resolve it. Yes, it's kinda unfair on some level that the algorithms used don't work so well on women/blacks/children/elderly as well as "white middle-aged men", but it's not unfair in the "government should do something about it" kind of way just yet. If someone gets misidentified, that doesn't mean police should just go out and "arrest" them. Rather we add a person into the mix and have them talk to that person, make a final call. That is actionable counter-case data that we can use to resolve the bias in the training sets. A lot of facial-recognition systems include human components in order to resolve errors. E.g. I've worked with facial-recognition software here in South Africa that was developed by Asian companies where the majority of their data-sets and tests were using Asian individuals, so one has to manage error risk using humans in the loop.
  - beardedwizard 7y ago
    How are you managing human bias in this model?
    - zo1 7y ago
      Depends on what kind of bias you want/need to manage? Corruption/bribery is a big risk factor in SA so that is usually the one most people worry about, rather than racial bias. One of the things you have to look at is "how" that bias may manifest and if bias can even be applied in any negative sense in the realm of control you give to the user. If you give someone a task where they are to compare two faces, and they don't like the demographic of that individual, I suppose they might pick incorrectly (because you only gave them true/false type options). But you can build in review processes, reporting, performance feedback, etc to mitigate that. You need not treat it as bias at all, merely bad performance/mistakes that need correcting.

dcow 7y ago

Because now the system is only going to perform at peak accuracy in some fantasy world split up into 6 races of equal part man, woman, child, and elderly. If you’re seeking a realistic model, you randomly sample actual data. There are many problems with socially engineering data sets. For one we would need to achieve consensus on what constitutes an appropriately representative (i.e. socially just) data set. And then we need a feedback loop to make sure that our dynamically changing inputs don’t knock our meticulously tuned output out of whack. And we’d need to continually adjust our model based on our changing definition of social justice. Finally we need to find social engineers with credentials in building perfect societies (I most certainly don’t trust Joe PM) to perform all this work. Sociologists have been arguing forever that humans are inherently racist. Maybe these “AI systems” are just proving their point. Perhaps this bias is more a reflection of our inability to maintain global context that is also maximally relevant within our local tribes. Maybe we should focus on acknowledging biases and working through them rather than trying to transcend into some unicorn world where everybody has all the context all the time and bias is impossible. Anyway, I see your census and raise you an FBI: white people commit way more crimes in the US [1]. So if your goal is to reduce total harm then social engineering is not your solution. Perhaps more accurately identifying the most frequent perpetrators (in the case of the US, white people) is exactly how you’d end up tuning your system... 1: https://ucr.fbi.gov/crime-in-the-u.s/2010/crime-in-the-u.s.-2010/tables/table-43/10tbl43a.xls https://ucr.fbi.gov/crime-in-the-u.s/2010/crime-in-the-u.s.-...

steveeq1 7y ago
> white people commit way more crimes in the US How is this even possible? Aren't black responsible for more than 50% of the murders despite being only 13% of the population? These stats don't add up.
- basseq 7y ago
  Did you click on the link? It literally shows you how these stats add up.
  - steveeq1 7y ago

_-o-_ 7y ago

This attitude is one of the many reasons why we can't have nice things. Instead of trying to elevate everyone to best possible standing you want to reduce all to the worst one.

AnthonyMouse 7y ago

> Why not change the training data to reflect an equal amount of the population? Officially the US has six “races” according to the Census Bureau. So, another option is to make each race ~16% of the training data. The difficulty with this is that you commonly don't have any control over the training data. Someone else collected it and you can use it but you can't choose what's in it. And in some cases you already have census-level data, which means you couldn't even increase the number of people with lower representation in the data set by collecting more data, because there are no such people not already in the data set. The only way to have equal representation would be to throw out data you do have on the groups with higher representation. Which is obviously objectionable because we need to be improving accuracy for the group with lower accuracy (if we can), not reducing accuracy for the group with greater accuracy just to make the numbers the same.