5 ms·

Thanks for the additional detail, but those all still boil down to dataset eventually, right? Those sound more like "yes, and" additions than the "no, you're r

Thanks for the additional detail, but those all still boil down to dataset eventually, right? Those sound more like "yes, and" additions than the "no, you're racist" shitfight we saw.

renewiltord 6y ago
In the interest of explaining (not arguing), I believe the distinction is sort of that the community at large recognizes performance only on certain metrics. Let's set aside the notion of whether something or someone is "racist" because that's a bit charged. We'll instead just talk about whether we're creating generally capable systems. For instance, if I were to champion metric A which purports to measure performance on human faces but it really only rewards performance on Senegalese then models that do better in general may not be recognized for being better. In an isolated sense this is not a problem. However if the mainstream is that all metrics that are taken seriously are dataset-biased then we'll have an environment where the models will be trained on biased datasets in order to be successful. For instance if everyone uses LFW to determine how good facial recognition is, then Senegalese fine-tuned facial recognition tech will not be recognized as being good at facial recognition. So the argument is that dataset bias is built-in to our approach to the problem. I, personally, think that this isn't malice. We need benchmarks to judge approaches against each other. Benchmarks always have a first mover advantage and a massive path dependence issue. The first benchmarks aiming for things on humans do not reflect humanity accurately. These benchmarks became standard among the community. To be taken seriously you have to do well on benchmarks that are standard in the community. Dataset bias is then natural in newer approaches because the approaches are judged against how good they are against the inaccurate (if you will) benchmarks. So no one need be racist or anything for the end result of the field to end up being discriminatory. I don't think a successful approach is to call people racist over this. After all, it isn't malice that guides them. The discrimination comes from the sort of historical accident that has North facing up on a map. And no individual is really racist. It's sort of like the Bechdel test: no movie is crappy simply because it doesn't have two girls talking to each other, but if very few movies have two girls talking to each other about something other than boys, then it makes you think "hmmm, why's that the case".
- free_rms 6y ago
  Wouldn't it be a lot more productive to lobby for inclusion of more samples into the performance measurement datasets than to get into super-combative twitter pissfights with people who aren't even opposed to that? That's, like, not even culture war, it's just basic correctness of the reference datasets. If a fruit classifier was missing oranges, we'd just fix it and move on.
  - renewiltord 6y ago
    Haha, well, you know how it is. Everyone has their own opinion as to how to solve problems like this. Some people see their role as sort of fire alarms to say "Hey! This thing is happening!". You know, like how every thing has people at all stages in the process. Others might believe that this is one instance (the benchmarks being 'wrong') and that a better outcome would be that the people putting out this stuff are aware of ways that things can go wrong so they think about it in future (maybe this time it's these benchmarks, next time it's the other ones, and then maybe next time it's something inherent to the structure, whatever, it's a bit whack-a-mole to solve at the instance level). Like, for instance, HN has people who will bring up privacy violations of big tech constantly. They see their role as making sure the conversation is happening. Not justifying this. Just aiming to understand it. For my part, I prefer to take the approach you're talking about because I, too, think that the fastest path to this is getting the photos, labeling the photos, and then lobbying for inclusion. Ultimately, I think it's okay if things optimize fast for growth and then we fix up issues afterwards. So the people building the benchmark sets weren't able to get a set that's representative of humanity. Should they have waited till they could have done that? IMHO, no. Rapid release moves the state of the art forward and then we can put in all of these corrections as we move. Then there's the question of whether all-humans dataset is a good thing or if instead a thing that is white-humans and another that is black-humans is better. Anyway, all said, my personal approach to this problem (if I cared about it a lot, which I don't) would be to say "Current benchmarks and training data available bias towards certain races. I'd like to build X/Y to solve that. Here's what I have so far" etc. etc. I think positive engagement like that yields better results because the vast majority of scientists actually aren't weird race supremacists and the vast majority of AI researchers will gobble up any more data you give them which is segmented and labeled differently, the greedy bastards :D Some challenges that I can still see: * Getting the data. Might not actually exist. * Labeling the data. Probably needs some work. * Getting it into the benchmarks. It'll invalidate old scores, so there's just a product adoption problem here. I don't know how the community handles newer releases. As a last aside, I suspect that this conversation ended up the way it did because: a. It's charged. It's race-based differing outcomes. That's a sensitive subject. b. People feel unheard. This is natural. Like, this is not an 'interesting' problem. It's literally just a data error so the luminaries in the techniques part of the field aren't really that interested in it. And the techniques part is where the sexy is. c. This sort of thing has a tendency to escalate. One side says "You're not listening to what I say" and the other side says "I'm not racist. I don't get why you're calling me that" and before you know it it becomes "You have to be racist to be ignoring me" and whatnot and de-escalation becomes impossible. Especially because everyone rewards the loudest on each side. Honestly, I think it's quite interesting to observe and to understand as just a view into the human condition but we use AI models professionally in the GIS space and professionally we just don't go near this at all. No part of me finds it interesting to solve or to interact with the discussion in any way. I only sort of participated in this here because I think I managed some insight into what it is and I wanted to write that down because I wish someone else could have accelerated me into it. Anyway, I think that's all the insight I have on the subject, so I'm going to just leave it there. Any more and I'll be ass-pulling.

Thanks for the additional detail, but those all still boil down to dataset eventually, right? Those sound more like "yes, and" additions than the "no, you're racist" shitfight we saw.

renewiltord 6y ago

In the interest of explaining (not arguing), I believe the distinction is sort of that the community at large recognizes performance only on certain metrics. Let's set aside the notion of whether something or someone is "racist" because that's a bit charged. We'll instead just talk about whether we're creating generally capable systems. For instance, if I were to champion metric A which purports to measure performance on human faces but it really only rewards performance on Senegalese then models that do better in general may not be recognized for being better. In an isolated sense this is not a problem. However if the mainstream is that all metrics that are taken seriously are dataset-biased then we'll have an environment where the models will be trained on biased datasets in order to be successful. For instance if everyone uses LFW to determine how good facial recognition is, then Senegalese fine-tuned facial recognition tech will not be recognized as being good at facial recognition. So the argument is that dataset bias is built-in to our approach to the problem. I, personally, think that this isn't malice. We need benchmarks to judge approaches against each other. Benchmarks always have a first mover advantage and a massive path dependence issue. The first benchmarks aiming for things on humans do not reflect humanity accurately. These benchmarks became standard among the community. To be taken seriously you have to do well on benchmarks that are standard in the community. Dataset bias is then natural in newer approaches because the approaches are judged against how good they are against the inaccurate (if you will) benchmarks. So no one need be racist or anything for the end result of the field to end up being discriminatory. I don't think a successful approach is to call people racist over this. After all, it isn't malice that guides them. The discrimination comes from the sort of historical accident that has North facing up on a map. And no individual is really racist. It's sort of like the Bechdel test: no movie is crappy simply because it doesn't have two girls talking to each other, but if very few movies have two girls talking to each other about something other than boys, then it makes you think "hmmm, why's that the case".

free_rms 6y ago
Wouldn't it be a lot more productive to lobby for inclusion of more samples into the performance measurement datasets than to get into super-combative twitter pissfights with people who aren't even opposed to that? That's, like, not even culture war, it's just basic correctness of the reference datasets. If a fruit classifier was missing oranges, we'd just fix it and move on.
- renewiltord 6y ago
  Haha, well, you know how it is. Everyone has their own opinion as to how to solve problems like this. Some people see their role as sort of fire alarms to say "Hey! This thing is happening!". You know, like how every thing has people at all stages in the process. Others might believe that this is one instance (the benchmarks being 'wrong') and that a better outcome would be that the people putting out this stuff are aware of ways that things can go wrong so they think about it in future (maybe this time it's these benchmarks, next time it's the other ones, and then maybe next time it's something inherent to the structure, whatever, it's a bit whack-a-mole to solve at the instance level). Like, for instance, HN has people who will bring up privacy violations of big tech constantly. They see their role as making sure the conversation is happening. Not justifying this. Just aiming to understand it. For my part, I prefer to take the approach you're talking about because I, too, think that the fastest path to this is getting the photos, labeling the photos, and then lobbying for inclusion. Ultimately, I think it's okay if things optimize fast for growth and then we fix up issues afterwards. So the people building the benchmark sets weren't able to get a set that's representative of humanity. Should they have waited till they could have done that? IMHO, no. Rapid release moves the state of the art forward and then we can put in all of these corrections as we move. Then there's the question of whether all-humans dataset is a good thing or if instead a thing that is white-humans and another that is black-humans is better. Anyway, all said, my personal approach to this problem (if I cared about it a lot, which I don't) would be to say "Current benchmarks and training data available bias towards certain races. I'd like to build X/Y to solve that. Here's what I have so far" etc. etc. I think positive engagement like that yields better results because the vast majority of scientists actually aren't weird race supremacists and the vast majority of AI researchers will gobble up any more data you give them which is segmented and labeled differently, the greedy bastards :D Some challenges that I can still see: * Getting the data. Might not actually exist. * Labeling the data. Probably needs some work. * Getting it into the benchmarks. It'll invalidate old scores, so there's just a product adoption problem here. I don't know how the community handles newer releases. As a last aside, I suspect that this conversation ended up the way it did because: a. It's charged. It's race-based differing outcomes. That's a sensitive subject. b. People feel unheard. This is natural. Like, this is not an 'interesting' problem. It's literally just a data error so the luminaries in the techniques part of the field aren't really that interested in it. And the techniques part is where the sexy is. c. This sort of thing has a tendency to escalate. One side says "You're not listening to what I say" and the other side says "I'm not racist. I don't get why you're calling me that" and before you know it it becomes "You have to be racist to be ignoring me" and whatnot and de-escalation becomes impossible. Especially because everyone rewards the loudest on each side. Honestly, I think it's quite interesting to observe and to understand as just a view into the human condition but we use AI models professionally in the GIS space and professionally we just don't go near this at all. No part of me finds it interesting to solve or to interact with the discussion in any way. I only sort of participated in this here because I think I managed some insight into what it is and I wanted to write that down because I wish someone else could have accelerated me into it. Anyway, I think that's all the insight I have on the subject, so I'm going to just leave it there. Any more and I'll be ass-pulling.