4 ms·

So you are saying any data for which any input feature is correlated with race or sex is garbage. That's going to make doing useful data science incredibly dif

So you are saying any data for which any input feature is correlated with race or sex is garbage. That's going to make doing useful data science incredibly difficult with anything involving humans.

hannasanarion 8y ago
It makes it exactly as difficult as the problem actually is. If you start with garbage data, such as arrest records (black people are WAY more likely to get arrested than white people in the same circumstance), then the bias in your sample is going to result in bias in your model.
viraptor 8y ago
To some extent. More specifically, if those become confounding variables, the result may be partially useless. That's why you have to make sure you control for race and sex if they're relevant. https://en.m.wikipedia.org/wiki/Confounding https://en.m.wikipedia.org/wiki/Confounding
Obi_Juan_Kenobi 8y ago
Strawman Sex is easy to incorporate. Race is more difficult because vanishingly little work has been done with a concept of race based on anything resembling real population genetics, but it's not impossible in theory. Hint: if you really care about human biodiversity, you'll spend most of your time in Africa and some isolated islands, not looking at brown people in the west. The issue is GxE, where we see loads of racists and sexists just patently forget about 'E' and conclude that women or brown people are genetically inferior. Some will try to gussy that up as 'different' rather than inferior, but the dog-whistles may as well be air-raid sirens. 'Data science' is fine if you're AB testing websites. When you try to do real science, you'll find that the utter lack of research experience is .. a bit of a problem. Caring about PhDs isn't credentialism, it's wanting a plumber that has worked with pipes before. You have to actually perform research to get decent at it. It usually takes about a decade before you can honestly do it independently. If you can look back at what you did a year ago without cringing, you're not making that progress. Some schmuck with PANDAS and a Bio101 class 8 years ago isn't a scientist.

So you are saying any data for which any input feature is correlated with race or sex is garbage. That's going to make doing useful data science incredibly difficult with anything involving humans.

hannasanarion 8y ago

It makes it exactly as difficult as the problem actually is. If you start with garbage data, such as arrest records (black people are WAY more likely to get arrested than white people in the same circumstance), then the bias in your sample is going to result in bias in your model.

viraptor 8y ago

To some extent. More specifically, if those become confounding variables, the result may be partially useless. That's why you have to make sure you control for race and sex if they're relevant. https://en.m.wikipedia.org/wiki/Confounding https://en.m.wikipedia.org/wiki/Confounding

Obi_Juan_Kenobi 8y ago

Strawman Sex is easy to incorporate. Race is more difficult because vanishingly little work has been done with a concept of race based on anything resembling real population genetics, but it's not impossible in theory. Hint: if you really care about human biodiversity, you'll spend most of your time in Africa and some isolated islands, not looking at brown people in the west. The issue is GxE, where we see loads of racists and sexists just patently forget about 'E' and conclude that women or brown people are genetically inferior. Some will try to gussy that up as 'different' rather than inferior, but the dog-whistles may as well be air-raid sirens. 'Data science' is fine if you're AB testing websites. When you try to do real science, you'll find that the utter lack of research experience is .. a bit of a problem. Caring about PhDs isn't credentialism, it's wanting a plumber that has worked with pipes before. You have to actually perform research to get decent at it. It usually takes about a decade before you can honestly do it independently. If you can look back at what you did a year ago without cringing, you're not making that progress. Some schmuck with PANDAS and a Bio101 class 8 years ago isn't a scientist.