28 ms·

If the output seems to strongly favor some races over others, the expert debugs the inputs by temporarily swapping some out and rerunning. This is only a second

If the output seems to strongly favor some races over others, the expert debugs the inputs by temporarily swapping some out and rerunning. This is only a second line of defense beyond checking the inputs manually. Maybe the expert doesn't realize that some factor has a huge correlation with race without saying much about the candidate's merit. Beyond that, if there's a divergence between races, it is what it is. The expert shares what factors are being considered, and it's politics at that point.

godelski 4y ago
> If the output seems to strongly favor some races over others, You'd typically do this with a cost function. > the expert debugs the inputs by temporarily swapping some out and rerunning. The best way to do this is to use that cost function to optimize your parameters. > Of course, could do this without race data just by proofreading. I thought I've adequately demonstrated why this is much easier said than done. > Beyond that, if there's a divergence between races, it is what it is. Except it isn't. It depends if the divergence is because you've trained a model on poisoned data (i.e. the historical racism gives momentum to your model). These things are rather tricky. It really feels like we're going in circles here and I'm trying to get you to understand why we do what we do, but you're avoiding getting specific to support your assumptions. I work with data all day every day. I'm sure you've done some data analysis too, but to do high quality work you have to consider a lot of factors. It's okay to not have this domain expertise. You're a general software engineer and probably far better than I am at that. But it is a bit naive to argue with domain experts about something that isn't in your domain. It would be naive for me to argue with you about your expertise. That's okay though. Our world is highly specialized. No one can know everything. But that's why I wrote so much before, because this is a toxic pattern in our culture that we need to curb.
- hot_gril 4y ago
  Is your domain expertise in college admissions, data science, or a combination? You assuming this model should try and undo some effects of historical racism, but that isn't the engineers' job to decide. If that's the intended behavior, I understand that the difficulties involved are beyond my experience, and I would leave it to the data scientists. That's not what I think the goal of college admissions should be, and our government is making a decision on that now. > The best way to do this is to use that cost function to optimize your parameters. > I thought I've adequately demonstrated why this is much easier said than done. I understand how this works, but that's specifically what I don't want, for race (or other data that serves no purpose but to indicate the race, like name) to be part of the training data. Those inputs are manually selected at some level.
  - godelski 4y ago
    My domain expertise is in data science. We're not talking about specifically school admissions but something bigger: handling race in data. That's decently general. Were I to actually create a model for predicting admissions I'd definitely be working with admissions to see those specific nuances. But you're attempting to pigeonhole the conversation. > I understand how this works, but that's specifically what I don't want, for race to be part of the training data. Tough. The problem, is that it is really difficult to move. That's what you aren't understanding. Making a model that doesn't explicitly incorporate race doesn't mean you've removed the race variable. It is just hidden now. There's plenty of examples of this happening in real life and they're fairly easy to Google. The historical data you train your model on doesn't suddenly lose it's correlation to race (there's major confounding variables) just because you didn't explicitly incorporate it. Dealing with hidden variables is hard and that's all you're doing.

godelski 4y ago

> If the output seems to strongly favor some races over others, You'd typically do this with a cost function. > the expert debugs the inputs by temporarily swapping some out and rerunning. The best way to do this is to use that cost function to optimize your parameters. > Of course, could do this without race data just by proofreading. I thought I've adequately demonstrated why this is much easier said than done. > Beyond that, if there's a divergence between races, it is what it is. Except it isn't. It depends if the divergence is because you've trained a model on poisoned data (i.e. the historical racism gives momentum to your model). These things are rather tricky. It really feels like we're going in circles here and I'm trying to get you to understand why we do what we do, but you're avoiding getting specific to support your assumptions. I work with data all day every day. I'm sure you've done some data analysis too, but to do high quality work you have to consider a lot of factors. It's okay to not have this domain expertise. You're a general software engineer and probably far better than I am at that. But it is a bit naive to argue with domain experts about something that isn't in your domain. It would be naive for me to argue with you about your expertise. That's okay though. Our world is highly specialized. No one can know everything. But that's why I wrote so much before, because this is a toxic pattern in our culture that we need to curb.

hot_gril 4y ago
Is your domain expertise in college admissions, data science, or a combination? You assuming this model should try and undo some effects of historical racism, but that isn't the engineers' job to decide. If that's the intended behavior, I understand that the difficulties involved are beyond my experience, and I would leave it to the data scientists. That's not what I think the goal of college admissions should be, and our government is making a decision on that now. > The best way to do this is to use that cost function to optimize your parameters. > I thought I've adequately demonstrated why this is much easier said than done. I understand how this works, but that's specifically what I don't want, for race (or other data that serves no purpose but to indicate the race, like name) to be part of the training data. Those inputs are manually selected at some level.
- godelski 4y ago
  My domain expertise is in data science. We're not talking about specifically school admissions but something bigger: handling race in data. That's decently general. Were I to actually create a model for predicting admissions I'd definitely be working with admissions to see those specific nuances. But you're attempting to pigeonhole the conversation. > I understand how this works, but that's specifically what I don't want, for race to be part of the training data. Tough. The problem, is that it is really difficult to move. That's what you aren't understanding. Making a model that doesn't explicitly incorporate race doesn't mean you've removed the race variable. It is just hidden now. There's plenty of examples of this happening in real life and they're fairly easy to Google. The historical data you train your model on doesn't suddenly lose it's correlation to race (there's major confounding variables) just because you didn't explicitly incorporate it. Dealing with hidden variables is hard and that's all you're doing.