6 ms·

This isn't about overfitting. Race is almost exactly the opposite problem: the signal in the data is so strong that you can't help but pick it up, even after y

This isn't about overfitting. Race is almost exactly the opposite problem: the signal in the data is so strong that you can't help but pick it up, even after you censor everything you can think of. If you try to predict any kind of consumer behavior from a racially heterogeneous dataset you will end up finding something that correlates with race, because practically everything correlates with race.

mistermann 8y ago
Is this actually true, that everything correlates with race? I know many things do, but does it even exceed 50%?
- deleted 8y ago
  [deleted]
- mistermann 8y ago
  Hey mods....if you happen to monitor those of us with warnings, I wonder if you can run a query to determine my top 3 downvoters over time and see if a pattern emerges. This is starting to get rather annoying.
paulsutter 8y ago
> practically everything correlates with race. Hold on, excuse me?
- fossuser 8y ago
  It's easy for ML to pick up on proxies that are related to race even if you exclude what the race is explicitly - zip code and names are two common ones, but this can appear in a lot of different ways. It's particularly insidious too since a model that accidentally is making credit decisions based on race helps to perpetuate the inequity in the future data.
- deleted 8y ago
  [deleted]
- mlevental 8y ago
  lol. what this person is basically saying is that racism affects almost every part of a person's life. I know it seems unbelievable but "practically everything correlates with race" is the most nerd-intelligible way I've seen of explaining/expressing that racism is pervasive.
  - lambdaphagy 8y ago
    That's not what I'm saying.
    - mlevental 8y ago
      fine you're not actually saying it (though I think you should be) but the direct implication of what you're saying is that.
      - echotango 8y ago
        He is saying that race is correlated with almost everything, not racism. You seem to be claiming all statistical differences between racial groups are a result of racism. But black people for instance aren't more likey to listen to rap music because of racism.
        mlevental 8y ago
        >He is saying that race is correlated with almost everything, not racism. and i didn't say that he said that racism is correlated with anything.
    - CardenB 8y ago
      I can’t really interpret your comment any different way.
    - dang 8y ago
      You've been using this account primarily for political and ideological battle. That's destructive of what HN is for—regardless of your ideology—and we ban accounts that do it, as explained here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. If you want to keep commenting here, please (re-)read the guidelines and use this site as intended from now on. The intention is intellectual curiosity, and that is the first casualty of ideological war.

mistermann 8y ago

Is this actually true, that everything correlates with race? I know many things do, but does it even exceed 50%?

deleted 8y ago
[deleted]
mistermann 8y ago
Hey mods....if you happen to monitor those of us with warnings, I wonder if you can run a query to determine my top 3 downvoters over time and see if a pattern emerges. This is starting to get rather annoying.

paulsutter 8y ago

> practically everything correlates with race. Hold on, excuse me?

fossuser 8y ago
It's easy for ML to pick up on proxies that are related to race even if you exclude what the race is explicitly - zip code and names are two common ones, but this can appear in a lot of different ways. It's particularly insidious too since a model that accidentally is making credit decisions based on race helps to perpetuate the inequity in the future data.
deleted 8y ago
[deleted]
mlevental 8y ago
lol. what this person is basically saying is that racism affects almost every part of a person's life. I know it seems unbelievable but "practically everything correlates with race" is the most nerd-intelligible way I've seen of explaining/expressing that racism is pervasive.
- lambdaphagy 8y ago
  That's not what I'm saying.
  - mlevental 8y ago
    fine you're not actually saying it (though I think you should be) but the direct implication of what you're saying is that.
    - echotango 8y ago
      He is saying that race is correlated with almost everything, not racism. You seem to be claiming all statistical differences between racial groups are a result of racism. But black people for instance aren't more likey to listen to rap music because of racism.
      - mlevental 8y ago
        >He is saying that race is correlated with almost everything, not racism. and i didn't say that he said that racism is correlated with anything.
  - CardenB 8y ago
    I can’t really interpret your comment any different way.
  - dang 8y ago
    You've been using this account primarily for political and ideological battle. That's destructive of what HN is for—regardless of your ideology—and we ban accounts that do it, as explained here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. If you want to keep commenting here, please (re-)read the guidelines and use this site as intended from now on. The intention is intellectual curiosity, and that is the first casualty of ideological war.