5 ms·

Am I understanding this right? They purged an entire dataset because one of its independent variables is how integrated a neighborhood is?

version_five 4y ago
Yes. It's an embarrassment.
mcs5280 4y ago
If we selectively hide things from view all the world's problems will go away
- mistrial9 4y ago
  .. with AI
- JeremyBanks 4y ago
  My Rights to be included as example data in a tool distribution
- mertd 4y ago
  Read the code? You can try to import it but you'll get a message explaining the problems with it and a link to where to find it shall you decide you'd like to use it anyway. It's the opposite of hiding.
  - ipaddr 4y ago
    Putting unnecessarily gates in front making the process non-standard is not the opposite of hiding.. perhaps somewhere inbetween
    - laidoffamazon 4y ago
      They suggest an alternative of the same type without these issues.
- tshaddox 4y ago
  This is a very small collection of toy datasets that this machine learning library includes for the purpose of illustrating the behavior of the library: http://scikit-learn.org/stable/datasets/toy_dataset.html http://scikit-learn.org/stable/datasets/toy_dataset.html It’s out of the scope of this library to publish all datasets in existence or highlight particular datasets that are relevant to particular societal problems. It’s literally just a few datasets so that you can play around with the ML library without downloading any external datasets. I think it’s fair to allow them to exercise reasonable discretion in their choice of which toy datasets to ship with their ML library.
ffssffss 4y ago
What's the point of such an incendiary comment? No, you aren't understanding it right. At worst you are offering a deliberately misleading interpretation. Here's what the link says: The Boston housing prices dataset has an ethical problem: as investigated in [1], the authors of this dataset engineered a non-invertible variable "B" assuming that racial self-segregation had a positive impact on house prices [2]. Furthermore the goal of the research that led to the creation of this dataset was to study the impact of air quality but it did not give adequate demonstration of the validity of this assumption. The scikit-learn maintainers therefore strongly discourage the use of this dataset unless the purpose of the code is to study and educate about ethical issues in data science and machine learning.
- josephcsible 4y ago
  The "B" variable measures how integrated a neighborhood is, and that snippet seems to be saying that its existence is the "ethical problem" that led them to purge the dataset. How is any of that different than what I said?
  - anigbrowl 4y ago
    [flagged]
    - trinsic2 4y ago
      Stop what?
      - anigbrowl 4y ago
        Expressing incredulity over one piece of context without acknowledging the existence of other parts which modify it. There was a full detailed explanation at the medium blog, and all that was necessary to do was follow the citation given and read it. Repeatedly proclaiming confusion when already in possession of the explanatory material, rather than challenging that explanation, is not a credible posture.
threeseed 4y ago
No. The dataset [1] defines B as: 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town And not sure how anyone can argue the dataset is worthy of being included. It is pretty offensive and misguided at minimum to argue that having more black people in your neighbourhood will depress housing prices. And for it to be solely because they are black and not to do with a range of other factors e.g. socio-economic. [1] http://lib.stat.cmu.edu/datasets/boston http://lib.stat.cmu.edu/datasets/boston
- josephcsible 4y ago
  Isn't that a measure of how integrated a neighborhood is? And even accepting for the sake of argument that such a variable is evil, why not just exclude it instead of ditching the whole dataset?
  - deleted 4y ago
    [deleted]
  - threeseed 4y ago
    a) No. If 50% of the community is African-Americans and live on one half and everyone else on the other then it would be exactly integrated. Except of course it isn't. b) It is not scikit-learn's responsibility to alter third party datasets.
    - CoastalCoder 4y ago
      > a) No. If 50% of the community is African-Americans and live on one half and everyone else on the other then it would be exactly integrated. Except of course it isn't. IIUC, you're arguing that "whole-town" level aggregation is misleading. So if we get more granular, we could do it my neighborhood, street, building, apartment/car/shelter, bedroom, bed, bed @ time of day, etc. Any one of those aggregation levels could hide interesting distinctions that could be made if only the data were reported with even more granularity. So are you arguing against aggregation in general? Or just whole-town aggregation specifically?
- 4y ago
anigbrowl 4y ago
No, you are not. https://medium.com/@docintangible/racist-data-destruction-113e3eff54a8 https://medium.com/@docintangible/racist-data-destruction-11...
loeg 4y ago
Close, but not exactly. One of its variables is how far its integration differs from 63% Black, squared. I.e., you cannot distinguish a 73% black neighborhood from a 53% black neighborhood with this variable. It's a bizarre variable and I guess I could see purging the column or at least suggesting it not be used, but I don't really understand why you'd delete the rest of the (sample) dataset on this basis.
- josephcsible 4y ago
  > Close, but not exactly. One of its variables is how far its integration differs from 63% Black, squared. > I.e., you cannot distinguish a 73% black neighborhood from a 53% black neighborhood with this variable. Isn't the point of that operation that a 1% black neighborhood and a 99% black neighborhood are both less integrated than a 50% black neighborhood? If you didn't do something like squaring, then wouldn't at least one of the former incorrectly register as more integrated than the latter? > It's a bizarre variable and I guess I could see purging the column or at least suggesting it not be used, but I don't really understand why you'd delete the rest of the (sample) dataset on this basis. Agreed.
  - loeg 4y ago
    Yeah, I mean, I guess I'd suggest a raw Bk column to preserve the original data and maybe just absolute value of difference instead of square of difference.
mcguire 4y ago
From https://fairlearn.org/main/user_guide/datasets/boston_housing_data.html https://fairlearn.org/main/user_guide/datasets/boston_housin...: "Thus, any models trained using this data that do not take special care to process B will learn to use mathematically encoded racism as a factor in house price prediction."
- loeg 4y ago
  House price prediction for prices in 1970s Boston, yes, where housing prices almost certainly reflected racist preferences. That seems like a (potentially) accurate model? ML models could also learn that the correlation between B and price is negative (i.e., that integration improves house prices). But the critics of the dataset all suggest that B and price are positively correlated.
- space_fountain 4y ago
  A confusion I sometimes have is where we should start pulling the string to unravel systematic racism. On some levels it can seem like if the purpose of this dataset was to analyze the impact of these variables on the prices humans (who were and are racist) are willing to pay than if B had predictive power (because people were racist), than excluding it in general seems wrong. On the other hand applying that across society is part of how we got redlining and in general a lot of the systematic part of systematic racism. Should we sort of handicap our models of human behavior to exclude race (and try to exclude proxies the model can find). I think regardless removing this model seems right since it's A a toy, B dated, and C liable to not be seen with the care it needs to be, but say if I'm trying to calculate who will be the next famous actor, is it wrong to include variables to allow the model to pick up on how shallow and say fat phobic society is (or just plain racist again)

version_five 4y ago

Yes. It's an embarrassment.

mcs5280 4y ago

If we selectively hide things from view all the world's problems will go away

mistrial9 4y ago
.. with AI
JeremyBanks 4y ago
My Rights to be included as example data in a tool distribution
mertd 4y ago
Read the code? You can try to import it but you'll get a message explaining the problems with it and a link to where to find it shall you decide you'd like to use it anyway. It's the opposite of hiding.
- ipaddr 4y ago
  Putting unnecessarily gates in front making the process non-standard is not the opposite of hiding.. perhaps somewhere inbetween
  - laidoffamazon 4y ago
    They suggest an alternative of the same type without these issues.
tshaddox 4y ago
This is a very small collection of toy datasets that this machine learning library includes for the purpose of illustrating the behavior of the library: http://scikit-learn.org/stable/datasets/toy_dataset.html http://scikit-learn.org/stable/datasets/toy_dataset.html It’s out of the scope of this library to publish all datasets in existence or highlight particular datasets that are relevant to particular societal problems. It’s literally just a few datasets so that you can play around with the ML library without downloading any external datasets. I think it’s fair to allow them to exercise reasonable discretion in their choice of which toy datasets to ship with their ML library.

ffssffss 4y ago

What's the point of such an incendiary comment? No, you aren't understanding it right. At worst you are offering a deliberately misleading interpretation. Here's what the link says: The Boston housing prices dataset has an ethical problem: as investigated in [1], the authors of this dataset engineered a non-invertible variable "B" assuming that racial self-segregation had a positive impact on house prices [2]. Furthermore the goal of the research that led to the creation of this dataset was to study the impact of air quality but it did not give adequate demonstration of the validity of this assumption. The scikit-learn maintainers therefore strongly discourage the use of this dataset unless the purpose of the code is to study and educate about ethical issues in data science and machine learning.

josephcsible 4y ago
The "B" variable measures how integrated a neighborhood is, and that snippet seems to be saying that its existence is the "ethical problem" that led them to purge the dataset. How is any of that different than what I said?
- anigbrowl 4y ago
  [flagged]
  - trinsic2 4y ago
    Stop what?
    - anigbrowl 4y ago
      Expressing incredulity over one piece of context without acknowledging the existence of other parts which modify it. There was a full detailed explanation at the medium blog, and all that was necessary to do was follow the citation given and read it. Repeatedly proclaiming confusion when already in possession of the explanatory material, rather than challenging that explanation, is not a credible posture.

threeseed 4y ago

No. The dataset [1] defines B as: 1000(Bk - 0.63)^2 where Bk is the proportion of blacks by town And not sure how anyone can argue the dataset is worthy of being included. It is pretty offensive and misguided at minimum to argue that having more black people in your neighbourhood will depress housing prices. And for it to be solely because they are black and not to do with a range of other factors e.g. socio-economic. [1] http://lib.stat.cmu.edu/datasets/boston http://lib.stat.cmu.edu/datasets/boston

josephcsible 4y ago
Isn't that a measure of how integrated a neighborhood is? And even accepting for the sake of argument that such a variable is evil, why not just exclude it instead of ditching the whole dataset?
- deleted 4y ago
  [deleted]
- threeseed 4y ago
  a) No. If 50% of the community is African-Americans and live on one half and everyone else on the other then it would be exactly integrated. Except of course it isn't. b) It is not scikit-learn's responsibility to alter third party datasets.
  - CoastalCoder 4y ago
    > a) No. If 50% of the community is African-Americans and live on one half and everyone else on the other then it would be exactly integrated. Except of course it isn't. IIUC, you're arguing that "whole-town" level aggregation is misleading. So if we get more granular, we could do it my neighborhood, street, building, apartment/car/shelter, bedroom, bed, bed @ time of day, etc. Any one of those aggregation levels could hide interesting distinctions that could be made if only the data were reported with even more granularity. So are you arguing against aggregation in general? Or just whole-town aggregation specifically?
4y ago

anigbrowl 4y ago

No, you are not. https://medium.com/@docintangible/racist-data-destruction-113e3eff54a8 https://medium.com/@docintangible/racist-data-destruction-11...

loeg 4y ago

Close, but not exactly. One of its variables is how far its integration differs from 63% Black, squared. I.e., you cannot distinguish a 73% black neighborhood from a 53% black neighborhood with this variable. It's a bizarre variable and I guess I could see purging the column or at least suggesting it not be used, but I don't really understand why you'd delete the rest of the (sample) dataset on this basis.

josephcsible 4y ago
> Close, but not exactly. One of its variables is how far its integration differs from 63% Black, squared. > I.e., you cannot distinguish a 73% black neighborhood from a 53% black neighborhood with this variable. Isn't the point of that operation that a 1% black neighborhood and a 99% black neighborhood are both less integrated than a 50% black neighborhood? If you didn't do something like squaring, then wouldn't at least one of the former incorrectly register as more integrated than the latter? > It's a bizarre variable and I guess I could see purging the column or at least suggesting it not be used, but I don't really understand why you'd delete the rest of the (sample) dataset on this basis. Agreed.
- loeg 4y ago
  Yeah, I mean, I guess I'd suggest a raw Bk column to preserve the original data and maybe just absolute value of difference instead of square of difference.

mcguire 4y ago

From https://fairlearn.org/main/user_guide/datasets/boston_housing_data.html https://fairlearn.org/main/user_guide/datasets/boston_housin...: "Thus, any models trained using this data that do not take special care to process B will learn to use mathematically encoded racism as a factor in house price prediction."

loeg 4y ago
House price prediction for prices in 1970s Boston, yes, where housing prices almost certainly reflected racist preferences. That seems like a (potentially) accurate model? ML models could also learn that the correlation between B and price is negative (i.e., that integration improves house prices). But the critics of the dataset all suggest that B and price are positively correlated.
space_fountain 4y ago
A confusion I sometimes have is where we should start pulling the string to unravel systematic racism. On some levels it can seem like if the purpose of this dataset was to analyze the impact of these variables on the prices humans (who were and are racist) are willing to pay than if B had predictive power (because people were racist), than excluding it in general seems wrong. On the other hand applying that across society is part of how we got redlining and in general a lot of the systematic part of systematic racism. Should we sort of handicap our models of human behavior to exclude race (and try to exclude proxies the model can find). I think regardless removing this model seems right since it's A a toy, B dated, and C liable to not be seen with the care it needs to be, but say if I'm trying to calculate who will be the next famous actor, is it wrong to include variables to allow the model to pick up on how shallow and say fat phobic society is (or just plain racist again)