3 ms·

Perhaps an unpopular opinion, but isn't the model just representing the world as it is, rather than what we want it to be? I'm all for the diversification of v

Perhaps an unpopular opinion, but isn't the model just representing the world as it is, rather than what we want it to be? I'm all for the diversification of various fields traditionally dominated by a specific cohort. But are data models the right place for that activism? It feels wrong to me to make the data biased and different from reality, even if for a good cause. Where does it end? You won't know which results from the model are based on typical distributions in fact and which are due to intentional tweaks for one agenda or another.

majormajor 4y ago
The right place to start is probably at the pre-K level. https://press.princeton.edu/books/hardcover/9780691171807/can-college-level-the-playing-field https://press.princeton.edu/books/hardcover/9780691171807/ca... / book discussed well at https://crookedtimber.org/2023/02/06/can-college-level-the-playing-field-no-it-really-cant/ https://crookedtimber.org/2023/02/06/can-college-level-the-p... But most people feel powerless to do anything about that so ... the discussion instead lands in whatever they are familiar with.
dragonwriter 4y ago
> Perhaps an unpopular opinion, but isn't the model just representing the world as it is, rather than what we want it to be? Probably not, becayse its trainung data almost certainly has biases compared to “the world as it is”, and the model itself has some error with regard to the training data. It's representing one view of the world seen through multiple levels of filters each with their own set of (systematic or random) biases. (Which is not too dissimilar to any individual human, but any widely used model's biases are likely to have much wider impact than any single human’s.)
- colordrops 4y ago
  Sure, it absolutely makes sense to adjust models to account for biased inputs. Say for instance the pictures you train on are from tundra but you want to use the model in a jungle region. But you should use vetted stats to determine the nature of the landscape to feed them model (e.g. 95% of land is jungle, so feed 95% jungle images), rather than feeding it tundra images because you think the jungle is too hot.
  - dragonwriter 4y ago
    If its a model for art, I’ll absolutely feed it tundra images if I think the jungle is (or just looks) too hot. Of course, I’ll also embrace the projected cool vibe of the resulting imagery as a feature, and won’t object to people pointing out that the model is biased toward tundra over jungle and that is something users who aren’t actively looking for that bias might want to be aware of.

majormajor 4y ago

The right place to start is probably at the pre-K level. https://press.princeton.edu/books/hardcover/9780691171807/can-college-level-the-playing-field https://press.princeton.edu/books/hardcover/9780691171807/ca... / book discussed well at https://crookedtimber.org/2023/02/06/can-college-level-the-playing-field-no-it-really-cant/ https://crookedtimber.org/2023/02/06/can-college-level-the-p... But most people feel powerless to do anything about that so ... the discussion instead lands in whatever they are familiar with.

dragonwriter 4y ago

> Perhaps an unpopular opinion, but isn't the model just representing the world as it is, rather than what we want it to be? Probably not, becayse its trainung data almost certainly has biases compared to “the world as it is”, and the model itself has some error with regard to the training data. It's representing one view of the world seen through multiple levels of filters each with their own set of (systematic or random) biases. (Which is not too dissimilar to any individual human, but any widely used model's biases are likely to have much wider impact than any single human’s.)

colordrops 4y ago
Sure, it absolutely makes sense to adjust models to account for biased inputs. Say for instance the pictures you train on are from tundra but you want to use the model in a jungle region. But you should use vetted stats to determine the nature of the landscape to feed them model (e.g. 95% of land is jungle, so feed 95% jungle images), rather than feeding it tundra images because you think the jungle is too hot.
- dragonwriter 4y ago
  If its a model for art, I’ll absolutely feed it tundra images if I think the jungle is (or just looks) too hot. Of course, I’ll also embrace the projected cool vibe of the resulting imagery as a feature, and won’t object to people pointing out that the model is biased toward tundra over jungle and that is something users who aren’t actively looking for that bias might want to be aware of.