4 ms·

> Daniel Ting, a clinician AI scientist at Duke–NUS Medical School in Singapore, agrees that probing the system for biases is essential to making sure that the

> Daniel Ting, a clinician AI scientist at Duke–NUS Medical School in Singapore, agrees that probing the system for biases is essential to making sure that the algorithm doesn’t penalize racial groups that are not well represented in the training data sets. What? What kind of "bias" would this entail? Or is this more of the "ethics" garbage that is used to lobotomize LLMs? The real scientific-technical problem and discussion about aligment (e.g. paperclip maker) has been replaced by some strange grift that opens doors for people without any useful knowledge into the New Big Thing where the money is.

deleted 3y ago
[deleted]
ceejayoz 3y ago
> What kind of "bias" would this entail? Something like half of medical students still falsely believe black people feel less pain than white people: https://www.nytimes.com/interactive/2019/08/14/magazine/racial-differences-doctors.html https://www.nytimes.com/interactive/2019/08/14/magazine/raci... Some sensors don't measure accurately on darker skin: https://www.chop.edu/news/oxygen-readings-may-be-affected-darker-skin-tones https://www.chop.edu/news/oxygen-readings-may-be-affected-da... etc. etc. etc.
ameister14 3y ago
If you are a member of a racial subgroup with a heightened risk of hypercholesterolemia, for example, but the training data set did not have sufficient representation of your subgroup, they may not request tests for this and will ignore what is actually the most likely diagnosis in favor of testing for diagnoses that more closely match other more dominant racial groups without your genetic markers.
- margorczynski 3y ago
  And how this is an "ethics" issue? It is simply bad data which should be corrected according to the goal of the system (giving a proper diagnosis). If I construct a ML system to classify animals into kingdoms but my data is lacking e.g. cats so it skews the results away from the goal is that an ethical issue?
  - ceejayoz 3y ago
    > And how this is an "ethics" issue? It is simply bad data which should be corrected according to the goal of the system (giving a proper diagnosis). You must correct the bad data before using it for actual patient care to avoid an ethics issue. This means we must consider the potential ethics issues of its use prior to implementation, right? > If I construct a ML system to classify animals into kingdoms but my data is lacking e.g. cats so it skews the results away from the goal is that an ethical issue? If you wind up mistreating cats as a result, sure.
    - margorczynski 3y ago
      Again, I don't see anything in what you've said as an ethics issue. Ethics issues are about choice - here it is a matter of mistake and bad data.
      - ceejayoz 3y ago
        The choice to use a flawed model to care for patients is an ethical question, one we are increasingly likely to be confronted with in the next few years.
  - ameister14 3y ago
    It's theoretically a medical ethics issue because the system is not ready for deployment, so early use could be counter to competent medical care or the standards of professionalism, both of which would be violations of the AMA principles of medical ethics.
- dekhn 3y ago
  A better term than "racial" in this case is "genetic history" as self-identified race only partly correlates with genetic history.
selimthegrim 3y ago
Isn’t there an issue with them detecting self-reported race when no visible correlates are present in the data

deleted 3y ago

[deleted]

ceejayoz 3y ago

> What kind of "bias" would this entail? Something like half of medical students still falsely believe black people feel less pain than white people: https://www.nytimes.com/interactive/2019/08/14/magazine/racial-differences-doctors.html https://www.nytimes.com/interactive/2019/08/14/magazine/raci... Some sensors don't measure accurately on darker skin: https://www.chop.edu/news/oxygen-readings-may-be-affected-darker-skin-tones https://www.chop.edu/news/oxygen-readings-may-be-affected-da... etc. etc. etc.

ameister14 3y ago

If you are a member of a racial subgroup with a heightened risk of hypercholesterolemia, for example, but the training data set did not have sufficient representation of your subgroup, they may not request tests for this and will ignore what is actually the most likely diagnosis in favor of testing for diagnoses that more closely match other more dominant racial groups without your genetic markers.

margorczynski 3y ago
And how this is an "ethics" issue? It is simply bad data which should be corrected according to the goal of the system (giving a proper diagnosis). If I construct a ML system to classify animals into kingdoms but my data is lacking e.g. cats so it skews the results away from the goal is that an ethical issue?
- ceejayoz 3y ago
  > And how this is an "ethics" issue? It is simply bad data which should be corrected according to the goal of the system (giving a proper diagnosis). You must correct the bad data before using it for actual patient care to avoid an ethics issue. This means we must consider the potential ethics issues of its use prior to implementation, right? > If I construct a ML system to classify animals into kingdoms but my data is lacking e.g. cats so it skews the results away from the goal is that an ethical issue? If you wind up mistreating cats as a result, sure.
  - margorczynski 3y ago
    Again, I don't see anything in what you've said as an ethics issue. Ethics issues are about choice - here it is a matter of mistake and bad data.
    - ceejayoz 3y ago
      The choice to use a flawed model to care for patients is an ethical question, one we are increasingly likely to be confronted with in the next few years.
- ameister14 3y ago
  It's theoretically a medical ethics issue because the system is not ready for deployment, so early use could be counter to competent medical care or the standards of professionalism, both of which would be violations of the AMA principles of medical ethics.
dekhn 3y ago
A better term than "racial" in this case is "genetic history" as self-identified race only partly correlates with genetic history.

selimthegrim 3y ago

Isn’t there an issue with them detecting self-reported race when no visible correlates are present in the data