4 ms·

> Many other sciences have IRB and human factors approvals for experiments that involve or may involve humans. As far as I know, those are for experiments that

by defen 6y ago

> Many other sciences have IRB and human factors approvals for experiments that involve or may involve humans. As far as I know, those are for experiments that directly interact with humans as part of the experiment itself. They exist because of a long history of unethical researchers testing interventions (or lack thereof) on people without their knowledge or informed consent - e.g. Tuskegee Syphilis Experiment, MK-ULTRA, Stanford prison experiment (there are a lot). The closest equivalent in CS of directly-interacting experiments would be HCI research as you said. I could also see a strong argument being made for research that uses the creative or copyrighted output of a person without their consent - for example, their face as part of training facial recognition software, their written words as part of training GPT-3, their voice as part of training voice recognition software, etc. However the incident in question is really about a different kind of thing - it's asking researchers to speculate on the future ramifications of their research as it pertains to progressive ideals - in practice this means, "How could this negatively affect minorities or the environment?" These aren't inherently bad things to think about, but as you get further and further away from concrete applications of ML, it begins to look more and more like a religious ritual than something that is actually trying to address the stated problems.

joshuamorton 6y ago
> As far as I know, those are for experiments that directly interact with humans as part of the experiment itself. Or use certain kinds of data from humans. For example, a machine learning project on medical data (even anonymized medical data) probably needs an IRB approval to gain access to, say, a dataset of cancer imaging. A facial recognition model doesn't. What makes those fundamentally different (arguably the facial image is more identifying). > However the incident in question is really about a different kind of thing - it's asking researchers to speculate on the future ramifications of their research as it pertains to progressive ideals I see no actual support for this claim in any of the actual documentation on the broader impact statement or its use. The Neurips docs link to 4 blog posts and papers for inspiration/advice to a broader-impact-statemet-author[0][1][2][3][4]. None focus on minorities or the environment in the way you describe, unless you mean "environment" in the broadest sense of "society". [0]: https://nips.cc/Conferences/2020/PaperInformation/NeurIPS-FAQ https://nips.cc/Conferences/2020/PaperInformation/NeurIPS-FA... ("How should I write the Broader Impact section?"_ [1]: https://brenthecht.com/papers/FCADIscussions_NegativeImpactsPost_032918.pdf https://brenthecht.com/papers/FCADIscussions_NegativeImpacts... [2]: https://brenthecht.medium.com/suggestions-for-writing-neurips-2020-broader-impacts-statements-121da1b765bf https://brenthecht.medium.com/suggestions-for-writing-neurip... [3]: https://medium.com/@GovAI/a-guide-to-writing-the-neurips-impact-statement-4293b723f832 https://medium.com/@GovAI/a-guide-to-writing-the-neurips-imp... [4]: https://arxiv.org/pdf/1912.06979.pdf https://arxiv.org/pdf/1912.06979.pdf
- defen 6y ago
  It's in link number [3] that you posted in the section called "C — Types of societal impact"
  - joshuamorton 6y ago
    I'm still a bit confused, yes, those are two possible concerns. I didn't deny that. You seem to imply that there's some kind of purity test here, and I'm missing the kind of logical connection between the 7 potential categories of broader impacts listed in that list, each with multiple subcategories, and your claim >it's asking researchers speculate on the future ramifications of their research as it pertains to progressive ideals - in practice this means, "How could this negatively affect minorities or the environment?" The framing appears to be far broader than just that, and the "ritualistic" concern you note appears to be unsubstantiated given the other areas for broad impact, and in practice given that for a lot of concrete ML applications, the broader impact statements appeared to be mostly focused on things like efficiency (or just platitudinal), as noted by other commenters.
    - defen 6y ago
      > in practice given that for a lot of concrete ML applications, the broader impact statements appeared to be mostly focused on things like efficiency (or just platitudinal), as noted by other commenters. If they're just platitudes or remarks about efficiency, why are some people so strongly opposed to including them, and some people so strongly in favor of including them? Basically, why does anyone care?
      - joshuamorton 6y ago
        As I understand it, the idea was that neurips said "hey lets try adding them to everything, and see what happens". And a lot of people gave a huge amount of backlash to this experiment for, as far as I can tell, reasons that amount to this being a form of liberal/leftist censorship. We're talking about, in this case, at least one person who is so scared of "wokism" that after Neurips changed its name from Nips to Neurips, he advocated for them to change it back, to, uhh, stick it to woke people I think?
mcguire 6y ago
Some might suggest those speculations might have been worthwhile for, say, nuclear weapons research. Personally, I don't; I've acclimated myself to the idea that anything that can be done, will be done. But it is something to consider when your favorite dystopia appears immanent.
zeckalpha 6y ago
That threshold does not stand, for example herbicides don’t interact directly with people but the chemists have to worry about the ethics of their work.
dekhn 6y ago
I don't think that's why IRBs exist. IRBs exist to shield research organizations from risk. That risk can come in many kinds: the research may be unethical, or would have bad PR, or could be dangerous, but the goal of the IRB is to make a determination that protects the interests of the research organization. Think of it like the HR department- it may pitch itself as a resource for line employees, but it exists to prevent the company from doing things that would get it into trouble.

by defen 6y ago

joshuamorton 6y ago

> As far as I know, those are for experiments that directly interact with humans as part of the experiment itself. Or use certain kinds of data from humans. For example, a machine learning project on medical data (even anonymized medical data) probably needs an IRB approval to gain access to, say, a dataset of cancer imaging. A facial recognition model doesn't. What makes those fundamentally different (arguably the facial image is more identifying). > However the incident in question is really about a different kind of thing - it's asking researchers to speculate on the future ramifications of their research as it pertains to progressive ideals I see no actual support for this claim in any of the actual documentation on the broader impact statement or its use. The Neurips docs link to 4 blog posts and papers for inspiration/advice to a broader-impact-statemet-author[0][1][2][3][4]. None focus on minorities or the environment in the way you describe, unless you mean "environment" in the broadest sense of "society". [0]: https://nips.cc/Conferences/2020/PaperInformation/NeurIPS-FAQ https://nips.cc/Conferences/2020/PaperInformation/NeurIPS-FA... ("How should I write the Broader Impact section?"_ [1]: https://brenthecht.com/papers/FCADIscussions_NegativeImpactsPost_032918.pdf https://brenthecht.com/papers/FCADIscussions_NegativeImpacts... [2]: https://brenthecht.medium.com/suggestions-for-writing-neurips-2020-broader-impacts-statements-121da1b765bf https://brenthecht.medium.com/suggestions-for-writing-neurip... [3]: https://medium.com/@GovAI/a-guide-to-writing-the-neurips-impact-statement-4293b723f832 https://medium.com/@GovAI/a-guide-to-writing-the-neurips-imp... [4]: https://arxiv.org/pdf/1912.06979.pdf https://arxiv.org/pdf/1912.06979.pdf

defen 6y ago
It's in link number [3] that you posted in the section called "C — Types of societal impact"
- joshuamorton 6y ago
  I'm still a bit confused, yes, those are two possible concerns. I didn't deny that. You seem to imply that there's some kind of purity test here, and I'm missing the kind of logical connection between the 7 potential categories of broader impacts listed in that list, each with multiple subcategories, and your claim >it's asking researchers speculate on the future ramifications of their research as it pertains to progressive ideals - in practice this means, "How could this negatively affect minorities or the environment?" The framing appears to be far broader than just that, and the "ritualistic" concern you note appears to be unsubstantiated given the other areas for broad impact, and in practice given that for a lot of concrete ML applications, the broader impact statements appeared to be mostly focused on things like efficiency (or just platitudinal), as noted by other commenters.
  - defen 6y ago
    > in practice given that for a lot of concrete ML applications, the broader impact statements appeared to be mostly focused on things like efficiency (or just platitudinal), as noted by other commenters. If they're just platitudes or remarks about efficiency, why are some people so strongly opposed to including them, and some people so strongly in favor of including them? Basically, why does anyone care?
    - joshuamorton 6y ago
      As I understand it, the idea was that neurips said "hey lets try adding them to everything, and see what happens". And a lot of people gave a huge amount of backlash to this experiment for, as far as I can tell, reasons that amount to this being a form of liberal/leftist censorship. We're talking about, in this case, at least one person who is so scared of "wokism" that after Neurips changed its name from Nips to Neurips, he advocated for them to change it back, to, uhh, stick it to woke people I think?

mcguire 6y ago

Some might suggest those speculations might have been worthwhile for, say, nuclear weapons research. Personally, I don't; I've acclimated myself to the idea that anything that can be done, will be done. But it is something to consider when your favorite dystopia appears immanent.

zeckalpha 6y ago

That threshold does not stand, for example herbicides don’t interact directly with people but the chemists have to worry about the ethics of their work.

dekhn 6y ago

I don't think that's why IRBs exist. IRBs exist to shield research organizations from risk. That risk can come in many kinds: the research may be unethical, or would have bad PR, or could be dangerous, but the goal of the IRB is to make a determination that protects the interests of the research organization. Think of it like the HR department- it may pitch itself as a resource for line employees, but it exists to prevent the company from doing things that would get it into trouble.