6 ms·

Which is truly an exceptionally dumb position to take. Many other sciences have IRB and human factors approvals for experiments that involve or may involve huma

Which is truly an exceptionally dumb position to take. Many other sciences have IRB and human factors approvals for experiments that involve or may involve humans. Computer Science (and specifically ML, a lot of HCI research does involve human subjects approval) ignores all of the potential for harm here. We've seen that go terribly wrong in other fields, and in the ML field.

croissants 6y ago
I agree with the general conclusion that machine learning (just like any way of making decisions) involves ethics, but I disagree with the conclusion that every NeurIPS submission should have a concluding 1-2 paragraphs about the ethics of the submission. I reviewed for NeurIPS this year, and almost all of the ethics sections consisted of vacuous stuff along the lines of "uh, I guess...being efficient is good, might save some energy? and our method is pretty efficient, so that's nice". I think a system where area chairs skim abstracts and solicit these ethics paragraphs where they seem necessary -- so, not for the billionth paper obtaining pretty bounds for some variant of convex optimization -- makes more sense. I know that "think of the machine learning researchers!" is not a take that engenders sympathy, but we're probably talking thousands of human hours spent writing these things. It's not nothing.
- joshuamorton 6y ago
  Fair enough (I'm not a Neurips reviewer or author), but I can accept the idea that convex optimization papers likely don't need to pre-write a societal impact statement (although I think that, as controversial as the paper may be, stuff like the Gebru et. al. paper on large language models ethical concerns shows that there probably is room for more interesting thought than just "ehhh, efficiency" in many places). But the subtext of this letter goes fairly deep (and to be clear this isn't meant as a response to you specifically): - While it doesn't specifically say it, this letter does appear to be at least somewhat anti-ethical concerns in general, which isn't good. - At least one of the signatories of this letter has not, in my opinion at least, followed the guidelines in this letter. Insinuating that a colleague of holds an opinion because they watch too much online porn is in no way civil, and is absolutely a personal attack. - Given the "disagreement" between Domingos and Anandkumar, who as others mention, sits on the ACM editorial board, this could be an attempt to censure or "cancel" her for her personal views. This is antithetical to the values held in the letter itself, and leaves a sour taste.
  - throwawaysea 6y ago
    > Given the "disagreement" between Domingos and Anandkumar, who as others mention, sits on the ACM editorial board, this could be an attempt to censure or "cancel" her for her personal views. This is antithetical to the values held in the letter itself, and leaves a sour taste. Shouldn't her actions at least be censured? Her behavior has corrupted the science community, limited speech, created a culture of fear, and is ultimately inhibiting others' from freely disseminating their findings to the rest of the community (at NeurIPS and elsewhere) unless the author and/or their work passes a political purity test, effectively. If that's not discouraged, then I think science and open discourse will continue to be under attack.
    - joshuamorton 6y ago
      Which actions? What specifically has she done that "corrupted the science community", that "limited speech", that "created a culture of fear", and that is ultimately "inhibiting others from freely disseminating their findings"? List the particular actions she took that did such things. I don't think there are any. The closest you can get, I feel, is that Domingos and perhaps some anonymous others felt that her list of people who she wished to educate and discuss with was going to be used to attack them. This never actually happened though. Domingos claimed that the list would be used in such a way, and then it was deleted. I could just as easily claim that Domingos created a culture of fear by making hyperbolistic claims about the things that Anandkumar would do.
      - julianmarq 6y ago
        https://web.archive.org/web/20201214014408/https://twitter.com/AnimaAnandkumar/status/1338298561692254209 https://web.archive.org/web/20201214014408/https://twitter.c... Don't know what you were hoping to show with this comment, maybe you were unaware that there were archive links of her tweets. In any case, Domingos wasn't wrong, yet for some reason you seem far less willing to give him the benefit of the doubt, while asking others to do so for people you seem to agree with. Even in this case, for a person who clearly doesn't deserve it. I am curious as to why.
- jrumbut 6y ago
  I hate to say that your story made the ethics paragraph sound like a worthwhile exercise. If they didn't have an amazing canned answer it's because they weren't thinking about it enough to have something interesting to say. Maybe not needed for every venue every year forever, but I like this better than requiring it just from ethics sensitive papers. Presumably the people writing about swarms of killer drones have some justification or are fine with being supervillains, the field is trying to figure out where the other blindspots are and that requires broader search efforts. Edit: I just want to add this is the first I'm hearing about this letter and I'm not entirely sure what it's even in response to. Since my work is health adjacent, I haven't had time this year to know much about conferences I wasn't involved in.
  - deleted 6y ago
    [deleted]
defen 6y ago
> Many other sciences have IRB and human factors approvals for experiments that involve or may involve humans. As far as I know, those are for experiments that directly interact with humans as part of the experiment itself. They exist because of a long history of unethical researchers testing interventions (or lack thereof) on people without their knowledge or informed consent - e.g. Tuskegee Syphilis Experiment, MK-ULTRA, Stanford prison experiment (there are a lot). The closest equivalent in CS of directly-interacting experiments would be HCI research as you said. I could also see a strong argument being made for research that uses the creative or copyrighted output of a person without their consent - for example, their face as part of training facial recognition software, their written words as part of training GPT-3, their voice as part of training voice recognition software, etc. However the incident in question is really about a different kind of thing - it's asking researchers to speculate on the future ramifications of their research as it pertains to progressive ideals - in practice this means, "How could this negatively affect minorities or the environment?" These aren't inherently bad things to think about, but as you get further and further away from concrete applications of ML, it begins to look more and more like a religious ritual than something that is actually trying to address the stated problems.
- joshuamorton 6y ago
  > As far as I know, those are for experiments that directly interact with humans as part of the experiment itself. Or use certain kinds of data from humans. For example, a machine learning project on medical data (even anonymized medical data) probably needs an IRB approval to gain access to, say, a dataset of cancer imaging. A facial recognition model doesn't. What makes those fundamentally different (arguably the facial image is more identifying). > However the incident in question is really about a different kind of thing - it's asking researchers to speculate on the future ramifications of their research as it pertains to progressive ideals I see no actual support for this claim in any of the actual documentation on the broader impact statement or its use. The Neurips docs link to 4 blog posts and papers for inspiration/advice to a broader-impact-statemet-author[0][1][2][3][4]. None focus on minorities or the environment in the way you describe, unless you mean "environment" in the broadest sense of "society". [0]: https://nips.cc/Conferences/2020/PaperInformation/NeurIPS-FAQ https://nips.cc/Conferences/2020/PaperInformation/NeurIPS-FA... ("How should I write the Broader Impact section?"_ [1]: https://brenthecht.com/papers/FCADIscussions_NegativeImpactsPost_032918.pdf https://brenthecht.com/papers/FCADIscussions_NegativeImpacts... [2]: https://brenthecht.medium.com/suggestions-for-writing-neurips-2020-broader-impacts-statements-121da1b765bf https://brenthecht.medium.com/suggestions-for-writing-neurip... [3]: https://medium.com/@GovAI/a-guide-to-writing-the-neurips-impact-statement-4293b723f832 https://medium.com/@GovAI/a-guide-to-writing-the-neurips-imp... [4]: https://arxiv.org/pdf/1912.06979.pdf https://arxiv.org/pdf/1912.06979.pdf

croissants 6y ago

I agree with the general conclusion that machine learning (just like any way of making decisions) involves ethics, but I disagree with the conclusion that every NeurIPS submission should have a concluding 1-2 paragraphs about the ethics of the submission. I reviewed for NeurIPS this year, and almost all of the ethics sections consisted of vacuous stuff along the lines of "uh, I guess...being efficient is good, might save some energy? and our method is pretty efficient, so that's nice". I think a system where area chairs skim abstracts and solicit these ethics paragraphs where they seem necessary -- so, not for the billionth paper obtaining pretty bounds for some variant of convex optimization -- makes more sense. I know that "think of the machine learning researchers!" is not a take that engenders sympathy, but we're probably talking thousands of human hours spent writing these things. It's not nothing.

joshuamorton 6y ago
Fair enough (I'm not a Neurips reviewer or author), but I can accept the idea that convex optimization papers likely don't need to pre-write a societal impact statement (although I think that, as controversial as the paper may be, stuff like the Gebru et. al. paper on large language models ethical concerns shows that there probably is room for more interesting thought than just "ehhh, efficiency" in many places). But the subtext of this letter goes fairly deep (and to be clear this isn't meant as a response to you specifically): - While it doesn't specifically say it, this letter does appear to be at least somewhat anti-ethical concerns in general, which isn't good. - At least one of the signatories of this letter has not, in my opinion at least, followed the guidelines in this letter. Insinuating that a colleague of holds an opinion because they watch too much online porn is in no way civil, and is absolutely a personal attack. - Given the "disagreement" between Domingos and Anandkumar, who as others mention, sits on the ACM editorial board, this could be an attempt to censure or "cancel" her for her personal views. This is antithetical to the values held in the letter itself, and leaves a sour taste.
- throwawaysea 6y ago
  > Given the "disagreement" between Domingos and Anandkumar, who as others mention, sits on the ACM editorial board, this could be an attempt to censure or "cancel" her for her personal views. This is antithetical to the values held in the letter itself, and leaves a sour taste. Shouldn't her actions at least be censured? Her behavior has corrupted the science community, limited speech, created a culture of fear, and is ultimately inhibiting others' from freely disseminating their findings to the rest of the community (at NeurIPS and elsewhere) unless the author and/or their work passes a political purity test, effectively. If that's not discouraged, then I think science and open discourse will continue to be under attack.
  - joshuamorton 6y ago
    Which actions? What specifically has she done that "corrupted the science community", that "limited speech", that "created a culture of fear", and that is ultimately "inhibiting others from freely disseminating their findings"? List the particular actions she took that did such things. I don't think there are any. The closest you can get, I feel, is that Domingos and perhaps some anonymous others felt that her list of people who she wished to educate and discuss with was going to be used to attack them. This never actually happened though. Domingos claimed that the list would be used in such a way, and then it was deleted. I could just as easily claim that Domingos created a culture of fear by making hyperbolistic claims about the things that Anandkumar would do.
    - julianmarq 6y ago
      https://web.archive.org/web/20201214014408/https://twitter.com/AnimaAnandkumar/status/1338298561692254209 https://web.archive.org/web/20201214014408/https://twitter.c... Don't know what you were hoping to show with this comment, maybe you were unaware that there were archive links of her tweets. In any case, Domingos wasn't wrong, yet for some reason you seem far less willing to give him the benefit of the doubt, while asking others to do so for people you seem to agree with. Even in this case, for a person who clearly doesn't deserve it. I am curious as to why.
jrumbut 6y ago
I hate to say that your story made the ethics paragraph sound like a worthwhile exercise. If they didn't have an amazing canned answer it's because they weren't thinking about it enough to have something interesting to say. Maybe not needed for every venue every year forever, but I like this better than requiring it just from ethics sensitive papers. Presumably the people writing about swarms of killer drones have some justification or are fine with being supervillains, the field is trying to figure out where the other blindspots are and that requires broader search efforts. Edit: I just want to add this is the first I'm hearing about this letter and I'm not entirely sure what it's even in response to. Since my work is health adjacent, I haven't had time this year to know much about conferences I wasn't involved in.
- deleted 6y ago
  [deleted]

defen 6y ago

> Many other sciences have IRB and human factors approvals for experiments that involve or may involve humans. As far as I know, those are for experiments that directly interact with humans as part of the experiment itself. They exist because of a long history of unethical researchers testing interventions (or lack thereof) on people without their knowledge or informed consent - e.g. Tuskegee Syphilis Experiment, MK-ULTRA, Stanford prison experiment (there are a lot). The closest equivalent in CS of directly-interacting experiments would be HCI research as you said. I could also see a strong argument being made for research that uses the creative or copyrighted output of a person without their consent - for example, their face as part of training facial recognition software, their written words as part of training GPT-3, their voice as part of training voice recognition software, etc. However the incident in question is really about a different kind of thing - it's asking researchers to speculate on the future ramifications of their research as it pertains to progressive ideals - in practice this means, "How could this negatively affect minorities or the environment?" These aren't inherently bad things to think about, but as you get further and further away from concrete applications of ML, it begins to look more and more like a religious ritual than something that is actually trying to address the stated problems.

joshuamorton 6y ago
> As far as I know, those are for experiments that directly interact with humans as part of the experiment itself. Or use certain kinds of data from humans. For example, a machine learning project on medical data (even anonymized medical data) probably needs an IRB approval to gain access to, say, a dataset of cancer imaging. A facial recognition model doesn't. What makes those fundamentally different (arguably the facial image is more identifying). > However the incident in question is really about a different kind of thing - it's asking researchers to speculate on the future ramifications of their research as it pertains to progressive ideals I see no actual support for this claim in any of the actual documentation on the broader impact statement or its use. The Neurips docs link to 4 blog posts and papers for inspiration/advice to a broader-impact-statemet-author[0][1][2][3][4]. None focus on minorities or the environment in the way you describe, unless you mean "environment" in the broadest sense of "society". [0]: https://nips.cc/Conferences/2020/PaperInformation/NeurIPS-FAQ https://nips.cc/Conferences/2020/PaperInformation/NeurIPS-FA... ("How should I write the Broader Impact section?"_ [1]: https://brenthecht.com/papers/FCADIscussions_NegativeImpactsPost_032918.pdf https://brenthecht.com/papers/FCADIscussions_NegativeImpacts... [2]: https://brenthecht.medium.com/suggestions-for-writing-neurips-2020-broader-impacts-statements-121da1b765bf https://brenthecht.medium.com/suggestions-for-writing-neurip... [3]: https://medium.com/@GovAI/a-guide-to-writing-the-neurips-impact-statement-4293b723f832 https://medium.com/@GovAI/a-guide-to-writing-the-neurips-imp... [4]: https://arxiv.org/pdf/1912.06979.pdf https://arxiv.org/pdf/1912.06979.pdf