5 ms·

There are some relatively minor issues with this apology that appear to already have ample discussion here, and I'll not repeat it. I want something more: I wa

There are some relatively minor issues with this apology that appear to already have ample discussion here, and I'll not repeat it. I want something more: I want to hear from the sponsoring faculty, research ethics board, and editors of the journal that published the article. There appear to be some systemic issues in addition to the investigators' ill-considered project. How was it that this research, which is clearly unethical, ended up being published? I've sat on an IRB: this study would not even have been a close call. Did sponsoring faculty even send it for IRB approval? Did they disclose that they were misleading their subjects? If so, did the IRB approve it? Did it make any recommendations? When submitted for publication, did they state they had IRB approval? Did they disclose that they were misleading experimental subjects? Did any reviewers express ethical concerns? Were the ethics of this study discussed by editors? Because there's either some serious systemic flaws in the review and publication process for this paper or the investigators engaged in serious misconduct. If the latter an apology (while absolutely required) is far from sufficient to address the issue. If the former, then there are several other apologies due, along with confirmation that the process will be reviewed and corrected.

neatze 5y ago
What are your thoughts about this article[0], my reading or article is that; author fails in similar way (to some extent) as researchers and there's IRB in regards to ethics of such research. [0] https://dave-dittrich.medium.com/security-research-ethics-review-cdcabf1bbabf https://dave-dittrich.medium.com/security-research-ethics-re...
- ajarmst 5y ago
  It's an insightful review into some of the issues and regulations around this. I'm not American, so our local rules are different, and I resigned from my institution's IRB several years ago, and these issues have become more fraught in this period. However, the way I was trained to look at these issues were around the concepts of harm, both actual and potential. In this particular case, as Dittrich notes, the questions are around the ethics of using deception in research. Deception does have a role in legitimate research. Arguably, double-blind experiments, the sine qua non of medical research have a fundamental component of deception. They also represent the most common way the ethical dilemma is resolved: subjects are told that they may be deceived, and they have an opportunity to give informed consent. That could have been done in this case: have project leads inform people working on the project that, in the interest of evaluating the patching process, patches that are incorrect or which introduce vulnerabilities may be submitted by researchers. That, of course, would require some senior members of the organization be aware the study was going on. That last is the way penetration testing and red team investigations of security resolve the ethical question---and distinguish themselves from mere vandals and criminals. Other ways to resolve it include collecting data without deception: instead of introducing flawed or malicious patches themselves, researchers identify such patches that have historically been submitted and then review the processes that led to their acceptance or rejection. This is more difficult, but might arguably produce better results. In the case that there are few or no such cases on record, then I would question the value of doing the study at all: deceiving people to study a phenomena that doesn't appear to occur at an appreciable rate is difficult to justify. There's a simple heuristic: if you're studying a group of human beings that you are not a member of, and for which no members are consciously participating, you must be extremely careful. The general rule in anthropological and sociological research is that you do not lie to your subjects. There are cases where the value of the research is sufficient, and for which no other options are available, to break that rule. But they are rare and the utility must be clearly shown and carefully reviewed by a qualified third party. This experiment doesn't come close. There will certainly be those those willing to argue that this isn't human experimentation and thus does not require ethical review. If your experiment depends on misleading human beings---directly or by omission---then it requires an ethical review. It is unethical to waste people's time to no purpose. In the case of an open source project where volunteers are donating their time, it is particularly egregious: they were squandering volunteers' time. In effect they were destroying part of the contribution people made to a project they care about. That requires a very clear justification, which this particular project absolutely does not provide. I recall a conversation with other IRB members shortly after the Sokal Hoax became known. Our general consensus was that it was hilarious but absolutely unethical if considered as an experiment.
  - neatze 5y ago
    Thank you for such detailed response. Can there be ethical possibility (highly remote one) where an study (assuming it is objectively justified) conducted with deception, but without prior informed consent at all. (eg. human subjects will not know that there's time is used for another purpose)
    - strgcmc 5y ago
      Doesn't this happen all the time? Many psychology studies are done by bringing in test subjects, asking for their consent to be interviewed or tested under the guise of studying X, when in fact the researchers are looking to evaluate Y instead? E.g. I the researcher ask if you consent to spending 30min completing a series of tasks to sort objects by their shape, presumably because I want to study your ability to recognize shapes. However, what I am actually studying is the group dynamics, of how well you and others in the group cooperate or have conflict over your tasks.
      - ajarmst 5y ago
        TLDR: Yes, but deceiving a subject about the details of how they are being studied is different from deceiving them about the fact that they are a research subject. Yes, this happens all the time. But note the salient features: (1) the subjects are aware that they are research subjects and have agreed to participate (albeit without full knowledge of how the collected data will be analyzed). They have agreed to be studied, and have agreed that their time may be used in pursuit of this research. (2) All such studies undergo a very stringent ethical review and are usually monitored closely by third parties (at least since Milgram made it extremely clear that this was a necessary policy). These issues are complex and difficult to navigate---which is precisely why we have review boards. Every experiment has to be evaluated to balance the requirement to act ethically with the value of the research data to be collected. Skipping that requirement is unacceptable. In my experience, the moment a research team starts looking for reasons not to classify what they are doing as human experimentation is the moment when it becomes extremely clear that they need board review.
- neoflame 5y ago
  IANAL, but the claim that this research was exempt under 45 CFR 46.104(d)(2) seems suspect to me. (i) doesn't seem to apply because Linux kernel developers are required to go by their real names for licensing reasons (cf. the rules regarding Signed-off-by). (ii) seems dubious given that the authors themselves argue that they need reviewer consent to release information about the authors' malicious patches. Note in particular that both exemption categories are concerned with what information the researchers have ("information ... recorded" in (i), "any disclosure ... would not" in (ii)), not what they publish, so the idea that they need consent to publish this information seems to imply that they needed consent to collect it.
  - neatze 5y ago
    My biggest point of confusion from article is that regulation(s) do not require explicit consent from human subjects on use of there's time, irrespective of what information is collected.
- gnud 5y ago
  Both this guy, and the researchers themselves (in their article), suggests that the kernel adds "don't submit know-bad patches" to their code of conduct. So it's sort of hard to take them seriously as human beings, and not just caricatures.
trop 5y ago
Another query about IRB and human subjects: In the case of software which is not only written/maintained by a community of humans, but used by a (vast) community of humans, do the latter also become "human subjects" as well in such an experiment?
- ajarmst 5y ago
  Only one of those groups is having their time and effort squandered. To me, it comes down to whether someone is being harmed. Researchers use statistics and anonymized demographic data on large populations all the time. You could argue that the population in question have become research subjects, but (1) they are not being harmed and (2) they are not being actively misled by the investigators. The moment this group started submitting patches in bad faith, they were actively misleading their subjects, and they are harming those subjects by using their time and effort without consent.

neatze 5y ago

What are your thoughts about this article[0], my reading or article is that; author fails in similar way (to some extent) as researchers and there's IRB in regards to ethics of such research. [0] https://dave-dittrich.medium.com/security-research-ethics-review-cdcabf1bbabf https://dave-dittrich.medium.com/security-research-ethics-re...

ajarmst 5y ago
It's an insightful review into some of the issues and regulations around this. I'm not American, so our local rules are different, and I resigned from my institution's IRB several years ago, and these issues have become more fraught in this period. However, the way I was trained to look at these issues were around the concepts of harm, both actual and potential. In this particular case, as Dittrich notes, the questions are around the ethics of using deception in research. Deception does have a role in legitimate research. Arguably, double-blind experiments, the sine qua non of medical research have a fundamental component of deception. They also represent the most common way the ethical dilemma is resolved: subjects are told that they may be deceived, and they have an opportunity to give informed consent. That could have been done in this case: have project leads inform people working on the project that, in the interest of evaluating the patching process, patches that are incorrect or which introduce vulnerabilities may be submitted by researchers. That, of course, would require some senior members of the organization be aware the study was going on. That last is the way penetration testing and red team investigations of security resolve the ethical question---and distinguish themselves from mere vandals and criminals. Other ways to resolve it include collecting data without deception: instead of introducing flawed or malicious patches themselves, researchers identify such patches that have historically been submitted and then review the processes that led to their acceptance or rejection. This is more difficult, but might arguably produce better results. In the case that there are few or no such cases on record, then I would question the value of doing the study at all: deceiving people to study a phenomena that doesn't appear to occur at an appreciable rate is difficult to justify. There's a simple heuristic: if you're studying a group of human beings that you are not a member of, and for which no members are consciously participating, you must be extremely careful. The general rule in anthropological and sociological research is that you do not lie to your subjects. There are cases where the value of the research is sufficient, and for which no other options are available, to break that rule. But they are rare and the utility must be clearly shown and carefully reviewed by a qualified third party. This experiment doesn't come close. There will certainly be those those willing to argue that this isn't human experimentation and thus does not require ethical review. If your experiment depends on misleading human beings---directly or by omission---then it requires an ethical review. It is unethical to waste people's time to no purpose. In the case of an open source project where volunteers are donating their time, it is particularly egregious: they were squandering volunteers' time. In effect they were destroying part of the contribution people made to a project they care about. That requires a very clear justification, which this particular project absolutely does not provide. I recall a conversation with other IRB members shortly after the Sokal Hoax became known. Our general consensus was that it was hilarious but absolutely unethical if considered as an experiment.
- neatze 5y ago
  Thank you for such detailed response. Can there be ethical possibility (highly remote one) where an study (assuming it is objectively justified) conducted with deception, but without prior informed consent at all. (eg. human subjects will not know that there's time is used for another purpose)
  - strgcmc 5y ago
    Doesn't this happen all the time? Many psychology studies are done by bringing in test subjects, asking for their consent to be interviewed or tested under the guise of studying X, when in fact the researchers are looking to evaluate Y instead? E.g. I the researcher ask if you consent to spending 30min completing a series of tasks to sort objects by their shape, presumably because I want to study your ability to recognize shapes. However, what I am actually studying is the group dynamics, of how well you and others in the group cooperate or have conflict over your tasks.
    - ajarmst 5y ago
      TLDR: Yes, but deceiving a subject about the details of how they are being studied is different from deceiving them about the fact that they are a research subject. Yes, this happens all the time. But note the salient features: (1) the subjects are aware that they are research subjects and have agreed to participate (albeit without full knowledge of how the collected data will be analyzed). They have agreed to be studied, and have agreed that their time may be used in pursuit of this research. (2) All such studies undergo a very stringent ethical review and are usually monitored closely by third parties (at least since Milgram made it extremely clear that this was a necessary policy). These issues are complex and difficult to navigate---which is precisely why we have review boards. Every experiment has to be evaluated to balance the requirement to act ethically with the value of the research data to be collected. Skipping that requirement is unacceptable. In my experience, the moment a research team starts looking for reasons not to classify what they are doing as human experimentation is the moment when it becomes extremely clear that they need board review.
neoflame 5y ago
IANAL, but the claim that this research was exempt under 45 CFR 46.104(d)(2) seems suspect to me. (i) doesn't seem to apply because Linux kernel developers are required to go by their real names for licensing reasons (cf. the rules regarding Signed-off-by). (ii) seems dubious given that the authors themselves argue that they need reviewer consent to release information about the authors' malicious patches. Note in particular that both exemption categories are concerned with what information the researchers have ("information ... recorded" in (i), "any disclosure ... would not" in (ii)), not what they publish, so the idea that they need consent to publish this information seems to imply that they needed consent to collect it.
- neatze 5y ago
  My biggest point of confusion from article is that regulation(s) do not require explicit consent from human subjects on use of there's time, irrespective of what information is collected.
gnud 5y ago
Both this guy, and the researchers themselves (in their article), suggests that the kernel adds "don't submit know-bad patches" to their code of conduct. So it's sort of hard to take them seriously as human beings, and not just caricatures.

trop 5y ago

Another query about IRB and human subjects: In the case of software which is not only written/maintained by a community of humans, but used by a (vast) community of humans, do the latter also become "human subjects" as well in such an experiment?

ajarmst 5y ago
Only one of those groups is having their time and effort squandered. To me, it comes down to whether someone is being harmed. Researchers use statistics and anonymized demographic data on large populations all the time. You could argue that the population in question have become research subjects, but (1) they are not being harmed and (2) they are not being actively misled by the investigators. The moment this group started submitting patches in bad faith, they were actively misleading their subjects, and they are harming those subjects by using their time and effort without consent.