11 ms·

I don't think there have been any recent comments from anyone at U.Mn. So, back when the original research (happened last year) the following clarification was

by kdbg 5y ago

I don't think there have been any recent comments from anyone at U.Mn. So, back when the original research (happened last year) the following clarification was offered by Qiushi Wu and Kangjie Lu which atleast paints their research in somewhat better light: https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.pdf https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.... That said the current incident seems to have gone beyond the limits of that one and is a new incident. I just thought it would be fair to include their "side"

kstenerud 5y ago
From their explanation: (3). We send the incorrect minor patches to the Linux community through email to seek their feedback. (4). Once any maintainer of the community responds to the email, indicating “looks good”, we immediately point out the introduced bug and request them to not go ahead to apply the patch. At the same time, we point out the correct fixing of the bug and provide our proper patch. In all the three cases, maintainers explicitly acknowledged and confirmed to not move forward with the incorrect patches. This way, we ensure that the incorrect patches will not be adopted or committed into the Git tree of Linux. ------------------------ But this shows a distinct lack of understanding of the problem: > This is not ok, it is wasting our time, and we will have to report this, > AGAIN, to your university... ------------------------ You do not experiment on people without their consent. This is in fact the very FIRST point of the Nuremberg code: 1. The voluntary consent of the human subject is absolutely essential.
- chenzhekl 5y ago
  Yeah, it is a bit disrespectful for kernel maintainers without gaining their approvals ahead of time.
  - moron4hire 5y ago
    Disrespecting some programmers on the internet is, while not nice, also not a high crime.
- throwawaybbq1 5y ago
  Holy cow!! I'm a researcher and don't understand how they thought it would be okay to not do an IRB, and how an IRB would not catch this. The linked PDF by the parent post is quite illustrative. The first few paras seem to be downplaying the severity of what they did (did not introduce actual bugs into the kernel) but that is not the bloody problem. They experimented on people (maintainers) without consent and wasted their time (maybe other effects too .. e.g. making them vary of future commits from universities)! I'm appalled.
  - ORioN63 5y ago
    It's not _the_ problem, but it's an actual problem. If you follow the thread, it seems they did manage to get a few approved: https://lore.kernel.org/linux-nfs/YH%2F8jcoC1ffuksrf@kroah.com/ https://lore.kernel.org/linux-nfs/YH%2F8jcoC1ffuksrf@kroah.c... I agree this whole thing paints a really ugly picture, but it seems to validate the original concerns?
    - varjag 5y ago
      Even if those they did get approved were actual security holes (not benign decoys), all that it validates is no human is infallible. Well CONGRATULATIONS.
      - Tempest1981 5y ago
        Right. And you would need a larger sample size to determine what % of the time that occurs, on average. But even then, is that useful and valid information? And is it actionable? (And if so, what is the cost of the action, and the opportunity cost of lost fixes in other areas?)
    - Throwaway951200 5y ago
      Open Source is not water proof if known committer, from well known faculty (in this case University of Minnesota) decides to send buggy patches. However, this was catched relatively quickly, but the behavior even after being caught is reprehensible: > You, and your group, have publicly admitted to sending known-buggy patches to see how the kernel community would react to them, and published a paper based on that work. > > Now you submit a new series of obviously-incorrect patches again, so what am I supposed to think of such a thing? If they kept doing it even after being caught, is beyond understandable.
- jeroenhd 5y ago
  In any university I've ever been to, this would be a gross violation of ethics with very unpleasant consequences. Informed consent is crucial when conducting experiments. If this behaviour is tolerated by the University of Minnesota (and it appears to be so) then I suppose that's another institution on my list of unreliable research. I do wonder what the legal consequences are. Would knowingly and willfully introducing bad code constitute a form of vandalism?
  - dd82 5y ago
    >>>On the Feasibility of Stealthily Introducing Vulnerabilities in Open-Source Software via Hypocrite Commits Qiushi Wu, and Kangjie Lu. To appear in Proceedings of the 42nd IEEE Symposium on Security and Privacy (Oakland'21). Virtual conference, May 2021. from Lu's list of publications at https://www-users.cs.umn.edu/~kjlu/ https://www-users.cs.umn.edu/~kjlu/ Seems like a conference presentation at IEEE at minimum?
    - xucheng 5y ago
      IEEE S&P is actually one of the top conferences in the field of computer security. It does mention some guidance on ethical consideration. > If a paper raises significant ethical and/or legal concerns, it might be rejected based on these concerns. https://www.ieee-security.org/TC/SP2021/cfpapers.html https://www.ieee-security.org/TC/SP2021/cfpapers.html So if the kernel maintainers report the issue to the S&P PC, the paper could potentially be rejected.
    - corty 5y ago
      Which shows that IEEE also has a problem with research ethics if they accepted such a paper.
      - svarog-run 5y ago
        IEEE is a garbage organization. Or atleast their India chapter is. 3 out of 5 professors in our university would recommend to avoid any paper published by Indians from IEEE. Here in India, publishing trash papers with the help of one's 'influence' is a common occurrence
- jan_Inkepa 5y ago
  In this post they say the patches come from a static analyser and they accuse the other person of slander for their criticisms > I respectfully ask you to cease and desist from making wild accusations that are bordering on slander. > These patches were sent as part of a new static analyzer that I wrote and it's sensitivity is obviously not great. I sent patches on the hopes to get feedback. We are not experts in the linux kernel and repeatedly making these statements is disgusting to hear. ( https://lore.kernel.org/linux-nfs/YH%2FfM%2FTsbmcZzwnX@kroah.com/ https://lore.kernel.org/linux-nfs/YH%2FfM%2FTsbmcZzwnX@kroah... ) How does that fit in with your explanation?
  - temp 5y ago
    >I sent patches on the hopes to get feedback They did not say that they were hoping for feedback on their tool when they submitted the patch, they lied about their code doing something it does not. >How does that fit in with your explanation? It fits in the narrative of doing hypocritical changes to the project.
    - jan_Inkepa 5y ago
      But lashing out when confronted after the fact? (I can't figure out how to browse to the messages that contain said purported 'slander' - maybe it is indeed terrible slander). Normally after the show is over one stops with the performance... edit: oh, ok I guess that post with the accusations was mid-performance? Not inconsistent, so, maybe (I'm still not clear what the timeline is).
  - jedimastert 5y ago
    > (3). We send the incorrect minor patches to the Linux community through email to seek their feedback. Sounds like they knew exactly what they were doing.
  - op00to 5y ago
    It’s a lie, that’s how it fits.
  - azernik 5y ago
- jedimastert 5y ago
  They apparently didn't consider this "human research" As I understand it, any "experiment" involving other people that weren't explicitly informed of the experiment before hand needs to be a lot more carefully considered than what they did here.
  - lithos 5y ago
    Makes sense considering how open source people are treated.
- ajb 5y ago
  > You do not experiment on people without their consent. Exactly this. Research involving human participants is supposed to have been approved by the University's Institutional Review Board; the kernel developers can complain to it: https://research.umn.edu/units/irb/about-us/contact-us https://research.umn.edu/units/irb/about-us/contact-us It would be interesting to see what these researches told the IRB they were doing (if they bothered). Edited to add: From the link in GP: "The IRB of UMN reviewed the study and determined that this is not human research (a formal IRB exempt letter was obtained)" Okay so this IRB needs to be educated about this. Probably someone in the kernel team should draft an open letter to them and get everyone to sign it (rather than everyone spamming the IRB contact form) T
  - hobofan 5y ago
    According to their website[0]: > IRB exempt was issued [0]: https://www-users.cs.umn.edu/~kjlu/ https://www-users.cs.umn.edu/~kjlu/
    - pacbard 5y ago
      A few things about IRB approval. 1. You have to submit for review any work involving human subjects before you start interacting with them. The authors clearly state that they sought retroactive approval after being questioned about their work. That would be a big red flag for my IRB and they wouldn't approve work retroactively. 2. There are multiple levels of IRB approval. The lowest is non regulated, which means that the research falls outside of human subject research. Individual researchers can self-certify work as non regulated or get a non-regulated letter from their IRB. From there, it goes from exempt to various degrees of regulated. Exempt research means that it is research involving human subjects that is exempt from continued IRB review past the initial approval. That means that IRB has found that their research involves human subjects but falls within one (or more) of the exceptions for continued review. In order to be exempt, a research project must meet one of the exemptions categories (see here https://hrpp.msu.edu/help/required/exempt-categories.html https://hrpp.msu.edu/help/required/exempt-categories.html for a list). The requirements changed in 2018, so what they had to show depends on when they first received their exempt status. The bottom line is that the research needs to (a) have less than minimal risks for participants and (b) needs to be benign in nature. In my opinion, this research doesn't meet these requirements as there are significant risks to participants to both their professional reputation and future employability for having publicly merged a malicious patch. They also pushed intentionally malicious patches, so I am not sure if the research is benign to begin with. 3. Even if a research project is found exempt from IRB review, participants still need to consent to participate in it and need to be informed of the risks and benefits of the research project. It seems that they didn't consent their participants before their participation in the research project. Consent letters usually use a common template that clearly states the goals for the research project, lists the possible risks and benefits of participating in it, states the name and contact information of the PI, and data retention policies. IRB could approve projects without proactive participant consent but those are automatically "bumped up" to full IRB approval and approvals are given only in very specific circumstances. Plus, once a participant removes their consent to participate in a research project, the research team needs to stop all interactions with them and destroy all data collected from them. It seems that the kernel maintainers did not receive the informed consent materials before starting their involvement with the research project and have expressed their desire not to participate in the research after finding out they were participating in it, so the interaction with them should stop and any data collected from them should be destroyed. 4. My impression is that they got IRB approval on a technicality. That is, their research is on the open source community and its processes rather than the individual people that participate in them. My impression of their paper is that they are very careful in addressing the "Linux community" and they really never talk about their interaction with people in the paper (e.g., there is no data collection section or a description of their interactions on the mailing list). Instead, it's my impression that they present the patches that they submitted as happening "naturally" in the community and that they are describing publicly available interactions. That seems to be a little misleading of what actually happened and their role in producing and submitting the patches.
- simias 5y ago
  It does seem rather unethical, but I must admit that I find the topic very interesting. They should definitely have asked for consent before starting with the "attack", but if they did manage to land security vulnerabilities despite the review process it's a very worrying result. And as far as I understand they did manage to do just that? I think it shows that this type of study might well be needed, it just needs to be done better and with the consent of the maintainers.
  - bezout 5y ago
    “Hey, we are going to submit some patches that contain vulnerabilities. All right?” If they do so, the maintainers become more vigilant and the experiment fails. But, the key to the experiment is that maintainers are not vigilant as they should be. It’s not an attack to the maintainers though, but to the process.
    - tapland 5y ago
      In penetration testing you are doing the same thing, but you get the go-ahead for someone responsible for the project or organization since they are interested in the results as well. A red team without approval is just a group of criminals. They must have been able to find active projects with a centralized leadership they could ask for permission.
      - bezout 5y ago
        I don’t know much about penetration testing so excuse me for the dumb question: are you required to disclose the exact methods that you’re going to use?
        baremetal 5y ago
        usually the discussion is around the end goals, rather than the means. But both are game for discussion.
        hn8788 5y ago
- qPM9l3XJrF 5y ago
  Meh, this means a lot of viral social experiments on Youtube violate the Nuremberg code...
  - XorNot 5y ago
    Yes and? This isn't a "gotcha" - people shouldn't do this.
    - vntok 5y ago
      Nah. They aren't experimenting on people, they are experimenting on organizational processes. A very different thing.
    - qPM9l3XJrF 5y ago
      Yes, and people generally don't seem upset by viral Youtube social experiments. The Nuremberg code may be the status quo and nothing more. No one here is trying to justify the code on its merits, just blindly quoting it as an authority. Here's another idea: If it's ethical to do it in a non-experimental context, it's also ethical to do it in an experimental context. So if it's OK to walk up to a stranger and ask them a weird question, it's also OK to do it in the context of a Youtube social experiment. Anything other than this is blatantly anti-scientific IMO. It is IRBs that need reform. They're self-justifying bureaucratic cruft: https://slatestarcodex.com/2017/08/29/my-irb-nightmare/ https://slatestarcodex.com/2017/08/29/my-irb-nightmare/
- unyttigfjelltol 5y ago
  In the last year when it came to experimental Covid-19 projections, modeling and population-wide recommendations from major academic centers, the IRB's were silent and academics did essentially whatever they wanted, regardless of "consent" from the populations that were the subjects of their speculative hypotheses.
- bezout 5y ago
  You could argue that they are doing the maintainers a favor. Bad actors could exploit this, and the researchers are showing that maintainers are not paying enough attention. If I were at the receiving end, I’d think checking a patch multiple times before accepting it.
  - jnxx 5y ago
    > Bad actors could exploit this, and the researchers are showing that maintainers are not paying enough attention. And this is anything new? And if I blow a hammer over your head while you are not suspecting it, does this prove anything else than that I am thug? Does it help you? Honestly?
  - UncleMeat 5y ago
    I'm sure that they thought this. But this is a bit like doing unsolicited pentests or breaking the locks on somebody's home at night without their permission. If people didn't ask for it and consent, it is unethical. And further, pretty much everybody knows that malicious actors - if they tried hard enough - would be able to sneak through hard to find vulns.
- canadianfella 5y ago
  > This is in fact the very FIRST point of the Nuremberg code Stretch Armstrong over here.
- shoto_io 5y ago
  > indicating “looks good” I wonder how many zero days have been included already, for example by nation state actors...
- dcolkitt 5y ago
  > You do not experiment on people without their consent. Applied strictly, wouldn’t every single A/B test done by a product team be considered unethical? From a common sense standpoint, it seems to me this is more about medical experiments. Yesterday I put some of my kids toys away without telling them to see if they’d notice and still play with them. I don’t think I need IRB approval.
  - avisser 5y ago
    > it seems to me this is more about medical experiments Psychology and sociology are both subject to the IRB as well. Regardless of their department, this feels like a psychology experiment.
    - bezout 5y ago
      This is a huge stretch. It’s more of a technical or operational experiment. They are testing the review process, not the maintainers.
      - Werewolf255 5y ago
        "I was testing how the bank processes having a ton of cash taken out by someone without an account, I wasn't testing the staff or police response, geez!"
  - atq2119 5y ago
    > wouldn’t every single A/B test done by a product team be considered unethical? Potentially yes, actually. I still think it should be possible to run some A/B tests, but a lot depends on the underlying motivation. The distance between such tests and malicious psychological manipulation can be very, very small.
  - yowlingcat 5y ago
    > Applied strictly, wouldn’t every single A/B test done by a product team be considered unethical? Assuming this isn't being asked as a rhetorical question, I think that's exactly what turned the now infamous Facebook A/B test into a perceived unethical mass manipulation of human emotions. A lot of folks are now justifiably upset and skeptical of Facebook (and big tech) as a result. So to answer your question: yes, if that test moves into territory that would feel like manipulation once the subject is aware of it. Maybe especially so because users are conceivably making a /choice/ to use said product and may switch to an alternative (or simply divest) if trust is lost.
- tziki 5y ago
  > You do not experiment on people without their consent. By this logic eg. resume callback studies aiming to study bias in the workforce would be impossible.
- jeltz 5y ago
  But this is all a lie. If you read the linked thread you till see that they refused to admit to their experiment and even sent a new, differently broken patch.
- splithalf 5y ago
  There is sometimes an exception for things like interviews when n is only a couple of people. This was clearly unethical and it’s certain that at least some of those involved knew that. It’s common knowledge universities.
- Blikkentrekker 5y ago
  > You do not experiment on people without their consent. This is in fact the very FIRST point of the Nuremberg code: > 1. The voluntary consent of the human subject is absolutely essential. Which is rather useless, as for many experiments to work, participants have to either be lied to, or kept in the dark as to the nature of the experiment, so whatever “consent” they give is not informed consent. They simply consent to “participate in an experiment” without being informed as to the qualities thereof so that they truly know what they are signing up for. Of course, it's quite common in the U.S.A. to perform practice medical checkups on patients who are going under narcosis for an unrelated operations, and they never consented to that, but the hospitals and physicians that partake in that are not sanctioned as it's “tradition”. Know well that so-called “human rights” have always been, and shall always be, a show of air that lack substance.
  - everybodyknows 5y ago
    > quite common in the U.S.A. to perform practice medical checkups on patients who are going under narcosis for an unrelated operations Fascinating. Can you provide links?
    - Blikkentrekker 5y ago
      https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7223770/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7223770/ https://ctexaminer.com/2021/03/20/explicit-consent-for-pelvic-and-prostate-exams-a-case-for-ct-house-bill-5067/ https://ctexaminer.com/2021/03/20/explicit-consent-for-pelvi... https://www.forbes.com/sites/paulhsieh/2018/05/14/pelvic-exams-on-anesthetized-women-without-consent-a-troubling-and-outdated-practice/?sh=3a9fce078462 https://www.forbes.com/sites/paulhsieh/2018/05/14/pelvic-exa... Most one can find of it also only deals with “intimate parts”; I am quite sceptical that this is the only thing that medical students require practice on and I think it more likely that the media only cares in this case and that in fact it is routine with many more body parts.
- deleted 5y ago
  [deleted]
- ret2plt 5y ago
  > You do not experiment on people without their consent. This is in fact the very FIRST point of the Nuremberg code: > 1. The voluntary consent of the human subject is absolutely essential. The Nuremberg code is explicitly about medical research, so it doesn't apply here. More generally, I think that the magnitude of the intervention is also relevant, and that an absolutist demand for informed consent in all - including the most trivial - cases is quite silly. Now, in this specific case I would agree that wasting people's time is an intervention that's big enough to warrant some scrutiny, but the black-and-white way of some people to phrase this really irks me. PS: I think people in these kinds of debate tend to talk past one another, so let me try to illustrate where I'm coming from with an experiment I came across recently: To study how the amount of tips waiters get changes in various circumstances, some psychologists conducted an experiment where the waiter would randomly either give the guests some chocolate with the bill, or not (control condition)[0] This is, of course, perfectly innocuous, but an absolutist claim about research ethics ("You do not experiment on people without their consent.") would make research like this impossible without any benefit. [0] https://onlinelibrary.wiley.com/doi/epdf/10.1111/j.1559-1816.2002.tb00216.x https://onlinelibrary.wiley.com/doi/epdf/10.1111/j.1559-1816...
- fouric 5y ago
  I'm confused - how is this an experiment on humans? Which humans? As far as I can tell, this has nothing to do with humans, and everything to do with the open-source review process - and if one thinks that it counts as a human experiment because humans are involved, wouldn't that logic apply equally to pentesting? For that matter, what's the difference between this and pentesting?
  - db48x 5y ago
    Penetration testing is only ethical when you are hired by the organization you are testing. Also, IRB review is only for research funded by the federal government. If you’re testing your kid’s math abilities, you’re doing an experiment on humans, and you’re entirely responsible for determining whether this is ethical or not, and without the aid of an IRB as a second opinion. Even then, successfully getting through the IRB process doesn’t guarantee that your study is ethical, only that it isn’t egregiously unethical. I suspect that if this researcher got IRB approval, then the IRB didn’t realize that these patches could end up in a released kernel. This would adversely affect the users of billions of Linux machines world–wide. Wasting half an hour of a reviewer’s time is not a concern by comparison.
  - chillfox 5y ago
    Consent! Usually when an organization is pen-tested it consented to being pen-tested (likely even requesting it). Here there were no contact with the Linux foundation to gain consent for the experiment.
- de6u99er 5y ago
  >You do not experiment on people without their consent. This is in fact the very FIRST point of the Nuremberg code: >1. The voluntary consent of the human subject is absolutely essential. Does this also apply to scrapping people's data?
detaro 5y ago
The fact that they took the feedback last time and decided "lets do more of that" is already a big red flag.
- dd82 5y ago
  >>>On the Feasibility of Stealthily Introducing Vulnerabilities in Open-Source Software via Hypocrite Commits Qiushi Wu, and Kangjie Lu. To appear in Proceedings of the 42nd IEEE Symposium on Security and Privacy (Oakland'21). Virtual conference, May 2021. from https://www-users.cs.umn.edu/~kjlu/ https://www-users.cs.umn.edu/~kjlu/ If the original research results in a paper and IEEE conference presentation, why not? There's no professional consequences for this conduct, apparently.
  - hobofan 5y ago
    Given that this conference hasn't happened yet, there should still be time for the affected people to report the inappropriate conduct to the organizers and possibly get the paper pulled.
    - throwawaybbq1 5y ago
      FYI .. many ACM conferences are now asking explicitly if an IRB was required, and if so, was it received. This does not prevent researchers from saying IRB doesn't apply, but perhaps it can be caught during peer review. Btw .. I posted a few times on the thread, and want to acknowledge that researchers are humans, and humans do make mistakes. Thankfully in this case, the direct consequence was time wasted, and this is a teaching moment for all involved. In my humble opinion, the researchers should acknowledge in stronger terms they screwed up, do a post-mortem on how this happened, and everyone (including the researchers) should move on with their lives.
      - dd82 5y ago
        Given current academia which puts a significant negative on discussing why research failed, I doubt your idea of post-mortems, public or private, will gain any traction. https://academia.stackexchange.com/questions/732/why-dont-researchers-publish-failed-experiments https://academia.stackexchange.com/questions/732/why-dont-re.... seems to list out reasons why not to do postmortems
Igorvelky 5y ago
they are mentally retarded END OF STATEMENT
andi999 5y ago
Their first suggestion to the process is pure gold:"OSS projects would be suggested to update the code of conduct, something like “By submitting the patch, I agree to not intend to introduce bugs”" Like somebody picking your locks, and suggesting, 'to stop this one approach would be to post a sign "do not pick"'
- op00to 5y ago
  The sign is to remind honest people that the lock is important, and we do not appreciate game playing here.
  - andi999 5y ago
    It is ok to put the sign. But not for the person who transgressed to suggest 'why dont you put a sign'
  - thedanbob 5y ago
    Honest people don’t see a lock and think, “Ok, they don’t want me going in there, but I bet they would appreciate some free pentesting.”
bogwog 5y ago
This does paint there side better, but it also makes me wonder if they're being wrongly accused of this current round of patches? That clarification says that they only submitted 3 patches, and that they used a random email address when doing so (so presumably no @umn.edu). These ~200 patches from UMN being reverted might have nothing to do with these researchers at all. Hopefully someone from the university clarifies what's happening soon before the angry mob tries to eat the wrong people.
- db48x 5y ago
  The study you’re quoting was a previous study by the same research group, from last year.

by kdbg 5y ago

kstenerud 5y ago

From their explanation: (3). We send the incorrect minor patches to the Linux community through email to seek their feedback. (4). Once any maintainer of the community responds to the email, indicating “looks good”, we immediately point out the introduced bug and request them to not go ahead to apply the patch. At the same time, we point out the correct fixing of the bug and provide our proper patch. In all the three cases, maintainers explicitly acknowledged and confirmed to not move forward with the incorrect patches. This way, we ensure that the incorrect patches will not be adopted or committed into the Git tree of Linux. ------------------------ But this shows a distinct lack of understanding of the problem: > This is not ok, it is wasting our time, and we will have to report this, > AGAIN, to your university... ------------------------ You do not experiment on people without their consent. This is in fact the very FIRST point of the Nuremberg code: 1. The voluntary consent of the human subject is absolutely essential.

chenzhekl 5y ago
Yeah, it is a bit disrespectful for kernel maintainers without gaining their approvals ahead of time.
- moron4hire 5y ago
  Disrespecting some programmers on the internet is, while not nice, also not a high crime.
throwawaybbq1 5y ago
Holy cow!! I'm a researcher and don't understand how they thought it would be okay to not do an IRB, and how an IRB would not catch this. The linked PDF by the parent post is quite illustrative. The first few paras seem to be downplaying the severity of what they did (did not introduce actual bugs into the kernel) but that is not the bloody problem. They experimented on people (maintainers) without consent and wasted their time (maybe other effects too .. e.g. making them vary of future commits from universities)! I'm appalled.
- ORioN63 5y ago
  It's not _the_ problem, but it's an actual problem. If you follow the thread, it seems they did manage to get a few approved: https://lore.kernel.org/linux-nfs/YH%2F8jcoC1ffuksrf@kroah.com/ https://lore.kernel.org/linux-nfs/YH%2F8jcoC1ffuksrf@kroah.c... I agree this whole thing paints a really ugly picture, but it seems to validate the original concerns?
  - varjag 5y ago
    Even if those they did get approved were actual security holes (not benign decoys), all that it validates is no human is infallible. Well CONGRATULATIONS.
    - Tempest1981 5y ago
      Right. And you would need a larger sample size to determine what % of the time that occurs, on average. But even then, is that useful and valid information? And is it actionable? (And if so, what is the cost of the action, and the opportunity cost of lost fixes in other areas?)
  - Throwaway951200 5y ago
    Open Source is not water proof if known committer, from well known faculty (in this case University of Minnesota) decides to send buggy patches. However, this was catched relatively quickly, but the behavior even after being caught is reprehensible: > You, and your group, have publicly admitted to sending known-buggy patches to see how the kernel community would react to them, and published a paper based on that work. > > Now you submit a new series of obviously-incorrect patches again, so what am I supposed to think of such a thing? If they kept doing it even after being caught, is beyond understandable.
jeroenhd 5y ago
In any university I've ever been to, this would be a gross violation of ethics with very unpleasant consequences. Informed consent is crucial when conducting experiments. If this behaviour is tolerated by the University of Minnesota (and it appears to be so) then I suppose that's another institution on my list of unreliable research. I do wonder what the legal consequences are. Would knowingly and willfully introducing bad code constitute a form of vandalism?
- dd82 5y ago
  >>>On the Feasibility of Stealthily Introducing Vulnerabilities in Open-Source Software via Hypocrite Commits Qiushi Wu, and Kangjie Lu. To appear in Proceedings of the 42nd IEEE Symposium on Security and Privacy (Oakland'21). Virtual conference, May 2021. from Lu's list of publications at https://www-users.cs.umn.edu/~kjlu/ https://www-users.cs.umn.edu/~kjlu/ Seems like a conference presentation at IEEE at minimum?
  - xucheng 5y ago
    IEEE S&P is actually one of the top conferences in the field of computer security. It does mention some guidance on ethical consideration. > If a paper raises significant ethical and/or legal concerns, it might be rejected based on these concerns. https://www.ieee-security.org/TC/SP2021/cfpapers.html https://www.ieee-security.org/TC/SP2021/cfpapers.html So if the kernel maintainers report the issue to the S&P PC, the paper could potentially be rejected.
  - corty 5y ago
    Which shows that IEEE also has a problem with research ethics if they accepted such a paper.
    - svarog-run 5y ago
      IEEE is a garbage organization. Or atleast their India chapter is. 3 out of 5 professors in our university would recommend to avoid any paper published by Indians from IEEE. Here in India, publishing trash papers with the help of one's 'influence' is a common occurrence
jan_Inkepa 5y ago
In this post they say the patches come from a static analyser and they accuse the other person of slander for their criticisms > I respectfully ask you to cease and desist from making wild accusations that are bordering on slander. > These patches were sent as part of a new static analyzer that I wrote and it's sensitivity is obviously not great. I sent patches on the hopes to get feedback. We are not experts in the linux kernel and repeatedly making these statements is disgusting to hear. ( https://lore.kernel.org/linux-nfs/YH%2FfM%2FTsbmcZzwnX@kroah.com/ https://lore.kernel.org/linux-nfs/YH%2FfM%2FTsbmcZzwnX@kroah... ) How does that fit in with your explanation?
- temp 5y ago
  >I sent patches on the hopes to get feedback They did not say that they were hoping for feedback on their tool when they submitted the patch, they lied about their code doing something it does not. >How does that fit in with your explanation? It fits in the narrative of doing hypocritical changes to the project.
  - jan_Inkepa 5y ago
    But lashing out when confronted after the fact? (I can't figure out how to browse to the messages that contain said purported 'slander' - maybe it is indeed terrible slander). Normally after the show is over one stops with the performance... edit: oh, ok I guess that post with the accusations was mid-performance? Not inconsistent, so, maybe (I'm still not clear what the timeline is).
- jedimastert 5y ago
  > (3). We send the incorrect minor patches to the Linux community through email to seek their feedback. Sounds like they knew exactly what they were doing.
- op00to 5y ago
  It’s a lie, that’s how it fits.
- azernik 5y ago
jedimastert 5y ago
They apparently didn't consider this "human research" As I understand it, any "experiment" involving other people that weren't explicitly informed of the experiment before hand needs to be a lot more carefully considered than what they did here.
- lithos 5y ago
  Makes sense considering how open source people are treated.
ajb 5y ago
> You do not experiment on people without their consent. Exactly this. Research involving human participants is supposed to have been approved by the University's Institutional Review Board; the kernel developers can complain to it: https://research.umn.edu/units/irb/about-us/contact-us https://research.umn.edu/units/irb/about-us/contact-us It would be interesting to see what these researches told the IRB they were doing (if they bothered). Edited to add: From the link in GP: "The IRB of UMN reviewed the study and determined that this is not human research (a formal IRB exempt letter was obtained)" Okay so this IRB needs to be educated about this. Probably someone in the kernel team should draft an open letter to them and get everyone to sign it (rather than everyone spamming the IRB contact form) T
- hobofan 5y ago
  According to their website[0]: > IRB exempt was issued [0]: https://www-users.cs.umn.edu/~kjlu/ https://www-users.cs.umn.edu/~kjlu/
  - pacbard 5y ago
    A few things about IRB approval. 1. You have to submit for review any work involving human subjects before you start interacting with them. The authors clearly state that they sought retroactive approval after being questioned about their work. That would be a big red flag for my IRB and they wouldn't approve work retroactively. 2. There are multiple levels of IRB approval. The lowest is non regulated, which means that the research falls outside of human subject research. Individual researchers can self-certify work as non regulated or get a non-regulated letter from their IRB. From there, it goes from exempt to various degrees of regulated. Exempt research means that it is research involving human subjects that is exempt from continued IRB review past the initial approval. That means that IRB has found that their research involves human subjects but falls within one (or more) of the exceptions for continued review. In order to be exempt, a research project must meet one of the exemptions categories (see here https://hrpp.msu.edu/help/required/exempt-categories.html https://hrpp.msu.edu/help/required/exempt-categories.html for a list). The requirements changed in 2018, so what they had to show depends on when they first received their exempt status. The bottom line is that the research needs to (a) have less than minimal risks for participants and (b) needs to be benign in nature. In my opinion, this research doesn't meet these requirements as there are significant risks to participants to both their professional reputation and future employability for having publicly merged a malicious patch. They also pushed intentionally malicious patches, so I am not sure if the research is benign to begin with. 3. Even if a research project is found exempt from IRB review, participants still need to consent to participate in it and need to be informed of the risks and benefits of the research project. It seems that they didn't consent their participants before their participation in the research project. Consent letters usually use a common template that clearly states the goals for the research project, lists the possible risks and benefits of participating in it, states the name and contact information of the PI, and data retention policies. IRB could approve projects without proactive participant consent but those are automatically "bumped up" to full IRB approval and approvals are given only in very specific circumstances. Plus, once a participant removes their consent to participate in a research project, the research team needs to stop all interactions with them and destroy all data collected from them. It seems that the kernel maintainers did not receive the informed consent materials before starting their involvement with the research project and have expressed their desire not to participate in the research after finding out they were participating in it, so the interaction with them should stop and any data collected from them should be destroyed. 4. My impression is that they got IRB approval on a technicality. That is, their research is on the open source community and its processes rather than the individual people that participate in them. My impression of their paper is that they are very careful in addressing the "Linux community" and they really never talk about their interaction with people in the paper (e.g., there is no data collection section or a description of their interactions on the mailing list). Instead, it's my impression that they present the patches that they submitted as happening "naturally" in the community and that they are describing publicly available interactions. That seems to be a little misleading of what actually happened and their role in producing and submitting the patches.
simias 5y ago
It does seem rather unethical, but I must admit that I find the topic very interesting. They should definitely have asked for consent before starting with the "attack", but if they did manage to land security vulnerabilities despite the review process it's a very worrying result. And as far as I understand they did manage to do just that? I think it shows that this type of study might well be needed, it just needs to be done better and with the consent of the maintainers.
- bezout 5y ago
  “Hey, we are going to submit some patches that contain vulnerabilities. All right?” If they do so, the maintainers become more vigilant and the experiment fails. But, the key to the experiment is that maintainers are not vigilant as they should be. It’s not an attack to the maintainers though, but to the process.
  - tapland 5y ago
    In penetration testing you are doing the same thing, but you get the go-ahead for someone responsible for the project or organization since they are interested in the results as well. A red team without approval is just a group of criminals. They must have been able to find active projects with a centralized leadership they could ask for permission.
    - bezout 5y ago
      I don’t know much about penetration testing so excuse me for the dumb question: are you required to disclose the exact methods that you’re going to use?
      - baremetal 5y ago
        usually the discussion is around the end goals, rather than the means. But both are game for discussion.
      - hn8788 5y ago
qPM9l3XJrF 5y ago
Meh, this means a lot of viral social experiments on Youtube violate the Nuremberg code...
- XorNot 5y ago
  Yes and? This isn't a "gotcha" - people shouldn't do this.
  - vntok 5y ago
    Nah. They aren't experimenting on people, they are experimenting on organizational processes. A very different thing.
  - qPM9l3XJrF 5y ago
    Yes, and people generally don't seem upset by viral Youtube social experiments. The Nuremberg code may be the status quo and nothing more. No one here is trying to justify the code on its merits, just blindly quoting it as an authority. Here's another idea: If it's ethical to do it in a non-experimental context, it's also ethical to do it in an experimental context. So if it's OK to walk up to a stranger and ask them a weird question, it's also OK to do it in the context of a Youtube social experiment. Anything other than this is blatantly anti-scientific IMO. It is IRBs that need reform. They're self-justifying bureaucratic cruft: https://slatestarcodex.com/2017/08/29/my-irb-nightmare/ https://slatestarcodex.com/2017/08/29/my-irb-nightmare/
unyttigfjelltol 5y ago
In the last year when it came to experimental Covid-19 projections, modeling and population-wide recommendations from major academic centers, the IRB's were silent and academics did essentially whatever they wanted, regardless of "consent" from the populations that were the subjects of their speculative hypotheses.
bezout 5y ago
You could argue that they are doing the maintainers a favor. Bad actors could exploit this, and the researchers are showing that maintainers are not paying enough attention. If I were at the receiving end, I’d think checking a patch multiple times before accepting it.
- jnxx 5y ago
  > Bad actors could exploit this, and the researchers are showing that maintainers are not paying enough attention. And this is anything new? And if I blow a hammer over your head while you are not suspecting it, does this prove anything else than that I am thug? Does it help you? Honestly?
- UncleMeat 5y ago
  I'm sure that they thought this. But this is a bit like doing unsolicited pentests or breaking the locks on somebody's home at night without their permission. If people didn't ask for it and consent, it is unethical. And further, pretty much everybody knows that malicious actors - if they tried hard enough - would be able to sneak through hard to find vulns.
canadianfella 5y ago
> This is in fact the very FIRST point of the Nuremberg code Stretch Armstrong over here.
shoto_io 5y ago
> indicating “looks good” I wonder how many zero days have been included already, for example by nation state actors...
dcolkitt 5y ago
> You do not experiment on people without their consent. Applied strictly, wouldn’t every single A/B test done by a product team be considered unethical? From a common sense standpoint, it seems to me this is more about medical experiments. Yesterday I put some of my kids toys away without telling them to see if they’d notice and still play with them. I don’t think I need IRB approval.
- avisser 5y ago
  > it seems to me this is more about medical experiments Psychology and sociology are both subject to the IRB as well. Regardless of their department, this feels like a psychology experiment.
  - bezout 5y ago
    This is a huge stretch. It’s more of a technical or operational experiment. They are testing the review process, not the maintainers.
    - Werewolf255 5y ago
      "I was testing how the bank processes having a ton of cash taken out by someone without an account, I wasn't testing the staff or police response, geez!"
- atq2119 5y ago
  > wouldn’t every single A/B test done by a product team be considered unethical? Potentially yes, actually. I still think it should be possible to run some A/B tests, but a lot depends on the underlying motivation. The distance between such tests and malicious psychological manipulation can be very, very small.
- yowlingcat 5y ago
  > Applied strictly, wouldn’t every single A/B test done by a product team be considered unethical? Assuming this isn't being asked as a rhetorical question, I think that's exactly what turned the now infamous Facebook A/B test into a perceived unethical mass manipulation of human emotions. A lot of folks are now justifiably upset and skeptical of Facebook (and big tech) as a result. So to answer your question: yes, if that test moves into territory that would feel like manipulation once the subject is aware of it. Maybe especially so because users are conceivably making a /choice/ to use said product and may switch to an alternative (or simply divest) if trust is lost.
tziki 5y ago
> You do not experiment on people without their consent. By this logic eg. resume callback studies aiming to study bias in the workforce would be impossible.
jeltz 5y ago
But this is all a lie. If you read the linked thread you till see that they refused to admit to their experiment and even sent a new, differently broken patch.
splithalf 5y ago
There is sometimes an exception for things like interviews when n is only a couple of people. This was clearly unethical and it’s certain that at least some of those involved knew that. It’s common knowledge universities.
Blikkentrekker 5y ago
> You do not experiment on people without their consent. This is in fact the very FIRST point of the Nuremberg code: > 1. The voluntary consent of the human subject is absolutely essential. Which is rather useless, as for many experiments to work, participants have to either be lied to, or kept in the dark as to the nature of the experiment, so whatever “consent” they give is not informed consent. They simply consent to “participate in an experiment” without being informed as to the qualities thereof so that they truly know what they are signing up for. Of course, it's quite common in the U.S.A. to perform practice medical checkups on patients who are going under narcosis for an unrelated operations, and they never consented to that, but the hospitals and physicians that partake in that are not sanctioned as it's “tradition”. Know well that so-called “human rights” have always been, and shall always be, a show of air that lack substance.
- everybodyknows 5y ago
  > quite common in the U.S.A. to perform practice medical checkups on patients who are going under narcosis for an unrelated operations Fascinating. Can you provide links?
  - Blikkentrekker 5y ago
    https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7223770/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7223770/ https://ctexaminer.com/2021/03/20/explicit-consent-for-pelvic-and-prostate-exams-a-case-for-ct-house-bill-5067/ https://ctexaminer.com/2021/03/20/explicit-consent-for-pelvi... https://www.forbes.com/sites/paulhsieh/2018/05/14/pelvic-exams-on-anesthetized-women-without-consent-a-troubling-and-outdated-practice/?sh=3a9fce078462 https://www.forbes.com/sites/paulhsieh/2018/05/14/pelvic-exa... Most one can find of it also only deals with “intimate parts”; I am quite sceptical that this is the only thing that medical students require practice on and I think it more likely that the media only cares in this case and that in fact it is routine with many more body parts.
deleted 5y ago
[deleted]
ret2plt 5y ago
> You do not experiment on people without their consent. This is in fact the very FIRST point of the Nuremberg code: > 1. The voluntary consent of the human subject is absolutely essential. The Nuremberg code is explicitly about medical research, so it doesn't apply here. More generally, I think that the magnitude of the intervention is also relevant, and that an absolutist demand for informed consent in all - including the most trivial - cases is quite silly. Now, in this specific case I would agree that wasting people's time is an intervention that's big enough to warrant some scrutiny, but the black-and-white way of some people to phrase this really irks me. PS: I think people in these kinds of debate tend to talk past one another, so let me try to illustrate where I'm coming from with an experiment I came across recently: To study how the amount of tips waiters get changes in various circumstances, some psychologists conducted an experiment where the waiter would randomly either give the guests some chocolate with the bill, or not (control condition)[0] This is, of course, perfectly innocuous, but an absolutist claim about research ethics ("You do not experiment on people without their consent.") would make research like this impossible without any benefit. [0] https://onlinelibrary.wiley.com/doi/epdf/10.1111/j.1559-1816.2002.tb00216.x https://onlinelibrary.wiley.com/doi/epdf/10.1111/j.1559-1816...
fouric 5y ago
I'm confused - how is this an experiment on humans? Which humans? As far as I can tell, this has nothing to do with humans, and everything to do with the open-source review process - and if one thinks that it counts as a human experiment because humans are involved, wouldn't that logic apply equally to pentesting? For that matter, what's the difference between this and pentesting?
- db48x 5y ago
  Penetration testing is only ethical when you are hired by the organization you are testing. Also, IRB review is only for research funded by the federal government. If you’re testing your kid’s math abilities, you’re doing an experiment on humans, and you’re entirely responsible for determining whether this is ethical or not, and without the aid of an IRB as a second opinion. Even then, successfully getting through the IRB process doesn’t guarantee that your study is ethical, only that it isn’t egregiously unethical. I suspect that if this researcher got IRB approval, then the IRB didn’t realize that these patches could end up in a released kernel. This would adversely affect the users of billions of Linux machines world–wide. Wasting half an hour of a reviewer’s time is not a concern by comparison.
- chillfox 5y ago
  Consent! Usually when an organization is pen-tested it consented to being pen-tested (likely even requesting it). Here there were no contact with the Linux foundation to gain consent for the experiment.
de6u99er 5y ago
>You do not experiment on people without their consent. This is in fact the very FIRST point of the Nuremberg code: >1. The voluntary consent of the human subject is absolutely essential. Does this also apply to scrapping people's data?

detaro 5y ago

The fact that they took the feedback last time and decided "lets do more of that" is already a big red flag.

dd82 5y ago
>>>On the Feasibility of Stealthily Introducing Vulnerabilities in Open-Source Software via Hypocrite Commits Qiushi Wu, and Kangjie Lu. To appear in Proceedings of the 42nd IEEE Symposium on Security and Privacy (Oakland'21). Virtual conference, May 2021. from https://www-users.cs.umn.edu/~kjlu/ https://www-users.cs.umn.edu/~kjlu/ If the original research results in a paper and IEEE conference presentation, why not? There's no professional consequences for this conduct, apparently.
- hobofan 5y ago
  Given that this conference hasn't happened yet, there should still be time for the affected people to report the inappropriate conduct to the organizers and possibly get the paper pulled.
  - throwawaybbq1 5y ago
    FYI .. many ACM conferences are now asking explicitly if an IRB was required, and if so, was it received. This does not prevent researchers from saying IRB doesn't apply, but perhaps it can be caught during peer review. Btw .. I posted a few times on the thread, and want to acknowledge that researchers are humans, and humans do make mistakes. Thankfully in this case, the direct consequence was time wasted, and this is a teaching moment for all involved. In my humble opinion, the researchers should acknowledge in stronger terms they screwed up, do a post-mortem on how this happened, and everyone (including the researchers) should move on with their lives.
    - dd82 5y ago
      Given current academia which puts a significant negative on discussing why research failed, I doubt your idea of post-mortems, public or private, will gain any traction. https://academia.stackexchange.com/questions/732/why-dont-researchers-publish-failed-experiments https://academia.stackexchange.com/questions/732/why-dont-re.... seems to list out reasons why not to do postmortems

Igorvelky 5y ago

they are mentally retarded END OF STATEMENT

andi999 5y ago

Their first suggestion to the process is pure gold:"OSS projects would be suggested to update the code of conduct, something like “By submitting the patch, I agree to not intend to introduce bugs”" Like somebody picking your locks, and suggesting, 'to stop this one approach would be to post a sign "do not pick"'

op00to 5y ago
The sign is to remind honest people that the lock is important, and we do not appreciate game playing here.
- andi999 5y ago
  It is ok to put the sign. But not for the person who transgressed to suggest 'why dont you put a sign'
- thedanbob 5y ago
  Honest people don’t see a lock and think, “Ok, they don’t want me going in there, but I bet they would appreciate some free pentesting.”

bogwog 5y ago

This does paint there side better, but it also makes me wonder if they're being wrongly accused of this current round of patches? That clarification says that they only submitted 3 patches, and that they used a random email address when doing so (so presumably no @umn.edu). These ~200 patches from UMN being reverted might have nothing to do with these researchers at all. Hopefully someone from the university clarifies what's happening soon before the angry mob tries to eat the wrong people.

db48x 5y ago
The study you’re quoting was a previous study by the same research group, from last year.