6 ms·

Here's a clarification from the Researchers over at UMN[1]. They claim that none of the Bogus patches were merged to the Stable code line : >Once any maintain

Here's a clarification from the Researchers over at UMN[1]. They claim that none of the Bogus patches were merged to the Stable code line : >Once any maintainer of the community responds to the email,indicating “looks good”,we immediately point out the introduced bug and request them to not go ahead to apply the patch. At the same time, we point out the correct fixing of the bug and provide our proper patch. In all the three cases, maintainers explicitly acknowledged and confirmed to not move forward with the incorrect patches. This way, we ensure that the incorrect patches will not be adopted or committed into the Git tree of Linux. I haven't been able to find out what the 3 patches which the reference are, but the discussions on Greg's UMN Revert patch [2] does indicate that some of the fixes have indeed been merged to Stable and are actually Bogus. [1] : https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.pdf https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.... [2] : https://lore.kernel.org/lkml/20210421130105.1226686-1-gregkh@linuxfoundation.org/ https://lore.kernel.org/lkml/20210421130105.1226686-1-gregkh...

notdang 5y ago
The main issue here is that it wastes the time of the reviewers and they did not address it in their reply.
- alanning 5y ago
  To help clarify for purposes of continuing the discussion the original research did address the issue of minimizing the time of the reviewers [1] [2]. Seems the maintainers were OK with that as no actions were taken other than an implied request to stop that kind of research. Now a different researcher from UMN, Aditya Pakki, has submitted a patch which contains bugs that seems to be attempting to do the same type of pen testing although the PhD student denied it. 1. Section IV.A of the paper, as pointed out by user MzxgckZtNqX5i in this comment: https://news.ycombinator.com/item?id=26890872 https://news.ycombinator.com/item?id=26890872 > Honoring maintainer efforts. The OSS communities are understaffed, and maintainers are mainly volunteers. We respect OSS volunteers and honor their efforts. Unfortunately, this experiment will take certain time of maintainers in reviewing the patches. To minimize the efforts, (1) we make the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we find three real minor issues (i.e., missing an error message, a memory leak, and a refcount bug), and our patches will ultimately contribute to fixing them. 2. Clarifications on the “hypocrite commit” work (FAQ) https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.pdf https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.... "* Does this project waste certain efforts of maintainers? Unfortunately, yes. We would like to sincerely apologize to the maintainers involved in the corresponding patch review process; this work indeed wasted their precious time. We had carefully considered this issue, but could not figure out a better solution in this study. However, to minimize the wasted time, (1) we made the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we tried hard to find three real bugs, and the patches ultimately contributed to fixing them."
- karsinkk 5y ago
  Agreed. This feel more like an involuntary social experiment and it just uses up the Kernel maintainers bandwidth. Reviewing code is difficult, even more so when the committer is set out to introduce bad code in the first place.
bogwog 5y ago
> I haven't been able to find out what the 3 patches which the reference are, but the discussions on Greg's UMN Revert patch [2] does indicate that some of the fixes have indeed been merged to Stable and are actually Bogus. That's because those are two separate incidents. The study which resulted in 3 patches was completed some time last year, but this new round of patches is something else. It's not clear whether the patches are coming from the same professor/group, but it seems like the author of these bogus patches is a Phd student working with the professor who conducted that study last year. So there is at least one connection. EDIT: also, those 3 patches were supposedly submitted using a fake email address according to the "clarification" document released after the paper was published. So they probably didn't use a @umn.edu email at all.
lbarrow 5y ago
The response makes the researchers seem clueless, arrogant, or both - are they really surprised that kernel maintainers would get pissed off at someone deliberately wasting their time? From the post: * Does this project waste certain efforts of maintainers? Unfortunately, yes. We would like to sincerely apologize to the maintainers involved in the corresponding patch review process; this work indeed wasted their precious time. We had carefully considered this issue, but could not figure out a better solution in this study. However, to minimize the wasted time, (1) we made the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we tried hard to find three real bugs, and the patches ultimately contributed to fixing them "Yes, this wastes maintainers time, but we decided we didn't care."
- SirSavary 5y ago
  Fascinating that the research was judged not to involve human subjects.... As someone not part of academia, how could this research be judged to not involve people? It _seems_ obvious to me that the entire premise is based around tricking/deceiving the kernel maintainers.
  - dumpsterdiver 5y ago
    Yeah, especially when the researcher begins gaslighting his subjects. He had the gall to call the maintainer's response "disgusting to hear", and then went on to ask for "the benefit of the doubt" after publishing a paper admitting that he decieved them. For comparison, imagine that you attented a small conference and unknowingly became a test subject, and when you sit down the chair shocks you with a jolt of electricity. You jump out of your chair and exclaim, "This seat shocked me!" Then the person giving the presentation walks to your seat and sits down and it doesn't shock him (because he's the one holding the button), and he then accuses you of wasting everyone's time. That's essentially what happened here.
  - shadowgovt 5y ago
    And given the apparent failure of UMN's IRB, banning UMN from contributing to the Linux kernel honestly seems like a correct approach to resolve the underlying issue.
foota 5y ago
It seems to me like the other patches were submitted in good faith, but that the maintainer no longer trusts them because of the other bad commits.
capableweb 5y ago
In the end, the damage has been done and the Linux developers are now going back and removing all patches from any user with a @umn.edu email. Not sure how the researchers didn't see how this would backfire, but it's a hopeless misuse of their time. I feel really bad for the developers who now have to spend their time fixing shit that shouldn't even be there, just because someone wanted to write a paper and their peers didn't see any problems either. How broken is academia really?
- q00r42 5y ago
  This, in of itself, is a finding. The researchers will justify their research with "we were banned which is a possible outcome of this kind of research..." I find this disingenuous. When a community of open source contributors is partially built on trust, then violators can and will be banned. The researchers should have approached the maintainers got get buy in, and setup a methodology where a maintainer would not interfere until a code merge was immanent, and just play referee in the mean time.
  - db48x 5y ago
    I don’t mind them publishing that result, as long as they make it clear that everyone from the university was banned, even people not affiliated with their research group. Of course anyone can get around that ban just by using a private email address (and the prior paper from the same research group started out using random gmail accounts rather than @umn.edu accounts), but making this point will hopefully prevent anyone from trying the same bad ideas.
- croh 5y ago
  I feel the same way. People don't understand how it is difficult to be a maintainer. This is very selfish behaviour. Appreciate Greg's strong stance against it.
- deleted 5y ago
  [deleted]
megous 5y ago
How can they be trusted though?
arendtio 5y ago
I wonder why they didn't just ask in advance. Something like 'we would like to test your review process over the next 6 months and will inform you before a critical patch hits the users', might have been a win-win scenario.
de6u99er 5y ago
It's disrespectful to people who are contributing their personal time while working for free on open source projects. With more than 60% of all acedemic publications not being reproducible [1], one would think academia has better things to do than wasting other people's time. [1] https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis

notdang 5y ago

The main issue here is that it wastes the time of the reviewers and they did not address it in their reply.

alanning 5y ago
To help clarify for purposes of continuing the discussion the original research did address the issue of minimizing the time of the reviewers [1] [2]. Seems the maintainers were OK with that as no actions were taken other than an implied request to stop that kind of research. Now a different researcher from UMN, Aditya Pakki, has submitted a patch which contains bugs that seems to be attempting to do the same type of pen testing although the PhD student denied it. 1. Section IV.A of the paper, as pointed out by user MzxgckZtNqX5i in this comment: https://news.ycombinator.com/item?id=26890872 https://news.ycombinator.com/item?id=26890872 > Honoring maintainer efforts. The OSS communities are understaffed, and maintainers are mainly volunteers. We respect OSS volunteers and honor their efforts. Unfortunately, this experiment will take certain time of maintainers in reviewing the patches. To minimize the efforts, (1) we make the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we find three real minor issues (i.e., missing an error message, a memory leak, and a refcount bug), and our patches will ultimately contribute to fixing them. 2. Clarifications on the “hypocrite commit” work (FAQ) https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.pdf https://www-users.cs.umn.edu/~kjlu/papers/clarifications-hc.... "* Does this project waste certain efforts of maintainers? Unfortunately, yes. We would like to sincerely apologize to the maintainers involved in the corresponding patch review process; this work indeed wasted their precious time. We had carefully considered this issue, but could not figure out a better solution in this study. However, to minimize the wasted time, (1) we made the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we tried hard to find three real bugs, and the patches ultimately contributed to fixing them."
karsinkk 5y ago
Agreed. This feel more like an involuntary social experiment and it just uses up the Kernel maintainers bandwidth. Reviewing code is difficult, even more so when the committer is set out to introduce bad code in the first place.

bogwog 5y ago

> I haven't been able to find out what the 3 patches which the reference are, but the discussions on Greg's UMN Revert patch [2] does indicate that some of the fixes have indeed been merged to Stable and are actually Bogus. That's because those are two separate incidents. The study which resulted in 3 patches was completed some time last year, but this new round of patches is something else. It's not clear whether the patches are coming from the same professor/group, but it seems like the author of these bogus patches is a Phd student working with the professor who conducted that study last year. So there is at least one connection. EDIT: also, those 3 patches were supposedly submitted using a fake email address according to the "clarification" document released after the paper was published. So they probably didn't use a @umn.edu email at all.

lbarrow 5y ago

The response makes the researchers seem clueless, arrogant, or both - are they really surprised that kernel maintainers would get pissed off at someone deliberately wasting their time? From the post: * Does this project waste certain efforts of maintainers? Unfortunately, yes. We would like to sincerely apologize to the maintainers involved in the corresponding patch review process; this work indeed wasted their precious time. We had carefully considered this issue, but could not figure out a better solution in this study. However, to minimize the wasted time, (1) we made the minor patches as simple as possible (all of the three patches are less than 5 lines of code changes); (2) we tried hard to find three real bugs, and the patches ultimately contributed to fixing them "Yes, this wastes maintainers time, but we decided we didn't care."

SirSavary 5y ago
Fascinating that the research was judged not to involve human subjects.... As someone not part of academia, how could this research be judged to not involve people? It _seems_ obvious to me that the entire premise is based around tricking/deceiving the kernel maintainers.
- dumpsterdiver 5y ago
  Yeah, especially when the researcher begins gaslighting his subjects. He had the gall to call the maintainer's response "disgusting to hear", and then went on to ask for "the benefit of the doubt" after publishing a paper admitting that he decieved them. For comparison, imagine that you attented a small conference and unknowingly became a test subject, and when you sit down the chair shocks you with a jolt of electricity. You jump out of your chair and exclaim, "This seat shocked me!" Then the person giving the presentation walks to your seat and sits down and it doesn't shock him (because he's the one holding the button), and he then accuses you of wasting everyone's time. That's essentially what happened here.
- shadowgovt 5y ago
  And given the apparent failure of UMN's IRB, banning UMN from contributing to the Linux kernel honestly seems like a correct approach to resolve the underlying issue.

foota 5y ago

It seems to me like the other patches were submitted in good faith, but that the maintainer no longer trusts them because of the other bad commits.

capableweb 5y ago

In the end, the damage has been done and the Linux developers are now going back and removing all patches from any user with a @umn.edu email. Not sure how the researchers didn't see how this would backfire, but it's a hopeless misuse of their time. I feel really bad for the developers who now have to spend their time fixing shit that shouldn't even be there, just because someone wanted to write a paper and their peers didn't see any problems either. How broken is academia really?

q00r42 5y ago
This, in of itself, is a finding. The researchers will justify their research with "we were banned which is a possible outcome of this kind of research..." I find this disingenuous. When a community of open source contributors is partially built on trust, then violators can and will be banned. The researchers should have approached the maintainers got get buy in, and setup a methodology where a maintainer would not interfere until a code merge was immanent, and just play referee in the mean time.
- db48x 5y ago
  I don’t mind them publishing that result, as long as they make it clear that everyone from the university was banned, even people not affiliated with their research group. Of course anyone can get around that ban just by using a private email address (and the prior paper from the same research group started out using random gmail accounts rather than @umn.edu accounts), but making this point will hopefully prevent anyone from trying the same bad ideas.
croh 5y ago
I feel the same way. People don't understand how it is difficult to be a maintainer. This is very selfish behaviour. Appreciate Greg's strong stance against it.
deleted 5y ago
[deleted]

megous 5y ago

How can they be trusted though?

arendtio 5y ago

I wonder why they didn't just ask in advance. Something like 'we would like to test your review process over the next 6 months and will inform you before a critical patch hits the users', might have been a win-win scenario.

de6u99er 5y ago

It's disrespectful to people who are contributing their personal time while working for free on open source projects. With more than 60% of all acedemic publications not being reproducible [1], one would think academia has better things to do than wasting other people's time. [1] https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis