12 ms·
Linux 5.13 Reverts and Fixes the Problematic University of Minnesota Patches
- dwheeler 5y agoJust to clarify, all the proposals that were intentionally vulnerable and were really vulnerabilities were not accepted in the first place. However, that event triggered review of all University of Minnesota proposals, and that's what's being discussed here.
- Glavnokoman 5y ago"ALL the proposals that were intentionally vulnerable and were really vulnerabilities were not accepted" The thing is there is no way you can actually know that. So this is not some kind of revenge. This is rather a valid precaution.
- Foxboron 5y ago>The thing is there is no way you can actually know that. We do know since the researchers have told the linux community, the university and IEEE what those patches where. Please do not spread further misinformation about the case. Please read the IEEE statement and the full Linux TAB review. https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_Statement.pdf https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_St... https://lkml.org/lkml/2021/5/5/1244 https://lkml.org/lkml/2021/5/5/1244
- 4ad 5y agoAh, so we're just supposed to trust the same people who tried introducing the vulnerabilities in the first place...
- Foxboron 5y agoThese are not independent malicious actors. These are researchers and students at a University. FUD does not serve any purpose here. And for full disclosure, I'm one of the four authors of the original complaint to IEEE back in December about the research. I fully believe all the facts have been put forth and there is no reason to spread misinformation about the incident.
- finnthehuman 5y ago>These are not independent malicious actors. These are researchers and students at a University. They wanted to prove that others are too trusting, they got exactly what they wanted: heightened suspicion to things that are normally expected to be done in good faith. I understand that inside the academic world the status imparted by "researchers and students at a University" is significant and important. For those of us outside the academic world, it's prudent and has no downside for us to drop our perception of them below the floor you'll continue granting them.
- hinkley 5y agoThis situation is in combination with the longstanding disconnect between academic researchers and applied sciences folks. I say “disconnect” as if that were a two way street, but most of the spherical cow thinkers are on the academic side. People doing real work bristle at their lack of exposure to reality. I worked while still in school and the difference was literally and figuratively night and day. I quickly became an informed consumer and it meant that sometimes in class I was sitting on my hands to keep from disagreeing with the teachers about why something is done a certain way. It was shocking how often they were wrong, and occasionally ass backward. Ultimately I couldn’t get out of there fast enough. But season this old recipe heavily with some “fuck around and find out” and things get pretty spicy.
- coldpie 5y agoThe fact that this paper was in the best 5% of papers submitted to this IEEE conference[1] tells you all you need to know about academia. [1] https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_Statement.pdf https://www.ieee-security.org/TC/SP2021/downloads/2021_PC_St...
- aspaceman 5y agoChrist the snark is infuriating. And pointless?
- mukesh610 5y agoThat's exactly the opposite of what the initial comment of this thread suggests.
- blumomo 5y ago"Please do not spread further misinformation about the case." Let's assume that most people are giving their opinions to their best knowledge. We shall be careful when telling someone to stop spreading misinformation as this is how fascism starts. "I am right, you are wrong, stop talking!"
- max1truc 5y ago+1 Godwin point
- rcxdude 5y agoEven when people are giving their opinions to their best knowledge they can be spreading misinformation.
- blumomo 5y agoIs it forbidden to do mistakes? In such cases is it a good idea to respond to your colleagues with "Please stop doing mistakes!" as the parent did?
- rcxdude 5y agoNo, but it may be reasonable to ask 'please do not repeat this particular mistake'
- shkkmo 5y agoBut in the particular case, the discussion centers around a matter of opinion on "should trust be extended to this particular set of statements from a known deciever". Throwing around accusations of spreading misinformation to further your opinion in an argument makes it harder to call out real misinformation.
- blumomo 5y agoDid you ever say such thing to someone who committed an error? How would you feel if someone (your partner, your colleague, your boss) told you such thing?
- gnud 5y agoWell, apparently they needed the bad publicity and "overreaction" to actually do that. The fact that they didn't communicate _clearly_ with the kernel about exactly which patches this was about at the time when they announced their paper is extremely icky, and makes my sympathy for later misunderstandings/misinformation very limited.
- Foxboron 5y agoThe entire incident has been disappointing. The researchers conducting the research thinking this was a good idea, the IRB review process at the UMN, IEEE accepting the IRB exception after ethical concerns where raised, and the overreaction from gregkh on the LKML. Hopefully there is a silver lining and we see better research collaboration between the kernel devs and researchers going forward. IEEE has a job to do around all of this going forward.
- inetknght 5y ago> the overreaction from gregkh on the LKML I don't think it was an overreaction. I think it was a very valid reaction.
- Foxboron 5y agoThere are a few issues with blaming someone for sending known malicious patches which just causes confusion and is outright wrong. Brad Spengler is a... character, but I do agree with him that greg started out on this wrong. (He also goes a bit further with his criticism that I don't agree with). However yes, review of the patches was proper but all this could have been done with less unfounded accusations from gregs side.
- neatze 5y agoCan you provide specifics what do you mean exactly by unfounded accusations ? Really curious who makes unfunded accusations ...
- kuu 5y agoThis is awful: > Based on the overall positive reviews and the recommendation of all reviewers to accept the work, the PC did not discuss this paper during the online PC meeting; [...] When, after acceptance, the authors tweeted the abstract of the work in November 2020, several people expressed concerns about human-subject research featured in this work. At that time, the PC chairs discussed these concerns [...]. As a result of these discussions, the PC chairs asked the authors to clarify the experiments with the University of Minnesota Institutional Review Board (IRB). We now acknowledge that this offer was a mistake Basically: they did not review it and once it was published and people complained, they reacted. What's the point of the PC then?
- neatze 5y agoWhat really happened (to the best of my understanding): - Researchers submitted paper to IEEE. - Researcher twitter about it. - Tweet was deleted, because people pointed out it was bad humans subject researcher. (consent and deception) - Other researchers not from UNM, filled complaints to IEEE. - Researcher mislead (so far seems like) IRB, arguably IRB failed to do a job and just rubber stamped human subject research exemption, after research was conducted ... - Paper got accepted to IEEE. - Researchers push more patches to Linux kernel. - Plonk email from Greg. - UNM response latter indirectly blaming only researchers but not IRB. - Paper get retracted from IEEE - IEEE Response letter. - We are here.
- atoav 5y agoNot trying to badmouth the university here, but having to trust the statement of those who broke your trust in the first place doesn't meet my definition of "knowing" something. Maybe "believe" would be better used here? Knowing would mean to know precisely what each of these changes does and whether they open up new vulnerabilities and then having confidence that all is well. Gaining this confidence requires work. And unless you put that work in, you are left to trusting/believing. (Edit: what I mean here is that the researchers could be totally nice and ethical people, but the Linux devs would still have to either take a risk by trusting them OR put in the work to check it all OR decide not to put in that work)
- deleted 5y ago[deleted]
- addingnumbers 5y ago80 developers reviewing for a month isn't enough work for you? They just put more man-hours into reviewing these commits than the contributors put into writing them.
- fartcannon 5y agoIt'll be interesting to see their next paper, where they describe in great depth how to waste 80 developers time for a month and still sneak in vulnerabilities.
- high_density 5y agoif 80 devs all read the same part same way, it won't make much of a difference. maybe the issue is we aren't using static analysis enough / don't have enough static analysis tools?
- neatze 5y ago> researchers could be totally nice and ethical people. A thorough review of the IRB documents revealed potential problems in the description of the experiments, and concluded that insufficient details about the experimental study were provided to the IRB. [0] [0]Read the above linked IEEE response latter.
- finnthehuman 5y agoI just read that IEEE statement, I'm glad someone else has started noticing the paper was bullshit in addition to unethical: > Investigation of these patches revealed that the description provided by the authors in the paper is ambiguous and in some cases misleading. The experiments do not provide convincing evidence to substantiate the main claims in the paper and the technical contributions of the work need to be revisited. Interesting that they list ethical considerations added to the review process, but are not adding content quality considerations to the review process. I think that's at least as embarrassing to PC. You can say that they erred in assuming the uni covered the ethics review, but what are they doing if they're accepting papers without checking that the papers support their own claims?
- rcxdude 5y agoYeah, the most hilarious part of the TAB report is that one of the hypocrite commits which was supposed to be incorrect was in fact accidentally correct because the authors failed to understand how the code worked. This was also the only commit which got accepted. This makes the conclusions and way data was presented in the paper extremely questionable.
- consp 5y ago> This makes the conclusions and way data was presented in the paper extremely questionable. This makes them invalid. Their entire claim is based on malicious code entering the kernel, not anonymous/fake name commits (which is a separate issue). If you take that away there is no actual paper, just a hypothesis which everyone already knew and thus does not warrant this much fanfare. They added nothing to the research field, bothered people with it while contributing to no-one but themselves. This is what we around here now call "Diederik Stapelen" (since he is a infamous example here): faking you data, using people to do/in your work, presenting it as true and gaining from it. It is omnipresent in some fields and in my opinion should be met with severe repercussions as it damages everyone else who was not part of it.
- Ar-Curunir 5y agoErr the PC is already supposed to vet for quality. That they failed to do so adequately is not because the policies omitted that requirement, but probably more because Oakland receives many many submissions
- cptskippy 5y ago> We do know since the researchers have told the linux community Replace researchers with hackers, CIA, North Korea, China, etc. Do you still have the warm fuzzies? The fact of the matter is they broke the trust and your expectation is that since they've been exposed they can be trusted again? No, if anything the event has shown that additional vetting and layers of scrutiny might be needed to protect against bad actors in the future should they be malicious. * I type good.
- oauea 5y agoSorry, why would you believe the words of bad actors who are known to lie? Can you be absolutely certain this isn't simply a continuation of their twisted "experiment"?
- bronson 5y agoThey lied in their abstract: https://twitter.com/SarahJamieLewis/status/1384876050207940608?s=20 https://twitter.com/SarahJamieLewis/status/13848760502079406... They lied to their IRB. What evidence do you have that they're telling the truth now?
- na85 5y ago>We do know since the researchers have told the linux community, the university and IEEE what those patches where. Please do not spread further misinformation about the case. We should just trust the liars when they pinky swear to tell the truth? There's a certain saying about what you win when you play shitty games that comes to mind. Being deceptive causes people to lose trust in you. Or, my personal favorite: fuck around, find out. UMN fucked around, now they're finding out.
- Glavnokoman 5y agoI can only suggest that you stop spreading your misbeliefs about the case. And also stop using the buzzwords the meaning of which you apparently do not understand.
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- foldr 5y agoAs usual, The Onion anticipated this: https://youtu.be/FpN_RjIaVw8 https://youtu.be/FpN_RjIaVw8
- wrs 5y agoJust from a code quality process standpoint, that’s an interesting result. Now I’m wondering what would happen if you picked a set of 150 random kernel patches and told 80 reviewers to re-review them assuming they could be malicious. I bet you’d find quite a few fixes.
- microtherion 5y agoI was wondering the same thing, and it appears it would be worth the effort, but getting enough high quality reviewers would be a problem. Maybe the NCAA could organize competitive code reviewing leagues? I bet you would get e.g. a highly motivated Caltech team reviewing USC contributed patches, and vice versa.
- bombcar 5y agoI'd suggest it sounds like a perfect project for some university students, but that may not go over very well right now.
- admax88q 5y agoProbably, but the fact that maintainers are stretched thin when it comes to time for code review is not really news.
- bluGill 5y agoThat is the openBSD model. Or at least it was 10 years ago when I last looked into it.
- burnished 5y agoThis would be an interesting study.
- dec0dedab0de 5y agoWhat is really sad, is that this could be a good pen-test for the kernel. Especially the idea of introducing bugs that are only vulnerabilities when they all come in together. If only they had contacted the Linux Foundation ahead of time to get permission, and set up terms, like a real pen-test. Then work could be done on detecting, and preventing these sorts of attacks, maybe resulting in a system that could help everyone. At the very least a database for security researchers that are registered, but unknown to maintainers, where they could store hashes of bad commits. Then a check if any of those commits made it through. I know the kernel makes use of rebasing, so that might not be the best approach technically, but something like that. To ease the pain of wasting developers time, sponsors could put up money that the maintainer gets if they catch it, and maybe a smaller amount if they have to revert it. EDIT: if the Linux Foundation said no, they could have tried another large open source project with a governing body, Apache, Python, Postgres, Firefox, etc. It wouldn't have been as flashy and high profile, but it would have still been the same research, and odds are you would find at least one project willing to participate.
- mimimi31 5y agoWhy would the Linux Foundation get to decide on if those researchers are allowed to experiment on and waste the time of volunteer developers?
- hinkley 5y agoThe Linux foundation is in a position to ensure that Linux users don’t accidentally get experimented on when the PRs are approved.
- dec0dedab0de 5y agoWhy would the Linux Foundation get to decide on if those researchers are allowed to experiment on and waste the time of volunteer developers? Granted, In the case of linux, it might make more sense if it were the combination of the Linux Foundation & Linus. It's their project, they can subject their volunteers to any tests they want. It may drive away some volunteers, but wether to take that risk or not, is up to the project to decide. For something as big as the kernel they may decide to get permission from the individual maintainers, maybe even limit the research to only the subsystems that agree to participate. In any case, the point I'm trying to make, is that this kind of testing may be beneficial, if the project is aware of it, and agrees to it. Who gets to decide on behalf of the project, would depend on the hierarchy of each project.
- greenwich26 5y agoThe guilt by association here is now approaching Biblical scales. Like the guy in the comments who wants to close down the entire Computer Science and Electrical Engineering departments at UMN, which probably employ/educate the best part of a thousand people. Ha! In general, the fury and seethe which this experiment inspired is amazing. IMO the real disgrace is not the experiment itself, but the response. The kernel developers need to stop being martyrs and playing blame games. They need to be rational and take responsibility for improving their own procedures. Because, if they didn't already have them, governments now have entire departments studying how to use deliberate vulnerabilities in open source projects, for military intelligence and other purposes. And they will not be deterred by the continued public flogging of the University of Minnesota.
- belval 5y agoMost maintainers are volunteer, meaning that they spend their evenings, weekends and holidays developing the kernel because they like it, it gives them a sense of worth and fulfillment that can be hard to come across. I don't know if you've ever talked to one, but they take a real pride in their work, most make a pitiful salary in comparison to FAANG levels but they still do it because it is full of interesting challenges you can't find anywhere else. UMN broke that trust. No one is asking the departements to close least of all the maintainers, but you have to understand that kernel devs are not a faceless machine. They spend their limited time on something that is used to make a prodigious amount of money, while never really getting any. In that context they are more than within their rights to be livid.
- nitrogen 5y agoIn that context they are more than within their rights to be livid. They do, and to a very largr extent the same exact overreaction happens in private orgs and we just get to see it when it's the Linux kernel. But at the end of the day, there was an overreaction, and it was public. Both the UMN and the kernel maintainers need to step up and make improvements now, and move forward with level heads. Have any of the kernel maintainers acknowledged the primary concern behind the misguided research? That is, (quoting parent comment) "governments now have entire departments studying how to use deliberate vulnerabilities in open source projects"?
- balozi 5y agoThe moral of this story is this: whatever you do, don't be the dweeb that gets their code closely reviewed by the kernel maintainers. (You are actually better off having it reviewed tho)
- AtNightWeCode 5y agoMaybe the people reviewing the changes should be unaware of who made them. I am biased and for sure look more closely at some developers pull requests than others.
- 63 5y agoFor the sake of spotting malicious actors, wouldn't it make more sense for reviewers to be aware, so they can focus their attention on patches from untrusted sources?
- teknopaul 5y agohonestly fsck these "pen testers", spend some time trying to make things better rather than trying to break shit and finger pointing. Starting to feel like pen testing is a broken profession.
- kzrdude 5y agoThese were not pen testers by profession.
- devit 5y agoSo what are we doing about all the maintainers that initially approved these commits that were later found to be incorrect?
- hda2 5y agothe same thing we do to a carpenter whose door get broken into in a robbery: nothing unless they were inexcusably incompetent or complicit.
- google234123 5y agoIsn't the moral of the story here that it's probably trival for organizations like the FSB/NSA/Chinese equivalent to get malicious patches accepted into Linux.
- maaand 5y agoThey already have malicious backdoors at bios level ref: intel-me
- deleted 5y ago[deleted]
- pumaontheprowl 5y agoNot after Linux banned University of Minnesota from submitting patches. We're safe now.
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- floor_ 5y agoAnyone else notice the site linked has the most crazy racist wigged out psychos on the comments/forums? Yikes.