20 ms·
FreeBSD Core Team Statement on FreeBSD Development Processes
- Metacelsus 6y agoLooks like they're committed to improving things.
- JediPig 6y agoFreeBSD is already 1000x better code quality than linux. Linux still has WONT FIX kernel bugs, WONT FIX bugs do not exist in freebsd that are userland based, they fix both kernel and userland.
- tgbugs 6y agoHere is a writeup that provides a good review of the context for this [0]. 0. https://arstechnica.com/gadgets/2021/03/buffer-overruns-license-violations-and-bad-code-freebsd-13s-close-call/ https://arstechnica.com/gadgets/2021/03/buffer-overruns-lice...
- kevans91 6y agoFWIW, "good review" is pretty debatable (source: was involved) -- it provides a decent overview, but you should read it with the thought in mind that the author did some pretty heavy cherry-picking to support their arguments. Also, this indeed is not "Kernel Debugging Quarterly."
- mbreese 6y agoI thought the Ars article was a pretty good overview for the context of this current issue and why there needs to be a statement on FreeBSD development practices at all. I don't expect Ars to get into too many kernel specifics, so won't knock them for that. But as far as bringing more attention to how the mess started in the first place (and was dealt with), I think the Ars article is a pretty good place to start. I mean, the question of how the rough-draft Wireguard implementation made it into the kernel in the first place without sufficient review is a pretty big question. I'm happy the FreeBSD team is addressing it directly.
- kevans91 6y agoYeah, sorry, I'm mostly lamenting that there's not a more objective overview. This whole situation is pretty tiring at this point, so it's unlikely that one will surface except as a post-mortem down the road.
- busterarm 6y agoI think that unfortunately a lot of people can't separate "this code is bad" from "this person is bad". We all have times when we don't ship our best work. Life happens. It's also worth noting to the readers that Donenfeld's criticism of bad code is completely dispassionate and that he isn't above criticising himself. He seems to be genuinely among the nicest people in the community. This seems unfortunate for mmacy and unfortunately exacerbated by the behavior of Netgate.
- loeg 6y ago> It's also worth noting to the readers that Donenfeld's criticism of bad code is completely dispassionate and that he isn't above criticising himself. Er, what? Donenfeld's hyperbole and wild characterizations are part of what fanned the flames and made this a tech-press mess instead of some quiet collaboration and bug reports.[0] > The first step was assessing the current state of the code the previous developer had dumped into the tree. It was not pretty. I imagined strange Internet voices jeering, “this is what gives C a bad name!” There were random sleeps added to “fix” race conditions, validation functions that just returned true, catastrophic cryptographic vulnerabilities, whole parts of the protocol unimplemented, kernel panics, security bypasses, overflows, random printf statements deep in crypto code, the most spectacular buffer overflows, and the whole litany of awful things that go wrong when people aren’t careful when they write C. While some details are based in reality, the paragraph goes well beyond the realm of truth. It makes totally unnecessarily jabs at Macy as "the previous developer." He is (broadly) a competent C/kernel developer. Yes, he did an inadequate job here. No, some Greek chorus isn't jeering about the C code just because stylistically it differs from how Donenfeld would write it. To my knowledge: * There was only a single "validation function that returned true," and it involved validating an ip address internal to a validated and decoded message from a wg peer. The message is already cryptographically verified; only peers that are part of the same mesh could spoof IPs outside of their configured range. (Donenfeld described this as validation functions, plural.) * Donenfeld's only ever found a single real buffer overflow. It's the one where Jumbo frames can cause heap overflow. His other buffer overflow claims are not realistic due to other constraints on the inputs. Mostly they seem to reflect stylistic preferences about using mallocarray(a, n) instead of malloc(a * n). So the claims of "spectacular" buffer overflow(s), plural, feels disingenuous. (I don't know what "spectacular" is supposed to mean in a cold technical critique, either.) Maybe this is "dispassionate," but it seems unnecessarily careless with the facts when writing technical criticism. To be clear, Netgate's press response to this was totally inappropriate and also just a dumb move. The public narrative would be more in their favor if they had been totally silent instead of posting the angry screed they did. Ars takes Donenfeld's hyperbole and runs with it, fact-checking only the easily verified claims. And there is some truthiness to it! Unfortunately, it's the rest of the communication that leaves something wanting. Anyway, I love wireguard and what Donenfeld has accomplished. I just wish the guy would be a bit more considerate and less colorful when writing sensitive emails. [0]: https://lists.zx2c4.com/pipermail/wireguard/2021-March/006494.html https://lists.zx2c4.com/pipermail/wireguard/2021-March/00649...
- generalizations 6y agoHonestly, the criticism of the freebsd code review process seems way overblown. To me, it looks like a smaller team that works well together in a less formal manner, and someone exploited that. They caught it, the person has lost trust, they all moved on. Big whup.
- unethical_ban 6y agoThe items they discuss, like more rigor on MRs, static analysis, etc. seem like no-brainers from five years ago. I'm not knocking them, I don't know the scope of their limitations. And they're doing the improvements now, so good on them. Just seems odd some things weren't being done previously.
- oblio 6y agoMy guess, after lurking around the internet for the past few decades and keeping an eye on what BSDs generally do, it's a mix of NIH/people that don't want to change their existing workflows and lack of financial resources. It's hard to say what the percentages are for the two.
- gmueckl 6y agoEven if you trust your team members to do a good job, another pair of eyes on the code before it goes in rarely hurt. Everybody slips up once in a while and a review has at least a chance of catching the worst mistakes early. Team size doesn't matter. You just need to bring a little discipline to the process.
- ajross 6y agoWell, it's not the end of the world (and I feel much more sympathy for this guy's poor tenants than I do for pfSense customers), but... Bad code going unreviewed from a single author into a main branch from which people build production systems is definitely beyond "Big whup" severity. The equivalent would be if one person pushed an unreviewed driver into Fedora Rawhide or an Ubuntu Beta, say. It's a clear violation of the principles behind the service the distro is supposed to be providing for you. There are a zillion ways to make sure code gets reviewed before merge. Linux does it informally via Signed-off-by headers and a tree structure of trusted maintainers. Services like github provided automated tooling to enforce review. FreeBSD needs to just pick one. It's 2021, for goodness sake. A fixed COMMITTERS list just isn't going to cut it.
- cryptonector 6y agoWow. Thanks for that link.
- Ericson2314 6y agoWow, evil landlords, C being bad. All my favorite things to discuss in one article!
- loeg 6y agoI would suggest the LWN article instead: https://news.ycombinator.com/item?id=26572370 https://news.ycombinator.com/item?id=26572370
- secondcoming 6y agoDid Coverity not pick up on any of the bugs that were later found? The folks behind PVStudio regularly spam the /r/cpp subreddit with actually very intersting articles about issues their static analyser finds in several open source projects. Does anyone here use it? (I do not work for them or use their product)
- loeg 6y agoCoverity has a bunch of false positives and true positives, and FreeBSD wasn't coverity-clean before this patch landed. No one in the FreeBSD dev community has time to watch every Coverity report that comes in.
- justin66 6y agoHow does that work, is there a periodic review of static analysis results?
- bluGill 6y agoIf the number of false positives in your project is > 10 static analysis is a waste of time. It doesn't matter if your project is 10 lines, or 100 million, 10 is the most false positives static analysis is allowed to have before it is shut off as useless. It is very hard to be a static analysis tool developer. All the easy cases have been done, most have been folded into the compiler. What is left is the hard cases where you have to figure out how to not trigger on the false positives. Get this wrong and your customers will dump you - doesn't matter if you get it wrong by not detection many problems, or you get it wrong by too many false positives. It doesn't help that once the customer has fixed all the problems they forget about them, but the false positives come up all the time. You can't get out by making it possible to mark false positives. Once you allow that everyone will just get in the habit of marking all messages as a false positive. I tracked one production bug to a line that was immediately preceded by a false positive suppression - the problem was real but the someone decided instead of thinking they would suppress the issue like every other one found.
- busterarm 6y agoNetgate doesn't exactly have the best reputation as it is. There is a reason there's a growing push of people leaving pfSense for OPNsense. Remember when they bought their competitor's domain and put up this? http://web.archive.org/web/20160314132836/http:/www.opnsense.com/ http://web.archive.org/web/20160314132836/http:/www.opnsense...
- TavsiE9s 6y agoThey were supposedly also squatting r/OPNSense, which is why the official subreddit is r/OPNsenseFirewall. Just checked: r/opnsense is a private community The moderators of r/opnsense have set this community to private. Only approved members can view and take part in its discussions.
- gonzo 6y agoNot Netgate
- zokier 6y agoKinda feel bad for them here though because they did things initially pretty much exactly right: paid real money to a developer with reasonable credentials to work in upstream to develop this feature with apparently best of intentions. That is pretty much textbook example how corporations can participate in open source (without in-house developers). And then it somehow blew on their face. The response then from Netgate was very much on-brand, they indeed have a history of colorful PR. And of course they are ultimately responsible for things they put their name on. But this also highlights how difficult participating in open source can be.
- busterarm 6y agoNetgate isn't exactly your average open source contributor either. In fact, their product is very much closed source. Even the areas of it that have been "open" have largely hidden, incomplete and non-buildable for the longest. It's really a mixed bag. They pay for a bit of development but throw their weight around in ways that are harmful to the project.
- cbsks 6y agoAs a FreeBSD user, this is great news! I hope that FreeBSD bounces back from this setback and gets stronger than ever.
- chromatin 6y agoContrasting this with another FOSS UNIX alternative is interesting. Although I initially thought the Illumos code review and commit processes (request to integrate; RTI) [0] [1] seemed overly-cumbersome and laborious (and to some extent, perhaps they are still slightly), I now have a much greater appreciation for the safety mechanisms built in. Strictly based on code quality and engineering I wish we could rewrite history, have Sun open up OpenSolaris sooner and faster (and perhaps with a different license) such that Illumos stood as a/the dominant Linux alternative today. But maybe that's just nostalgia talking. Best of luck to FreeBSD going forward; I'm sure this event will lead to even better processes in the future. [0] https://illumos.org/docs/contributing/ https://illumos.org/docs/contributing/ [1] https://illumos.org/docs/contributing/#code-review https://illumos.org/docs/contributing/#code-review
- busterarm 6y agoOpenBSD is surprisingly viable for most things. There's a few stacks it doesn't quite run, but it supports most of what I need to run professionally.
- craftyscrafty 6y agoThis isn't the first time FreeBSD's lack of code review in security-sensitive areas has been put in the spotlight, and probably won't be the last either. https://lists.freebsd.org/pipermail/freebsd-current/2015-February/054580.html https://lists.freebsd.org/pipermail/freebsd-current/2015-Feb... https://old.reddit.com/r/BSD/comments/cid3ud/freebsdsa1912telnet_were_these_claims_even/ https://old.reddit.com/r/BSD/comments/cid3ud/freebsdsa1912te... https://svnweb.freebsd.org/base?view=revision&revision=324394 https://svnweb.freebsd.org/base?view=revision&revision=32439... https://lists.freebsd.org/pipermail/svn-src-head/2018-June/115473.html https://lists.freebsd.org/pipermail/svn-src-head/2018-June/1... https://lists.freebsd.org/pipermail/freebsd-security/2018-January/009740.html https://lists.freebsd.org/pipermail/freebsd-security/2018-Ja...
- pdx6 6y agoI think the important thing is this commit was blocked -- there was QA process that caught it. This statement reads to me that there will be more code review moving forward. The whole situation is embarrassing to the 13 release process, but a lesson was learned rather than a dangerous release being created. I look forward to wg, and I understand the rush to want it out.
- jron 6y agoThis whole fiasco has made me question why I bother with pfsense for my home network. When I only require a simple NAT/port forwarding for torrents, OpenBSD seems like an obvious choice. Other than hardware support, is there another reason why OpenBSD hasn't taken off as a viable home router alternative to pfsense and opnsense? SecurityRouter is the only OpenBSD specific routing appliance that I'm aware of and it is no longer being developed. It also had a closed source backend and unknown licensing for the client.
- nimbius 6y agothere may be reasons you wish to reconsider OpenBSD https://isopenbsdsecu.re/ https://isopenbsdsecu.re/
- jron 6y agoHere is the presentation which provides more context than the slides alone: https://www.youtube.com/watch?v=3E9ga-CylWQ https://www.youtube.com/watch?v=3E9ga-CylWQ I've watched it before and it is compelling; however, at the end of the day, an OS that tries to take security seriously is probably better than one that doesn't. The OpenBSD code is small and I suspect that the defects/KLOC is far less than other projects capable of routing. I'd love to hear more critiques for using it as a home router though.
- rurban 6y agoYes. If you need 10x less performance, you'd choose OpenBSD. Otherwise you'd go for DragonFly
- jron 6y agoI have no doubt that DragonFly could handle higher throughput but for a home router at 1GbE or less, OpenBSD seems ideal assuming it doesn't add additional latency. Benchmarks are hard to find; if you know of any, please let me know.
- kaliszad 6y agoYeah, the whole Spectre/ Meltdown was handled very badly by Intel. Illumos, OpenBSD and others basically found out from the media/ mailing lists of other projects where people were told under NDA. Not a great position to be in, they did, what they could in the shortest time possible.
- _zrlg 6y agoI'm really not sure what the point of this message is. From a PR perspective it is an own goal. It was obvious from earlier statements that self-critique was underway. The right time to send another communication is once some improvement has been made. What is unfortunate is that the project has not publicly defended itself, which is what core should have addressed -- that the situation has been broadly and unfairly misreported. The history of wireguard is bizarre, Donnefeld is hellbent on total control of both the protocol and implementations. He blew the same gasket on NetBSD developers for implementing his protocol http://mail-index.netbsd.org/tech-net/2020/08/22/msg007842.html http://mail-index.netbsd.org/tech-net/2020/08/22/msg007842.h.... The real story here, that the Ars reporter missed because he allowed himself to be compromised by Donnefeld, is how this single person is accumulating and aiming a cult following as he chooses and what are the security and business implications of this in the future. This is a simple tunneling protocol. I don't expect WG to end well. At the very least, you are subject to public zero days and shakedowns if you don't do exactly what Donnefeld wants. A new black hat open source business model of monetization by mob rule and "scooping" low intellect reporters instead of license and implementation.
- ksec 6y agoUnfortunately the NetBSD side's story was not widely known or reported. Otherwise it would give a different perspective to the case here.
- deleted 6y ago[deleted]
- gonzo 6y agoSalter couldn’t even get the CEO’s name right, but he did manage to drag Macy’s spouse into his article. Guy is a muck-racking misogynist.
- tw04 6y agoCurious what the end result was though. It appears they collectively agreed to talk it out in September, then... nothing? Github shows a handful of commits, but nothing major. No chatter on the mailing list. NetBSD hasn't renamed "wireguard" which they said they were considering as an option if Jason was going to keep pushing back on the existing code. The lack of major changes to the code makes me think they also didn't take him up on his offer to scrap the whole thing and start from scratch or port the OpenBSD implementation. Anyone happen to have any further insight?
- alacombe 6y agoThis is standard behavior. When someone publish a +10kloc patch in an area maybe 1 or 2 other developers are competent in, nobody's gonna read it in details. Heck, I probably couldn't even review my code from 1 year ago without lots of metadata/comments attached to it.
- 1-6 6y agoI just love how all communications including official statements from freebsd.org get posted to the web as email threads. Nothing pretty, just utilitarian.