12 ms·

Wow. Netgate come off as incredibly unprofessional. According to the article linked and the info here in that email you linked this is my conclusion: * Netgat

by Jonnax 6y ago

Wow. Netgate come off as incredibly unprofessional. According to the article linked and the info here in that email you linked this is my conclusion: * Netgate tried to ship flawed code that has multiple security issues. * Jason Donenfeld, one of the lead Wireguard developers, went out of his way to work on rewriting it to be better in time for the 13.0 release of FreeBSD * This Netgate employee is angry that they weren't able to ship their bad code and starts throwing accusations of a smear campaign. Am I understanding what happened correctly? Because it really makes this Firewall/Router look really bad.

kbenson 6y ago
That was my impression too, then I went back a couple prior messages, and looked at the earlier announcement. Wihle Netgate looks to have overreacted (at least from the info we have), I can understand why they would be upset. This was in the original announcement: The first step was assessing the current state of the code the previous developer had dumped into the tree. It was not pretty. I imagined strange Internet voices jeering, “this is what gives C a bad name!” There were random sleeps added to “fix” race conditions, validation functions that just returned true, catastrophic cryptographic vulnerabilities, whole parts of the protocol unimplemented, kernel panics, security bypasses, overflows, random printf statements deep in crypto code, the most spectacular buffer overflows, and the whole litany of awful things that go wrong when people aren’t careful when they write C. Or, more simply, it seems typical of what happens when code ships that wasn’t meant to. It was essentially an incomplete half-baked implementation – nothing close to something anybody would want on a production machine. Matt had to talk me out of just insisting they pull the code entirely, and rework it more slowly and carefully for the next release cycle. I can understand being upset if that's how you're portrayed publicly.
- Jonnax 6y ago
  Well if it's true, then they were trying to put flawed code into freebsd which they would then ship to customers in their security product. They're not some random person but are representing their company with their code. If there was a security exploit with their Wireguard implementation, would Netgate get blamed or Wireguard?
- deleted 6y ago
  [deleted]
- ksec 6y ago
  Similar reaction here. My first impression was Netgate being an arse. But then when you read the announcement I kind of understand why Scott is angry. Because while the post may have been in "good faith" in an Open Development and Open Source world, it surely isn't in a professional and business world especially when the work is sponsored ( being paid ). Jason should have informed Netgate the quality of the code is shit in private and FreeBSD dev should have told Netgate will not be shipping any of it in Rel 13. It is then up to Netgate to decide What to do with their Rel 2.5
  - tptacek 6y ago
    WireGuard is an open-source project, and an important one. It seems to me that if you want to push to create the authoritative WireGuard implementation for a major open source OS, the commercial norms need to take a back seat.
  - tw04 6y ago
    > it surely isn't in a professional and business world especially when the work is sponsored ( being paid ). To play devil's advocate: Netgate isn't paying Jason, and they're taking his open source code to create a proprietary commercial project. I'd say Jason owes them exactly nothing in the way of courtesy or consideration. Could he have been more polite for the sake of being polite and community goodwill? Probably.
    - ksec 6y ago
      >and they're taking his open source code to create a proprietary commercial project. I am not sure if that is the case. Netgate seems to have used their old crappy sponsored work for their Pfsense. That is judging from the two pieces of information here. Jason doesn't need to be of consideration for Netgate. There could be other communication we dont know about. I can certainly understand why Scott is frustrated.
      - tw04 6y ago
        >I am not sure if that is the case. Netgate seems to have used their old crappy sponsored work for their Pfsense. Their sponsored work was based off of the Linux and OpenBSD code that Jason and others wrote. And even if it didn't utilize that code, you literally can't write a wireguard client without building on Jason's work.
- tomxor 6y ago
  I duno, if true about the code I find it very difficult to empathize with Netgate From what has been said it's not like they found and fixed a subtle and cryptic vulnerability in an otherwise reasonable implementation and then failed to disclose it properly. It's more like they turned over a rock and found a murder victim. The guy from Netgate is also coming across as very inward looking and seems to assume everyone else's motivations are also purely selfish (referring to his comment implying a "shower of contracts" they might receive for the publicity). His focus should be on how to prevent this mistake from happening in future.
- tw04 6y ago
  Keep in mind, back in February of 2020 when Kip Macy first announced that Netgate had hired him to port Wireguard, Jason offered to help. First Kip declines the offer, then seems to warm slightly to it, but ultimately appears to have not actually engaged Jason. If I'm Jason and I offer my help (for free), they don't take me up on my offer, then try to release code that would make my baby look quite ugly, I would probably also have a pretty severe reaction. Could Jason have been slightly more professional? Absolutely. But we're all human and I can't entirely blame him, I'm sure he was frustrated that he offered to help multiple times and they both didn't take him up on the offer, and tried to release a hatchet job with his name (indirectly) attached to it.
  - CameronNemo 6y ago
    Sounds like Jason should trademark Wireguard (the name). Or build an alternative brand. That way Netgate's actions, or the actions of other wireguard implementations, will not reflect on the reputation of his project/product/technology.
    - tw04 6y ago
      He did trademark the name. I don't think Jason is going to tell the FreeBSD project that they can't use the name "wireguard" for their implementation of "wireguard" just because Netgate put out shoddy code. It's not the FreeBSD project's fault. https://www.wireguard.com/trademark-policy/ https://www.wireguard.com/trademark-policy/
      - kaliszad 6y ago
        I don't think this is an accurate description of the responsibility hierarchy. If in fact the code by Netgate or associates/ contractors/ employees of Netgate is not of professional quality, it has no place in the FreeBSD codebase ready for a stable release. Ultimately, the FreeBSD core community/ developers are together responsible for the codebase even if you cannot hold them legally accountable because of the license. They together hand out and take back (deny) the commit rights (bit, whatever). If a highly sought after component _in the kernel_ is (at least to Jason's account) not even up to the lowest security and code quality standard it has no place in the code base in preparation for a stable or probably even a beta release. Other developers (and that's where Jason is completely in the right if his account is correct) should protest the inclusion of such possibly very bad code into the codebase more or less in late preparations for a release as I understand it. So it is first and foremost on Netgate, if Jason is right but right after that it is on the other responsible FreeBSD co-developers. I mean having so obviously bad code in any kernel of a modern operating system release would be really, really bad. There are many people and companies dependent on it that cannot really influence anything but have to endure the consequences either way. I mean, if you buy a storage appliance, a router or a firewall you trust the quality of the product to a degree and cannot really audit much even if you had the skill. You have to take the word for it and make some reasonable accomodations. No insurance is going the replace the full damage due to lost data to an attacker or a bug. Peoples lives sometimes indirectly depend on the full chain of competence and no insurance can resurrect the dead or right the good name of anybody. Remember, most of the time when you have to update anything for security reasons, somebody didn't understand the system fully or just plain messed up. The only exception is when the problem or times / requirements have changed (e.g. the computers got so fast, we have to transition to longer keys/ passwords whatever). So yeah, if Jason's account is accurate it is bad the code landed in the codebase at all and raises questions about the quality and security of FreeBSD. I mean, it is code directly meant for a secure-as-possible VPN and something that often directly interacts with the open internet. Surely such code should experience extra scrutiny. From the short personal interaction with Jason he came across as quite thoughtful and knowledgeable. Over the years, he and his supporters were able to convince many not so easy to convince people about the quality of Wireguard and some of its implementations. He and the supporters have shown a long term commitment and I am for these reasons inclined to trust Jason's judgement as well.
- axaxs 6y ago
  Yeah, same. Even if all of the above is true, it reads like an elaborate insult. And that's fine if that what the author set out to do for some reason. Pretending it wasn't after the fact isn't being honest, in my opinion. A more professional and neutral announcement could just talk about code that needs to be refactored due to some incompleteness and vulnerabilities.
  - tptacek 6y ago
    It's not an elaborate insult. To a much greater extent than in other security protocols, implementation security is a goal of WireGuard. The protocol itself was designed to support secure kernel implementations; for instance, it's designed in such a way as to not require on-demand dynamic memory allocation. It's part of the premise of the security model of WireGuard that it has secure kernel implementations. If you're building a kernel WireGuard implementation for a major open source OS without taking advantage of the WireGuard implementation design concepts, you're not really building WireGuard; you're building a compatible fork and calling it "WireGuard". The "ask" here from Jason was for everyone to slow their roll, take the flawed WireGuard implementation out of the tree, and give everyone a chance to make it more resilient. Considering the amount of work Jason had to go through to get WireGuard into the Linux tree, that seems like a very reasonable request. Instead, the WireGuard project seems to have been put into a position where they had to scramble to fix up an implementation that was being pushed into FreeBSD, as WireGuard qua WireGuard. I can imagine that being a frustrating experience. It certainly didn't generate the most political response ever, but I think you'd be reaching to call it a deliberate insult.
    - justaj 6y ago
      If the criticism is about the code instead of the person, it should never be interpreted as an insult. Proper developers have learned to split their ideas from their ego, and as such are able to receive harsh yet justified criticism concerning their code without being offended.
    - kbenson 6y ago
      > It's not an elaborate insult. My read on it wasn't that it was an elaborate insult, but more that it was far more denigrating than it needed to be, if he was trying to be professional. That doesn't mean it was purposeful, sometimes people just don't really associate the statements they make with how it may be perceived. I think it could have been communicated clearly and succinctly with something along the lines of: "The first step was assessing the current state of the code the previous developer had dumped into the tree. We noticed some quality problems, some unimplemented protocol sections and more concerning, security issues with the code. Given these issues, we considered asking they remove the code, but instead Matt convinced me that we should rework it slowly and carefully for the next release cycle." Notably, I think omission of the following inflammatory statements would have prevented a lot of problems: - "It was not pretty." - "I imagined strange Internet voices jeering, “this is what gives C a bad name!”" - "the most spectacular buffer overflows" - "the whole litany of awful things that go wrong when people aren’t careful when they write C." Whether those entirely subjective statements are accurate, they are not the things you say about someone else's work output when you expect a useful dialogue with them, which is exactly why they are considered unprofessional. I'm not defending Netgate's code here, or even the vehemence of their reaction and how they went about it, but merely noting that not only can I see how it devolved into this, I would go so far as to say it's obvious that this is why that type of language is avoided by most people trying to work professionally. Jason wrote some very unkind things, and Netgate blew up about it. There's enough blame here that they can both share some. > The "ask" here from Jason was for everyone to slow their roll, take the flawed WireGuard implementation out of the tree, and give everyone a chance to make it more resilient. Considering the amount of work Jason had to go through to get WireGuard into the Linux tree, that seems like a very reasonable request. Err, wasn't that actually not the ask, because he thought they wouldn't do so, so instead they worked it over in a short time-frame, only for it then to be removed when this argument broke out and it came to light?
- 1vuio0pswjnm7 6y ago
  Reminded me of the type of statements he made last year on another set of mailing lists: https://news.ycombinator.com/item?id=24430424 https://news.ycombinator.com/item?id=24430424 https://mail-index.netbsd.org/tech-net/2020/08/22/msg007842.html https://mail-index.netbsd.org/tech-net/2020/08/22/msg007842.... https://mail-index.NetBSD.org/current-users/2020/08/22/msg039406.html https://mail-index.NetBSD.org/current-users/2020/08/22/msg03... https://mail-index.NetBSD.org/tech-kern/2020/08/23/msg026693.html https://mail-index.NetBSD.org/tech-kern/2020/08/23/msg026693...
  - secondcoming 6y ago
    I don't think he intends to be patronising, but he certainly comes off that way.
    - kaliszad 6y ago
      I mean, Wireguard in these cases is a piece of code _in the kernel_, is meant to be directly in the path of a packet coming from an untrusted network and potentially does cryptography/ security critical stuff. Besides maybe memory management and such things, with what other pieces of code do you care more about quality and security than basically a VPN? People and companies that don't even know what Wireguard or a VPN for that matter is directly or indirectly depend on this and similar code being nearly perfect sometimes with their reputations or even lives. I mean, it is not a game that is taking too long to load or something, which is quite upsetting but I cannot imagine how that would be critical. The security of a kernel or a VPN is a very different story and we should all stop pretending this isn't the case. I hope (and am quite convinced) Jason and others want only the best for the respective kernels and the Wireguard implementation from the standpoint of quality and security.
stonogo 6y ago
NetGate spends a lot on FreeBSD development, which is great, but they also spend a lot of time running smear campaigns against people who offend them, which is ridiculous. They even started /r/opnsense on Reddit just to post shit-talking memes, and camp on the namespace to this day.

by Jonnax 6y ago

kbenson 6y ago

That was my impression too, then I went back a couple prior messages, and looked at the earlier announcement. Wihle Netgate looks to have overreacted (at least from the info we have), I can understand why they would be upset. This was in the original announcement: The first step was assessing the current state of the code the previous developer had dumped into the tree. It was not pretty. I imagined strange Internet voices jeering, “this is what gives C a bad name!” There were random sleeps added to “fix” race conditions, validation functions that just returned true, catastrophic cryptographic vulnerabilities, whole parts of the protocol unimplemented, kernel panics, security bypasses, overflows, random printf statements deep in crypto code, the most spectacular buffer overflows, and the whole litany of awful things that go wrong when people aren’t careful when they write C. Or, more simply, it seems typical of what happens when code ships that wasn’t meant to. It was essentially an incomplete half-baked implementation – nothing close to something anybody would want on a production machine. Matt had to talk me out of just insisting they pull the code entirely, and rework it more slowly and carefully for the next release cycle. I can understand being upset if that's how you're portrayed publicly.

Jonnax 6y ago
Well if it's true, then they were trying to put flawed code into freebsd which they would then ship to customers in their security product. They're not some random person but are representing their company with their code. If there was a security exploit with their Wireguard implementation, would Netgate get blamed or Wireguard?
deleted 6y ago
[deleted]
ksec 6y ago
Similar reaction here. My first impression was Netgate being an arse. But then when you read the announcement I kind of understand why Scott is angry. Because while the post may have been in "good faith" in an Open Development and Open Source world, it surely isn't in a professional and business world especially when the work is sponsored ( being paid ). Jason should have informed Netgate the quality of the code is shit in private and FreeBSD dev should have told Netgate will not be shipping any of it in Rel 13. It is then up to Netgate to decide What to do with their Rel 2.5
- tptacek 6y ago
  WireGuard is an open-source project, and an important one. It seems to me that if you want to push to create the authoritative WireGuard implementation for a major open source OS, the commercial norms need to take a back seat.
- tw04 6y ago
  > it surely isn't in a professional and business world especially when the work is sponsored ( being paid ). To play devil's advocate: Netgate isn't paying Jason, and they're taking his open source code to create a proprietary commercial project. I'd say Jason owes them exactly nothing in the way of courtesy or consideration. Could he have been more polite for the sake of being polite and community goodwill? Probably.
  - ksec 6y ago
    >and they're taking his open source code to create a proprietary commercial project. I am not sure if that is the case. Netgate seems to have used their old crappy sponsored work for their Pfsense. That is judging from the two pieces of information here. Jason doesn't need to be of consideration for Netgate. There could be other communication we dont know about. I can certainly understand why Scott is frustrated.
    - tw04 6y ago
      >I am not sure if that is the case. Netgate seems to have used their old crappy sponsored work for their Pfsense. Their sponsored work was based off of the Linux and OpenBSD code that Jason and others wrote. And even if it didn't utilize that code, you literally can't write a wireguard client without building on Jason's work.
tomxor 6y ago
I duno, if true about the code I find it very difficult to empathize with Netgate From what has been said it's not like they found and fixed a subtle and cryptic vulnerability in an otherwise reasonable implementation and then failed to disclose it properly. It's more like they turned over a rock and found a murder victim. The guy from Netgate is also coming across as very inward looking and seems to assume everyone else's motivations are also purely selfish (referring to his comment implying a "shower of contracts" they might receive for the publicity). His focus should be on how to prevent this mistake from happening in future.
tw04 6y ago
Keep in mind, back in February of 2020 when Kip Macy first announced that Netgate had hired him to port Wireguard, Jason offered to help. First Kip declines the offer, then seems to warm slightly to it, but ultimately appears to have not actually engaged Jason. If I'm Jason and I offer my help (for free), they don't take me up on my offer, then try to release code that would make my baby look quite ugly, I would probably also have a pretty severe reaction. Could Jason have been slightly more professional? Absolutely. But we're all human and I can't entirely blame him, I'm sure he was frustrated that he offered to help multiple times and they both didn't take him up on the offer, and tried to release a hatchet job with his name (indirectly) attached to it.
- CameronNemo 6y ago
  Sounds like Jason should trademark Wireguard (the name). Or build an alternative brand. That way Netgate's actions, or the actions of other wireguard implementations, will not reflect on the reputation of his project/product/technology.
  - tw04 6y ago
    He did trademark the name. I don't think Jason is going to tell the FreeBSD project that they can't use the name "wireguard" for their implementation of "wireguard" just because Netgate put out shoddy code. It's not the FreeBSD project's fault. https://www.wireguard.com/trademark-policy/ https://www.wireguard.com/trademark-policy/
    - kaliszad 6y ago
      I don't think this is an accurate description of the responsibility hierarchy. If in fact the code by Netgate or associates/ contractors/ employees of Netgate is not of professional quality, it has no place in the FreeBSD codebase ready for a stable release. Ultimately, the FreeBSD core community/ developers are together responsible for the codebase even if you cannot hold them legally accountable because of the license. They together hand out and take back (deny) the commit rights (bit, whatever). If a highly sought after component _in the kernel_ is (at least to Jason's account) not even up to the lowest security and code quality standard it has no place in the code base in preparation for a stable or probably even a beta release. Other developers (and that's where Jason is completely in the right if his account is correct) should protest the inclusion of such possibly very bad code into the codebase more or less in late preparations for a release as I understand it. So it is first and foremost on Netgate, if Jason is right but right after that it is on the other responsible FreeBSD co-developers. I mean having so obviously bad code in any kernel of a modern operating system release would be really, really bad. There are many people and companies dependent on it that cannot really influence anything but have to endure the consequences either way. I mean, if you buy a storage appliance, a router or a firewall you trust the quality of the product to a degree and cannot really audit much even if you had the skill. You have to take the word for it and make some reasonable accomodations. No insurance is going the replace the full damage due to lost data to an attacker or a bug. Peoples lives sometimes indirectly depend on the full chain of competence and no insurance can resurrect the dead or right the good name of anybody. Remember, most of the time when you have to update anything for security reasons, somebody didn't understand the system fully or just plain messed up. The only exception is when the problem or times / requirements have changed (e.g. the computers got so fast, we have to transition to longer keys/ passwords whatever). So yeah, if Jason's account is accurate it is bad the code landed in the codebase at all and raises questions about the quality and security of FreeBSD. I mean, it is code directly meant for a secure-as-possible VPN and something that often directly interacts with the open internet. Surely such code should experience extra scrutiny. From the short personal interaction with Jason he came across as quite thoughtful and knowledgeable. Over the years, he and his supporters were able to convince many not so easy to convince people about the quality of Wireguard and some of its implementations. He and the supporters have shown a long term commitment and I am for these reasons inclined to trust Jason's judgement as well.
axaxs 6y ago
Yeah, same. Even if all of the above is true, it reads like an elaborate insult. And that's fine if that what the author set out to do for some reason. Pretending it wasn't after the fact isn't being honest, in my opinion. A more professional and neutral announcement could just talk about code that needs to be refactored due to some incompleteness and vulnerabilities.
- tptacek 6y ago
  It's not an elaborate insult. To a much greater extent than in other security protocols, implementation security is a goal of WireGuard. The protocol itself was designed to support secure kernel implementations; for instance, it's designed in such a way as to not require on-demand dynamic memory allocation. It's part of the premise of the security model of WireGuard that it has secure kernel implementations. If you're building a kernel WireGuard implementation for a major open source OS without taking advantage of the WireGuard implementation design concepts, you're not really building WireGuard; you're building a compatible fork and calling it "WireGuard". The "ask" here from Jason was for everyone to slow their roll, take the flawed WireGuard implementation out of the tree, and give everyone a chance to make it more resilient. Considering the amount of work Jason had to go through to get WireGuard into the Linux tree, that seems like a very reasonable request. Instead, the WireGuard project seems to have been put into a position where they had to scramble to fix up an implementation that was being pushed into FreeBSD, as WireGuard qua WireGuard. I can imagine that being a frustrating experience. It certainly didn't generate the most political response ever, but I think you'd be reaching to call it a deliberate insult.
  - justaj 6y ago
    If the criticism is about the code instead of the person, it should never be interpreted as an insult. Proper developers have learned to split their ideas from their ego, and as such are able to receive harsh yet justified criticism concerning their code without being offended.
  - kbenson 6y ago
    > It's not an elaborate insult. My read on it wasn't that it was an elaborate insult, but more that it was far more denigrating than it needed to be, if he was trying to be professional. That doesn't mean it was purposeful, sometimes people just don't really associate the statements they make with how it may be perceived. I think it could have been communicated clearly and succinctly with something along the lines of: "The first step was assessing the current state of the code the previous developer had dumped into the tree. We noticed some quality problems, some unimplemented protocol sections and more concerning, security issues with the code. Given these issues, we considered asking they remove the code, but instead Matt convinced me that we should rework it slowly and carefully for the next release cycle." Notably, I think omission of the following inflammatory statements would have prevented a lot of problems: - "It was not pretty." - "I imagined strange Internet voices jeering, “this is what gives C a bad name!”" - "the most spectacular buffer overflows" - "the whole litany of awful things that go wrong when people aren’t careful when they write C." Whether those entirely subjective statements are accurate, they are not the things you say about someone else's work output when you expect a useful dialogue with them, which is exactly why they are considered unprofessional. I'm not defending Netgate's code here, or even the vehemence of their reaction and how they went about it, but merely noting that not only can I see how it devolved into this, I would go so far as to say it's obvious that this is why that type of language is avoided by most people trying to work professionally. Jason wrote some very unkind things, and Netgate blew up about it. There's enough blame here that they can both share some. > The "ask" here from Jason was for everyone to slow their roll, take the flawed WireGuard implementation out of the tree, and give everyone a chance to make it more resilient. Considering the amount of work Jason had to go through to get WireGuard into the Linux tree, that seems like a very reasonable request. Err, wasn't that actually not the ask, because he thought they wouldn't do so, so instead they worked it over in a short time-frame, only for it then to be removed when this argument broke out and it came to light?
1vuio0pswjnm7 6y ago
Reminded me of the type of statements he made last year on another set of mailing lists: https://news.ycombinator.com/item?id=24430424 https://news.ycombinator.com/item?id=24430424 https://mail-index.netbsd.org/tech-net/2020/08/22/msg007842.html https://mail-index.netbsd.org/tech-net/2020/08/22/msg007842.... https://mail-index.NetBSD.org/current-users/2020/08/22/msg039406.html https://mail-index.NetBSD.org/current-users/2020/08/22/msg03... https://mail-index.NetBSD.org/tech-kern/2020/08/23/msg026693.html https://mail-index.NetBSD.org/tech-kern/2020/08/23/msg026693...
- secondcoming 6y ago
  I don't think he intends to be patronising, but he certainly comes off that way.
  - kaliszad 6y ago
    I mean, Wireguard in these cases is a piece of code _in the kernel_, is meant to be directly in the path of a packet coming from an untrusted network and potentially does cryptography/ security critical stuff. Besides maybe memory management and such things, with what other pieces of code do you care more about quality and security than basically a VPN? People and companies that don't even know what Wireguard or a VPN for that matter is directly or indirectly depend on this and similar code being nearly perfect sometimes with their reputations or even lives. I mean, it is not a game that is taking too long to load or something, which is quite upsetting but I cannot imagine how that would be critical. The security of a kernel or a VPN is a very different story and we should all stop pretending this isn't the case. I hope (and am quite convinced) Jason and others want only the best for the respective kernels and the Wireguard implementation from the standpoint of quality and security.

stonogo 6y ago

NetGate spends a lot on FreeBSD development, which is great, but they also spend a lot of time running smear campaigns against people who offend them, which is ridiculous. They even started /r/opnsense on Reddit just to post shit-talking memes, and camp on the namespace to this day.