6 ms·

I understand what you are saying and for open source community to have a good reputation and keep its standards high, these important packages have to start bei

I understand what you are saying and for open source community to have a good reputation and keep its standards high, these important packages have to start being actually maintained by the enterprises that depend on these. Most of businesses use Linux for their servers mainly Ubuntu/Debian and its just crazy for them to be vulnerable in such ways. Imagine how many other places backdoors can be installed even right now, and many of them won’t be visible and wont have performance issues like this one. 0.5s is a big deal in software and if it was 50ms or 5ms no one would have known this possibly for years. I wonder how safe the software we use actually is.

ctrw 3y ago
If you want professional developers pay professional rates. A doctor asks for a hundred bucks for 15 min.
- michelsedgh 3y ago
  Yes and people are paying top money to Canonical and RedHat to maintain these software don’t you think? They have huge enterprise contracts and I feel like this should be on them? Im not blaming anyone it just goes to show how vulnerable they are. Also other linux distros, Fedora, arch, etc. they have the money and should make their software safe don’t you think?
  - Macha 3y ago
    Fedora is supported by Red Hat. And yes, I do think there is something to the fact that Red Hat is selling businesses commercial support for projects they have no expertise in, which probably means some of the pressure flows downwards from Red Hat's clients to the unpaid maintainers. Arch, (Gentoo, Nix, Mint, etc.) have no commercial aspect on the other hand and certainly doesn't have the money, but those distros are offered on the same "as is" basis as the downstream software.
  - Brian_K_White 3y ago
    Yes and haven't Canonical and RedHat and others already reacted and their users are fine? I see no problem.
    - michelsedgh 3y ago
      First, the problem is the reputation hit and the trust going away. Right now on Hacker News 2 maybe 3 of the posts on front page have been about this since the news broke out. Also when trust goes away, its very hard to come back. Secondly, the main problem that I see is how many other backdoors/dependencies are vulnerable that we might not know? They might not have performance issues. Also if this went unnoticed, in the long term it might have found its way and actually compromised people. I’m glad this was taken care of and no one was compromised.
      - Brian_K_White 3y ago
        Is there some implied proposal I missed?
        fsflover 3y ago
        https://news.ycombinator.com/item?id=39904034 https://news.ycombinator.com/item?id=39904034
        fmajid 3y ago
        OpenSSH is developed by the OpenBSD project and I have a lot more confidence in them than in any random "enterprise". The issue in this case is that Linux distros took it upon themselves to alter its code base by linking in libsystemd (thus also liblzma) for the dubious benefit of better systemd integration, which comes with a generous helping of attack surface.
  - WesolyKubeczek 3y ago
    > Yes and people are paying top money to Canonical and RedHat to maintain these software don’t you think? They have huge enterprise contracts Money alone makes poor filler material to patch such holes. Microsoft famously doesn't make their stuff for free, and yet somehow Windows systems have gained this reputation of having holes such that a train can go through. People build trust, take shortcuts, cut corners all the time. It's how we are wired. Doing otherwise is an energy sink for our bodies. We don't make ourselves overspend energy when we can.
  - ctrw 3y ago
    I'd hardly call a $400 per year per seat contract top dollar. There is software out there that is both much worse and costs in the tens to hundreds of thousands per seat per year.
    - fmajid 3y ago
      There are economies of scale for widely-used software.
      - ctrw 3y ago
        Reality disagrees. Software isn't tooth brushes. The simpler and more specialized it is the cheaper it is to maintain. The more complex it is the harder it is to maintain.
        fmajid 3y ago
        Software R&D is a fixed cost. An ERP package with its at most 10,000 installations amortizes that cost on far fewer users than Linux with its millions of licensed installs.
  - indymike 3y ago
    > Also other linux distros, Fedora, arch, etc. they have the money and should make their software safe don’t you think? Nope. With open source the user is responsible. It's been this way from day one. Come to think of it, almost every piece of software I've ever used that is commercial has a no warranties and not suited for purpose clause in the EULA... because all the components the vendor used have that clause in the license to the vendor.
- VancouverMan 3y ago
  There's no connection between how much money somebody charges for their services and the quality of the services that they in turn provide. Much of the worst professional service I've ever received, despite paying them a lot of money, has been from the example you gave, doctors. Doctors aren't alone, of course. The most expensive restaurants I've ever been to have been some of the worst dining experiences I've ever had, while friends or relatives inviting me over for a backyard barbecue have been among the best, for example. I've also found this to be the case for lawyers, accountants, teachers/professors, mechanics, and other well-compensated "professionals". Software hasn't been an exception, either. I've had more success with free software than I have had with the equivalent paid offerings many times over. In general, I've found that service quality is far more tied to the provider's level of passion for the task at hand, rather than anything to do with the amount of financial compensation that may be involved.
madeofpalk 3y ago
What are you actually proposing here? Any Ubuntu/Debian distribution contains hundreds or thousands of packages and transitive dependencies - it's unreasonable to expect corporations to contribute, even financially, to all of these. This is supposed to be what services such as Tidelift provide - companies pay them, and OSS maintainers sign up with them for funding (and other support? unsure). But it's tough to apply this through the whole OSS ecosystem. To me, this should be what Canonical's support contracts should be for. They're perfectly positioned to help mediate these types of issues.
- michelsedgh 3y ago
  Im proposing that high security packages like this that effect ssh and main parts of the OS critical for security and safety should be maintained by actual enterprises not by one developer. Im not saying they should maintain everything, only the high importance parts of it. They should make sure that one bad actor cant compromise the whole ecosystem. I really don’t think what im saying is that far out of reach or that crazy. Edit: also, why should a package like ssh have so many random dependencies? They should make sure these kind of packages have the least of dependencies especially on these kind of packages that have one maintainer to reduce the security risks?
  - em-bee 3y ago
    why should a package like ssh have so many random dependencies that's the critical question here. it's not that enterprises need to take over maintenance of xz, but that the critical packages and their dependencies need to be audited on a regular basis. the development of xz is ok. what it needs is help with code reviews. and if it can't receive those, then it needs to be removed as a dependency of openssh.
    - michelsedgh 3y ago
      Yes, exactly, these high security packages that every distro uses must be very tightly maintained and looked over with the least dependencies. SSH is one of the most important packages in security and it just feels clumsy that they are using such packages like XZ. Just WHY? Who made that decision? Why doesn’t all the distros work together maybe to maintain the security? Why depend on one person that does this as a hobby?
npteljes 3y ago
The software we use is not safe at all. If we don't have an idea what goes on in an open source projects, how could we even begin to know what goes on in a proprietary binary, or a service? People's dealings operate on trust, that's how, and trust can and will be abused, simple as that. >Most of businesses use Linux for their servers mainly Ubuntu/Debian Businesses should calculate with the risks, and also, use Long Term Support versions of software, not unstable bleeding edge fresh from the oven ones. Which they do. And so they were not vulnerable at all.
bdd8f1df777b 3y ago
Well, I'm certain Facebook would be glad if xz is replaced everywhere by zstd which they maintain. You can't force them to pay for xz when they already maintain a replacement.
cmiles74 3y ago
These open source projects have a good reputation and high standards because the developers freely volunteering their time care about the project. Forcing these developers to give their projects to Big Corp may not be the big improvement you are imagining. Software may have backdoors, I'm not convinced this couldn't have happened if a large company was managing the product. AFAICT, open source software has a better security track record, in general. IMHO, if it is anyone's job to look for and prevent these kinds of backdoors, it's probably companies like RedHat and Canonical. They bundle the tools with their products and they charge money for support.
- daymanstep 3y ago
  Backdoor detection is an unsolvable problem both in theory and in practice. We are talking about carefully hidden obfuscated backdoors in MLOC codebases, which will have unexplained binaries and arcane build scripts. No company, not even Google, can guarantee that they'll be able to catch every backdoor hidden in some arbitrary codebase. The only practical solution is for these companies to rewrite the core packages from scratch, written from the ground up so that they are easy to audit - no unexplained binaries, no arcane build scripts, etc.
- mdedetrich 3y ago
  > AFAICT, open source software has a better security track record, in general. iirc Studies was done on this and there was no measurable difference in security issues when it comes to open vs closed source. Just because the code is visible doesn't mean much, as you need to have the right eyes to actually notice security issues and a lot of open source projects don't have this. Also closed source projects have the advantage where if they have a massive company behind them, that company has the resources (i.e. $$$) to hire highly specialised people to look the security of the software. Open source projects usually don't have such resources, heartbleed and xz are indicative of that.
deleted 3y ago
[deleted]
fmajid 3y ago
It's not the maintenance of the projects the enterprises have to fund (though that is certainly welcome if the mantainers are willing) but the slow, painful work of vetting the code. Google does that with Project Zero but few companies are wealthy enough to afford that. The way out is economic, not technical: insurance, and mutualizing the cost of security audits. I wrote up my ideas on the subject here: https://blog.majid.info/supply-chain-vetting/ https://blog.majid.info/supply-chain-vetting/

ctrw 3y ago

If you want professional developers pay professional rates. A doctor asks for a hundred bucks for 15 min.

michelsedgh 3y ago
Yes and people are paying top money to Canonical and RedHat to maintain these software don’t you think? They have huge enterprise contracts and I feel like this should be on them? Im not blaming anyone it just goes to show how vulnerable they are. Also other linux distros, Fedora, arch, etc. they have the money and should make their software safe don’t you think?
- Macha 3y ago
  Fedora is supported by Red Hat. And yes, I do think there is something to the fact that Red Hat is selling businesses commercial support for projects they have no expertise in, which probably means some of the pressure flows downwards from Red Hat's clients to the unpaid maintainers. Arch, (Gentoo, Nix, Mint, etc.) have no commercial aspect on the other hand and certainly doesn't have the money, but those distros are offered on the same "as is" basis as the downstream software.
- Brian_K_White 3y ago
  Yes and haven't Canonical and RedHat and others already reacted and their users are fine? I see no problem.
  - michelsedgh 3y ago
    First, the problem is the reputation hit and the trust going away. Right now on Hacker News 2 maybe 3 of the posts on front page have been about this since the news broke out. Also when trust goes away, its very hard to come back. Secondly, the main problem that I see is how many other backdoors/dependencies are vulnerable that we might not know? They might not have performance issues. Also if this went unnoticed, in the long term it might have found its way and actually compromised people. I’m glad this was taken care of and no one was compromised.
    - Brian_K_White 3y ago
      Is there some implied proposal I missed?
      - fsflover 3y ago
        https://news.ycombinator.com/item?id=39904034 https://news.ycombinator.com/item?id=39904034
        fmajid 3y ago
        OpenSSH is developed by the OpenBSD project and I have a lot more confidence in them than in any random "enterprise". The issue in this case is that Linux distros took it upon themselves to alter its code base by linking in libsystemd (thus also liblzma) for the dubious benefit of better systemd integration, which comes with a generous helping of attack surface.
- WesolyKubeczek 3y ago
  > Yes and people are paying top money to Canonical and RedHat to maintain these software don’t you think? They have huge enterprise contracts Money alone makes poor filler material to patch such holes. Microsoft famously doesn't make their stuff for free, and yet somehow Windows systems have gained this reputation of having holes such that a train can go through. People build trust, take shortcuts, cut corners all the time. It's how we are wired. Doing otherwise is an energy sink for our bodies. We don't make ourselves overspend energy when we can.
- ctrw 3y ago
  I'd hardly call a $400 per year per seat contract top dollar. There is software out there that is both much worse and costs in the tens to hundreds of thousands per seat per year.
  - fmajid 3y ago
    There are economies of scale for widely-used software.
    - ctrw 3y ago
      Reality disagrees. Software isn't tooth brushes. The simpler and more specialized it is the cheaper it is to maintain. The more complex it is the harder it is to maintain.
      - fmajid 3y ago
        Software R&D is a fixed cost. An ERP package with its at most 10,000 installations amortizes that cost on far fewer users than Linux with its millions of licensed installs.
- indymike 3y ago
  > Also other linux distros, Fedora, arch, etc. they have the money and should make their software safe don’t you think? Nope. With open source the user is responsible. It's been this way from day one. Come to think of it, almost every piece of software I've ever used that is commercial has a no warranties and not suited for purpose clause in the EULA... because all the components the vendor used have that clause in the license to the vendor.
VancouverMan 3y ago
There's no connection between how much money somebody charges for their services and the quality of the services that they in turn provide. Much of the worst professional service I've ever received, despite paying them a lot of money, has been from the example you gave, doctors. Doctors aren't alone, of course. The most expensive restaurants I've ever been to have been some of the worst dining experiences I've ever had, while friends or relatives inviting me over for a backyard barbecue have been among the best, for example. I've also found this to be the case for lawyers, accountants, teachers/professors, mechanics, and other well-compensated "professionals". Software hasn't been an exception, either. I've had more success with free software than I have had with the equivalent paid offerings many times over. In general, I've found that service quality is far more tied to the provider's level of passion for the task at hand, rather than anything to do with the amount of financial compensation that may be involved.

madeofpalk 3y ago

What are you actually proposing here? Any Ubuntu/Debian distribution contains hundreds or thousands of packages and transitive dependencies - it's unreasonable to expect corporations to contribute, even financially, to all of these. This is supposed to be what services such as Tidelift provide - companies pay them, and OSS maintainers sign up with them for funding (and other support? unsure). But it's tough to apply this through the whole OSS ecosystem. To me, this should be what Canonical's support contracts should be for. They're perfectly positioned to help mediate these types of issues.

michelsedgh 3y ago
Im proposing that high security packages like this that effect ssh and main parts of the OS critical for security and safety should be maintained by actual enterprises not by one developer. Im not saying they should maintain everything, only the high importance parts of it. They should make sure that one bad actor cant compromise the whole ecosystem. I really don’t think what im saying is that far out of reach or that crazy. Edit: also, why should a package like ssh have so many random dependencies? They should make sure these kind of packages have the least of dependencies especially on these kind of packages that have one maintainer to reduce the security risks?
- em-bee 3y ago
  why should a package like ssh have so many random dependencies that's the critical question here. it's not that enterprises need to take over maintenance of xz, but that the critical packages and their dependencies need to be audited on a regular basis. the development of xz is ok. what it needs is help with code reviews. and if it can't receive those, then it needs to be removed as a dependency of openssh.
  - michelsedgh 3y ago
    Yes, exactly, these high security packages that every distro uses must be very tightly maintained and looked over with the least dependencies. SSH is one of the most important packages in security and it just feels clumsy that they are using such packages like XZ. Just WHY? Who made that decision? Why doesn’t all the distros work together maybe to maintain the security? Why depend on one person that does this as a hobby?

npteljes 3y ago

The software we use is not safe at all. If we don't have an idea what goes on in an open source projects, how could we even begin to know what goes on in a proprietary binary, or a service? People's dealings operate on trust, that's how, and trust can and will be abused, simple as that. >Most of businesses use Linux for their servers mainly Ubuntu/Debian Businesses should calculate with the risks, and also, use Long Term Support versions of software, not unstable bleeding edge fresh from the oven ones. Which they do. And so they were not vulnerable at all.

bdd8f1df777b 3y ago

Well, I'm certain Facebook would be glad if xz is replaced everywhere by zstd which they maintain. You can't force them to pay for xz when they already maintain a replacement.

cmiles74 3y ago

These open source projects have a good reputation and high standards because the developers freely volunteering their time care about the project. Forcing these developers to give their projects to Big Corp may not be the big improvement you are imagining. Software may have backdoors, I'm not convinced this couldn't have happened if a large company was managing the product. AFAICT, open source software has a better security track record, in general. IMHO, if it is anyone's job to look for and prevent these kinds of backdoors, it's probably companies like RedHat and Canonical. They bundle the tools with their products and they charge money for support.

daymanstep 3y ago
Backdoor detection is an unsolvable problem both in theory and in practice. We are talking about carefully hidden obfuscated backdoors in MLOC codebases, which will have unexplained binaries and arcane build scripts. No company, not even Google, can guarantee that they'll be able to catch every backdoor hidden in some arbitrary codebase. The only practical solution is for these companies to rewrite the core packages from scratch, written from the ground up so that they are easy to audit - no unexplained binaries, no arcane build scripts, etc.
mdedetrich 3y ago
> AFAICT, open source software has a better security track record, in general. iirc Studies was done on this and there was no measurable difference in security issues when it comes to open vs closed source. Just because the code is visible doesn't mean much, as you need to have the right eyes to actually notice security issues and a lot of open source projects don't have this. Also closed source projects have the advantage where if they have a massive company behind them, that company has the resources (i.e. $$$) to hire highly specialised people to look the security of the software. Open source projects usually don't have such resources, heartbleed and xz are indicative of that.

deleted 3y ago

[deleted]

fmajid 3y ago

It's not the maintenance of the projects the enterprises have to fund (though that is certainly welcome if the mantainers are willing) but the slow, painful work of vetting the code. Google does that with Project Zero but few companies are wealthy enough to afford that. The way out is economic, not technical: insurance, and mutualizing the cost of security audits. I wrote up my ideas on the subject here: https://blog.majid.info/supply-chain-vetting/ https://blog.majid.info/supply-chain-vetting/