5 ms·

I feel a lot less secure now knowing there is an automated pipeline glued with yaml and ruby, that takes a stranger's code and pushes it directly to my machine.

by stjo 5y ago

I feel a lot less secure now knowing there is an automated pipeline glued with yaml and ruby, that takes a stranger's code and pushes it directly to my machine. I'm sure the homebrew team tried hard to make sure it is as simple and restrictive as possible. As they are doing it for free and without warranty, they are not obligated to match any security standards. I am grateful, but I still wish it is done differently. Maybe instead of armchair complaining on HN I should donate...

aphextron 5y ago
I'm not sure yaml and ruby have anything to do with it. This is a common risk with all centralized package management systems. It's ultimately a tradeoff of trusting a third party for the speed and simplicity. The alternative is building everything from source, or hashing your own binaries. If anything, this is on Apple for the inexplicable fact that they haven't implemented native package management in macOS yet.
- stjo 5y ago
  Well, if they didn’t made the trade off between security (manual pull request review) and easy of maintenance (automatic pull request merge), this would have happened. If instead of parsing git diffs with ruby they used libgit (or something equivalent in how much battle tested it is), this wouldn’t have happened. If Apple provided a more capable package manager, this wouldn’t have happened. If the millions of customers of homebrew (as am I) gave them even a fraction of the monetary resources they need to build such a massive project, this wouldn’t have happened. As you say, there is no single responsible party for the vulnerability. But it is an interesting case too ponder about. If instead of responsibly disclosed, a malicious party decided to use it, this could’ve been very ugly. I hope it gets noticed by the community and better measures are taken.
  - thu2111 5y ago
    Well, it's also just bad engineering tbh. What they're trying to do here is take a workflow intended for humans to make arbitrary code changes and other humans to review those changes, then restrict it to be a workflow for machines to update two keys in a database. Why not just introduce a real API for people to submit new versions instead of hacking one together from tools never meant for this use case? Honestly I use brew and love it but the decision to make everything git and ruby is super hacky. And fundamental architectural decisions like that aren't something you can fix with donations.
    - stjo 5y ago
      Maybe, but this would require even more work, time and infrastructure.
    - nhoughto 5y ago
      yep is a founding decision, so they can use free git/github infrastructure to run their entire backend, and someone else pays for it. To do what you describe would require hosting something outside of that, which would cost $. Reminds me of at least 1 github engineering blogpost where they reference Homebrew repo(s) as pathological examples, repos that cause them pain to operate effectively because they just don't 'look' like almost anything else. Is what happens when you build your entire architecture around using someone elses free-tier / public service.
      - rmorey 5y ago
        would be interested to see that blog post if you happen to have a link, couldn’t find it googling “github engineering blog homebrew”
        tingletech 5y ago
        I can't find it either; but the post I remember was from someone from homebrew. IIRC, homebrew used to do a shallow clone (the first time it ran everyday, or something like that) of the repo with all the taps and casks or whatever is in it. A big old git repo. I think github asked them to change homebrew, and they fixed so that it uses the regular git commands vs trying to get tricky and do a shallow clone.
        zbentley 5y ago
        Not brew, but a very similar set of issues were faced by the GitHub team with the CocoaPods project, which at the time worked similar to Homebrew in that they used github as a CDN/host in a somewhat uncommon way: https://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Post-Mortem/ https://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Po... https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomment-193772935 https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm...
  - IgorPartola 5y ago
    To be fair, if Apple provided a more capable package manager, you wouldn’t be allowed to publish an HTTP library to it for fear it may make requests to lewd images.
  - elondaits 5y ago
    I wish Apple would provide a package manager... but just like with Linux we’d have to rely on PPAs. If you need to control how / when you update key packages (e.g., you need the latest PHP 7.4 and not 8, or node 12 instead of 15), you can’t use the official distribution... and then you’re still relying on a third party who might not have good security practices.
    - rswail 5y ago
      Macports is a package manager that gives you that control, including having multiple versions of packages installed.
    - bdamm 5y ago
      Really this is what Docker is all about. If you have environments that need certain preconditions then wrap them up in a container so they don’t break when the system updates.
- flpa 5y ago
  The exploit isn't a general risk of package management, but a weakness in scripted pull request handling. GitHub is the problem, and not for the first time (remember the Homakov exploit?). Homebrew is quite similar to FreeBSD ports, which do not have these issues. Incidentally, I find that a lot of packages in Homebrew have a very high packaging quality that puts several distributions to shame. I'm saying this as someone who had previously been suspicious of Homebrew, but then I looked at some of the packages.
btown 5y ago
For what it’s worth, Homebrew responded by entirely removing the automation, not just patching the specific vulnerability: https://brew.sh/2021/04/21/security-incident-disclosure/ https://brew.sh/2021/04/21/security-incident-disclosure/ To me this means they take the class of threats seriously. And in the first place, automating version bumps could actually improve availability of security hotfixes for brew-installed software, so it’s consistent with a good faith security stance. And, to be fair, there are plenty of NPM repositories with far fewer qualms about pushing untrusted code in dependencies to dev machines...
- jacquesm 5y ago
  NPM shouldn't be up for serious consideration as part of anything that needs to be secure after their series of incidents like these. If that's the new standard that is problematic.
  - tenacious_tuna 5y ago
    I think it's less about using NPM as a good-practice model, and more just as an example of a widespread tool with worse flaws. Not an excuse, just a comparison.
  - btown 5y ago
    https://github.com/stripe/stripe-js https://github.com/stripe/stripe-js strikes perhaps the most realistic balance possible, recognizing that NPM is an insecure place for their core JS logic that creates a PCI compliant iFrame, and so their NPM package is just a loader for a script tag hosted securely. And yet they encourage people to use NPM for the wrapper itself. Which is just as vulnerable to supply chain attacks as anything else on NPM. If this isn't tacit acknowledgement of a "new standard" I don't know what is. I absolutely agree that it's problematic.
- aequitas 5y ago
  This also makes the default auto update behaviour of Homebrew itself a double edged sword. On the one end it ensures people run into less issues because Homebrew is always up to date but on the other end an RCE like this will be propagated to every user automatically within a really short time.
myolxid1 5y ago
The automated pipeline doesn't exist any more. More than donations I think this requires volunteering to review pull requests, as they mention in the disclosure.

by stjo 5y ago

aphextron 5y ago

I'm not sure yaml and ruby have anything to do with it. This is a common risk with all centralized package management systems. It's ultimately a tradeoff of trusting a third party for the speed and simplicity. The alternative is building everything from source, or hashing your own binaries. If anything, this is on Apple for the inexplicable fact that they haven't implemented native package management in macOS yet.

stjo 5y ago
Well, if they didn’t made the trade off between security (manual pull request review) and easy of maintenance (automatic pull request merge), this would have happened. If instead of parsing git diffs with ruby they used libgit (or something equivalent in how much battle tested it is), this wouldn’t have happened. If Apple provided a more capable package manager, this wouldn’t have happened. If the millions of customers of homebrew (as am I) gave them even a fraction of the monetary resources they need to build such a massive project, this wouldn’t have happened. As you say, there is no single responsible party for the vulnerability. But it is an interesting case too ponder about. If instead of responsibly disclosed, a malicious party decided to use it, this could’ve been very ugly. I hope it gets noticed by the community and better measures are taken.
- thu2111 5y ago
  Well, it's also just bad engineering tbh. What they're trying to do here is take a workflow intended for humans to make arbitrary code changes and other humans to review those changes, then restrict it to be a workflow for machines to update two keys in a database. Why not just introduce a real API for people to submit new versions instead of hacking one together from tools never meant for this use case? Honestly I use brew and love it but the decision to make everything git and ruby is super hacky. And fundamental architectural decisions like that aren't something you can fix with donations.
  - stjo 5y ago
    Maybe, but this would require even more work, time and infrastructure.
  - nhoughto 5y ago
    yep is a founding decision, so they can use free git/github infrastructure to run their entire backend, and someone else pays for it. To do what you describe would require hosting something outside of that, which would cost $. Reminds me of at least 1 github engineering blogpost where they reference Homebrew repo(s) as pathological examples, repos that cause them pain to operate effectively because they just don't 'look' like almost anything else. Is what happens when you build your entire architecture around using someone elses free-tier / public service.
    - rmorey 5y ago
      would be interested to see that blog post if you happen to have a link, couldn’t find it googling “github engineering blog homebrew”
      - tingletech 5y ago
        I can't find it either; but the post I remember was from someone from homebrew. IIRC, homebrew used to do a shallow clone (the first time it ran everyday, or something like that) of the repo with all the taps and casks or whatever is in it. A big old git repo. I think github asked them to change homebrew, and they fixed so that it uses the regular git commands vs trying to get tricky and do a shallow clone.
      - zbentley 5y ago
        Not brew, but a very similar set of issues were faced by the GitHub team with the CocoaPods project, which at the time worked similar to Homebrew in that they used github as a CDN/host in a somewhat uncommon way: https://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Post-Mortem/ https://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Po... https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomment-193772935 https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm...
- IgorPartola 5y ago
  To be fair, if Apple provided a more capable package manager, you wouldn’t be allowed to publish an HTTP library to it for fear it may make requests to lewd images.
- elondaits 5y ago
  I wish Apple would provide a package manager... but just like with Linux we’d have to rely on PPAs. If you need to control how / when you update key packages (e.g., you need the latest PHP 7.4 and not 8, or node 12 instead of 15), you can’t use the official distribution... and then you’re still relying on a third party who might not have good security practices.
  - rswail 5y ago
    Macports is a package manager that gives you that control, including having multiple versions of packages installed.
  - bdamm 5y ago
    Really this is what Docker is all about. If you have environments that need certain preconditions then wrap them up in a container so they don’t break when the system updates.
flpa 5y ago
The exploit isn't a general risk of package management, but a weakness in scripted pull request handling. GitHub is the problem, and not for the first time (remember the Homakov exploit?). Homebrew is quite similar to FreeBSD ports, which do not have these issues. Incidentally, I find that a lot of packages in Homebrew have a very high packaging quality that puts several distributions to shame. I'm saying this as someone who had previously been suspicious of Homebrew, but then I looked at some of the packages.

btown 5y ago

For what it’s worth, Homebrew responded by entirely removing the automation, not just patching the specific vulnerability: https://brew.sh/2021/04/21/security-incident-disclosure/ https://brew.sh/2021/04/21/security-incident-disclosure/ To me this means they take the class of threats seriously. And in the first place, automating version bumps could actually improve availability of security hotfixes for brew-installed software, so it’s consistent with a good faith security stance. And, to be fair, there are plenty of NPM repositories with far fewer qualms about pushing untrusted code in dependencies to dev machines...

jacquesm 5y ago
NPM shouldn't be up for serious consideration as part of anything that needs to be secure after their series of incidents like these. If that's the new standard that is problematic.
- tenacious_tuna 5y ago
  I think it's less about using NPM as a good-practice model, and more just as an example of a widespread tool with worse flaws. Not an excuse, just a comparison.
- btown 5y ago
  https://github.com/stripe/stripe-js https://github.com/stripe/stripe-js strikes perhaps the most realistic balance possible, recognizing that NPM is an insecure place for their core JS logic that creates a PCI compliant iFrame, and so their NPM package is just a loader for a script tag hosted securely. And yet they encourage people to use NPM for the wrapper itself. Which is just as vulnerable to supply chain attacks as anything else on NPM. If this isn't tacit acknowledgement of a "new standard" I don't know what is. I absolutely agree that it's problematic.
aequitas 5y ago
This also makes the default auto update behaviour of Homebrew itself a double edged sword. On the one end it ensures people run into less issues because Homebrew is always up to date but on the other end an RCE like this will be propagated to every user automatically within a really short time.

myolxid1 5y ago

The automated pipeline doesn't exist any more. More than donations I think this requires volunteering to review pull requests, as they mention in the disclosure.