3 ms·

Well, if they didn’t made the trade off between security (manual pull request review) and easy of maintenance (automatic pull request merge), this would have ha

by stjo 5y ago

Well, if they didn’t made the trade off between security (manual pull request review) and easy of maintenance (automatic pull request merge), this would have happened. If instead of parsing git diffs with ruby they used libgit (or something equivalent in how much battle tested it is), this wouldn’t have happened. If Apple provided a more capable package manager, this wouldn’t have happened. If the millions of customers of homebrew (as am I) gave them even a fraction of the monetary resources they need to build such a massive project, this wouldn’t have happened. As you say, there is no single responsible party for the vulnerability. But it is an interesting case too ponder about. If instead of responsibly disclosed, a malicious party decided to use it, this could’ve been very ugly. I hope it gets noticed by the community and better measures are taken.

thu2111 5y ago
Well, it's also just bad engineering tbh. What they're trying to do here is take a workflow intended for humans to make arbitrary code changes and other humans to review those changes, then restrict it to be a workflow for machines to update two keys in a database. Why not just introduce a real API for people to submit new versions instead of hacking one together from tools never meant for this use case? Honestly I use brew and love it but the decision to make everything git and ruby is super hacky. And fundamental architectural decisions like that aren't something you can fix with donations.
- stjo 5y ago
  Maybe, but this would require even more work, time and infrastructure.
- nhoughto 5y ago
  yep is a founding decision, so they can use free git/github infrastructure to run their entire backend, and someone else pays for it. To do what you describe would require hosting something outside of that, which would cost $. Reminds me of at least 1 github engineering blogpost where they reference Homebrew repo(s) as pathological examples, repos that cause them pain to operate effectively because they just don't 'look' like almost anything else. Is what happens when you build your entire architecture around using someone elses free-tier / public service.
  - rmorey 5y ago
    would be interested to see that blog post if you happen to have a link, couldn’t find it googling “github engineering blog homebrew”
    - tingletech 5y ago
      I can't find it either; but the post I remember was from someone from homebrew. IIRC, homebrew used to do a shallow clone (the first time it ran everyday, or something like that) of the repo with all the taps and casks or whatever is in it. A big old git repo. I think github asked them to change homebrew, and they fixed so that it uses the regular git commands vs trying to get tricky and do a shallow clone.
    - zbentley 5y ago
      Not brew, but a very similar set of issues were faced by the GitHub team with the CocoaPods project, which at the time worked similar to Homebrew in that they used github as a CDN/host in a somewhat uncommon way: https://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Post-Mortem/ https://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Po... https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomment-193772935 https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm...
- myolxid1 5y ago
  I'd rather call it smart engineering that makes it easy for anyone to contribute. `brew bump-formula-pr` etc. are huge drivers behind Homebrew having among the largest number of contributors on GitHub. It would help them a lot more if users like us contributed and volunteered to help maintain it. It's crazy that cask has over 100000 PRs and gets a few hundred everyday. For a team of just 20-30, that's insanely hard to manage. Edit: spelling
  - thu2111 5y ago
    They could have a convenient command line submission process for new versions without requiring every such change to be an actual PR. A simple web server that takes a POST request and converts it into a PR behind the scenes would be simpler for users as well, as there's no need then to clutter up your own github account with forks of their repo (or even to have a GH account in the first place).
    - myolxid1 5y ago
      I imagine that having an additional server for a (somewhat understaffed) project run by volunteers introduces additional costs and additional maintenance burden. The current process is already convenient and all you need is a GitHub account. Having the account adds some identity or semblance of it at the very least. Without that? And with an additional server? Seems like a much wider attack surface compared to what's there now.
    - dcow 5y ago
      Thats what their GH action effectively _is_. They just used a poor choice of tool. They could write a program and host it on a web server and it still might screw up.
IgorPartola 5y ago
To be fair, if Apple provided a more capable package manager, you wouldn’t be allowed to publish an HTTP library to it for fear it may make requests to lewd images.
elondaits 5y ago
I wish Apple would provide a package manager... but just like with Linux we’d have to rely on PPAs. If you need to control how / when you update key packages (e.g., you need the latest PHP 7.4 and not 8, or node 12 instead of 15), you can’t use the official distribution... and then you’re still relying on a third party who might not have good security practices.
- rswail 5y ago
  Macports is a package manager that gives you that control, including having multiple versions of packages installed.
- bdamm 5y ago
  Really this is what Docker is all about. If you have environments that need certain preconditions then wrap them up in a container so they don’t break when the system updates.

by stjo 5y ago

thu2111 5y ago

Well, it's also just bad engineering tbh. What they're trying to do here is take a workflow intended for humans to make arbitrary code changes and other humans to review those changes, then restrict it to be a workflow for machines to update two keys in a database. Why not just introduce a real API for people to submit new versions instead of hacking one together from tools never meant for this use case? Honestly I use brew and love it but the decision to make everything git and ruby is super hacky. And fundamental architectural decisions like that aren't something you can fix with donations.

stjo 5y ago
Maybe, but this would require even more work, time and infrastructure.
nhoughto 5y ago
yep is a founding decision, so they can use free git/github infrastructure to run their entire backend, and someone else pays for it. To do what you describe would require hosting something outside of that, which would cost $. Reminds me of at least 1 github engineering blogpost where they reference Homebrew repo(s) as pathological examples, repos that cause them pain to operate effectively because they just don't 'look' like almost anything else. Is what happens when you build your entire architecture around using someone elses free-tier / public service.
- rmorey 5y ago
  would be interested to see that blog post if you happen to have a link, couldn’t find it googling “github engineering blog homebrew”
  - tingletech 5y ago
    I can't find it either; but the post I remember was from someone from homebrew. IIRC, homebrew used to do a shallow clone (the first time it ran everyday, or something like that) of the repo with all the taps and casks or whatever is in it. A big old git repo. I think github asked them to change homebrew, and they fixed so that it uses the regular git commands vs trying to get tricky and do a shallow clone.
  - zbentley 5y ago
    Not brew, but a very similar set of issues were faced by the GitHub team with the CocoaPods project, which at the time worked similar to Homebrew in that they used github as a CDN/host in a somewhat uncommon way: https://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Post-Mortem/ https://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Po... https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomment-193772935 https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomm...
myolxid1 5y ago
I'd rather call it smart engineering that makes it easy for anyone to contribute. `brew bump-formula-pr` etc. are huge drivers behind Homebrew having among the largest number of contributors on GitHub. It would help them a lot more if users like us contributed and volunteered to help maintain it. It's crazy that cask has over 100000 PRs and gets a few hundred everyday. For a team of just 20-30, that's insanely hard to manage. Edit: spelling
- thu2111 5y ago
  They could have a convenient command line submission process for new versions without requiring every such change to be an actual PR. A simple web server that takes a POST request and converts it into a PR behind the scenes would be simpler for users as well, as there's no need then to clutter up your own github account with forks of their repo (or even to have a GH account in the first place).
  - myolxid1 5y ago
    I imagine that having an additional server for a (somewhat understaffed) project run by volunteers introduces additional costs and additional maintenance burden. The current process is already convenient and all you need is a GitHub account. Having the account adds some identity or semblance of it at the very least. Without that? And with an additional server? Seems like a much wider attack surface compared to what's there now.
  - dcow 5y ago
    Thats what their GH action effectively _is_. They just used a poor choice of tool. They could write a program and host it on a web server and it still might screw up.

IgorPartola 5y ago

To be fair, if Apple provided a more capable package manager, you wouldn’t be allowed to publish an HTTP library to it for fear it may make requests to lewd images.

elondaits 5y ago

I wish Apple would provide a package manager... but just like with Linux we’d have to rely on PPAs. If you need to control how / when you update key packages (e.g., you need the latest PHP 7.4 and not 8, or node 12 instead of 15), you can’t use the official distribution... and then you’re still relying on a third party who might not have good security practices.

rswail 5y ago
Macports is a package manager that gives you that control, including having multiple versions of packages installed.
bdamm 5y ago
Really this is what Docker is all about. If you have environments that need certain preconditions then wrap them up in a container so they don’t break when the system updates.