4 ms·

> brew update will now be run automatically less often (every 24 hours rather than every 5 minutes) and these auto-updates will be much faster as they no longer

> brew update will now be run automatically less often (every 24 hours rather than every 5 minutes) and these auto-updates will be much faster as they no longer need to perform the slow git fetch of the huge homebrew/core and homebrew/cask taps’ Git repositories. Excellent! Thanks to the brew team for this — it was always frustrating to install something and have to wait for a full, slow update first.

febstar 4y ago
> no longer need to perform the slow git fetch Nice. Rust Cargo is also trialling out something similar.
atonse 4y ago
I was just wondering, have they considered storing all that in an SQLite file? And then you could refresh the file every 6 hours, say, and post it on a CDN?
- modernerd 4y ago
  Using SQLite instead of JSON could have worked too and would likely make the manifests smaller: > curl -sI https://formulae.brew.sh/api/formula.json https://formulae.brew.sh/api/formula.json | grep content-length => 19898457 (~19.9MB) > curl -sI https://formulae.brew.sh/api/cask.json https://formulae.brew.sh/api/cask.json | grep content-length => 4023930 (~4MB) But the JSON API has been around for a while so brew 4.0 just makes use of it by default. Brew 3.3 added an off-by-default HOMEBREW_INSTALL_FROM_API bool to let you install from brew's JSON-based API instead of checking out the large (and slow) homebrew/core and homebrew/taps repos: https://formulae.brew.sh/docs/api/ https://formulae.brew.sh/docs/api/ Brew 4 just makes that the default and deprecates the env var (you can use HOMEBREW_NO_INSTALL_FROM_API=1 if you still want to check out the git repos for some reason).
  - mikemcquaid 4y ago
    Homebrew actually passes `--compressed` here so it's smaller: > curl -sI --compressed https://formulae.brew.sh/api/formula.json https://formulae.brew.sh/api/formula.json | grep content-length => 3482583 (~3.3MB) > curl -sI --compressed https://formulae.brew.sh/api/cask.json https://formulae.brew.sh/api/cask.json | grep content-length => 722915 (~0.6MB)
    - modernerd 4y ago
      Oh, nice, thanks for the correction!
  - mdaniel 4y ago
    > instead of checking out the large (and slow) homebrew/core and homebrew/taps repos I actually wanted to ask why they don't just prune the history of those repos to cut down on their clone size? OT1H, I could actually imagine some software historian being curious what flags were required to build sqlite 3.6.20 from 2009 (872f50ac61d7) but OTOH building a whole new distribution system instead of effectively git-squash feels weird As someone who regularly patches those local repos to work around silliness, I was bitten by that newfound API business and I'm thankful that env-var exists. As someone who's seen The New Homebrew Way, I dread the day my HOMEBREW_NO_INSTALL_FROM_API gets taken away :-(
    - saghm 4y ago
      > OTOH building a whole new distribution system instead of effectively git-squash feels weird Or even just `clone` with `--depth=1`? Just because the history exists doesn't mean you have to fetch it.
      - mdaniel 4y ago
        I wasn't super sure how subsequent `git pull` works with a shallow clone, but if it works the same, then such an obvious fix makes the invention of that API solution even more painful
        saghm 4y ago
        IIRC, the only thing that will potentially need to be changed is manually fetching the tags, since those won't be brought in. You can just do `git fetch --tags` to get all of the tag metadata though (without actually needing to pull the actual commits they reference; it just gives you the ability to look up a commit hash with a tag so that you can fetch it later).
      - mook 4y ago
        I believe they used to do `--depth` but GitHub complained because their systems weren't optimized for that and it ended being cheaper for the servers to send you the whole thing (much easier to figure out what refs you needed, or something along those lines).
arthurcolle 4y ago
`export HOMEBREW_NO_AUTOUPDATE=1; brew install whatever` maybe even just create an aliased sh function that just prepends that instead of typing it out each time. I've been doing that forever. It really is incredibly annoying how it just keeps endlessly running without doing some gymnastics (yeah sure I could add that env var set to 1 in my zshrc but I mean its just a silly default behavior)

febstar 4y ago

> no longer need to perform the slow git fetch Nice. Rust Cargo is also trialling out something similar.

atonse 4y ago

I was just wondering, have they considered storing all that in an SQLite file? And then you could refresh the file every 6 hours, say, and post it on a CDN?

modernerd 4y ago
Using SQLite instead of JSON could have worked too and would likely make the manifests smaller: > curl -sI https://formulae.brew.sh/api/formula.json https://formulae.brew.sh/api/formula.json | grep content-length => 19898457 (~19.9MB) > curl -sI https://formulae.brew.sh/api/cask.json https://formulae.brew.sh/api/cask.json | grep content-length => 4023930 (~4MB) But the JSON API has been around for a while so brew 4.0 just makes use of it by default. Brew 3.3 added an off-by-default HOMEBREW_INSTALL_FROM_API bool to let you install from brew's JSON-based API instead of checking out the large (and slow) homebrew/core and homebrew/taps repos: https://formulae.brew.sh/docs/api/ https://formulae.brew.sh/docs/api/ Brew 4 just makes that the default and deprecates the env var (you can use HOMEBREW_NO_INSTALL_FROM_API=1 if you still want to check out the git repos for some reason).
- mikemcquaid 4y ago
  Homebrew actually passes `--compressed` here so it's smaller: > curl -sI --compressed https://formulae.brew.sh/api/formula.json https://formulae.brew.sh/api/formula.json | grep content-length => 3482583 (~3.3MB) > curl -sI --compressed https://formulae.brew.sh/api/cask.json https://formulae.brew.sh/api/cask.json | grep content-length => 722915 (~0.6MB)
  - modernerd 4y ago
    Oh, nice, thanks for the correction!
- mdaniel 4y ago
  > instead of checking out the large (and slow) homebrew/core and homebrew/taps repos I actually wanted to ask why they don't just prune the history of those repos to cut down on their clone size? OT1H, I could actually imagine some software historian being curious what flags were required to build sqlite 3.6.20 from 2009 (872f50ac61d7) but OTOH building a whole new distribution system instead of effectively git-squash feels weird As someone who regularly patches those local repos to work around silliness, I was bitten by that newfound API business and I'm thankful that env-var exists. As someone who's seen The New Homebrew Way, I dread the day my HOMEBREW_NO_INSTALL_FROM_API gets taken away :-(
  - saghm 4y ago
    > OTOH building a whole new distribution system instead of effectively git-squash feels weird Or even just `clone` with `--depth=1`? Just because the history exists doesn't mean you have to fetch it.
    - mdaniel 4y ago
      I wasn't super sure how subsequent `git pull` works with a shallow clone, but if it works the same, then such an obvious fix makes the invention of that API solution even more painful
      - saghm 4y ago
        IIRC, the only thing that will potentially need to be changed is manually fetching the tags, since those won't be brought in. You can just do `git fetch --tags` to get all of the tag metadata though (without actually needing to pull the actual commits they reference; it just gives you the ability to look up a commit hash with a tag so that you can fetch it later).
    - mook 4y ago
      I believe they used to do `--depth` but GitHub complained because their systems weren't optimized for that and it ended being cheaper for the servers to send you the whole thing (much easier to figure out what refs you needed, or something along those lines).

arthurcolle 4y ago

`export HOMEBREW_NO_AUTOUPDATE=1; brew install whatever` maybe even just create an aliased sh function that just prepends that instead of typing it out each time. I've been doing that forever. It really is incredibly annoying how it just keeps endlessly running without doing some gymnastics (yeah sure I could add that env var set to 1 in my zshrc but I mean its just a silly default behavior)