4 ms·

This is one of the places that the often-maligned Homebrew package manager does a good job. App dependencies in languages like Python and Lua are fetched manua

This is one of the places that the often-maligned Homebrew package manager does a good job. App dependencies in languages like Python and Lua are fetched manually, with exact filenames and hashes, and installed into an isolated location just for that app. If they turn out to be used by several apps, then they get packaged normally. I wish other package managers did it this way. It would make packaging complicated applications much easier and result in many fewer dependency conflicts, and allows devs to produce more stable software. It's basically static linking but for Python packages. Sometimes you want shared libraries, but for one-off dependencies it's possibly more trouble than it's worth.

lloydatkinson 3y ago
Nix/NixOS too
gcr 3y ago
I didn't know this about homebrew. Is this only true for installing homebrew python packages? To be honest, I would `brew install python3` or whatever, then manage my own per-project virtual environments with direnv, so I'd bypass all of homebrew's efforts to encapsulate and package python packages...
- nerdponx 3y ago
  It's at the discretion of the package maintainer, but it's pretty common. Just look at the `Formula.rb` for any standalone app in Python or Lua. Example: https://github.com/Homebrew/homebrew-core/blob/66c42fbd814b3a0c291fb872a569abc441747685/Formula/l/locust.rb#L26-L129 https://github.com/Homebrew/homebrew-core/blob/66c42fbd814b3... I think using per-project venv is still the sensible thing to do, not just in Python but using the equivalent in Lua (hererocks), Ruby (Bundler), etc. This is for system-wide installations of standalone CLI tools and other "apps". However if you do find something on PyPI that is a standalone app, but isn't in Homebrew, you can create an isolated env for it automatically using Pipx https://pypa.github.io/pipx/ https://pypa.github.io/pipx/.
cogman10 3y ago
You are basically describing appimage, flatpak, and to a lesser extent, snap and docker. AppImage is the closest analog as it will reuse files across images.
- nerdponx 3y ago
  I'm not sure about that. I think it's more akin to vendoring libraries in a C application by unpacking someone else's tarball into your own source tree. The overall philosophy is that of AppImage I suppose. Flatpak doesn't really count because you still have "runtimes" that other apps depend on.
hedora 3y ago
The article suggests using debian’s contrib package category in this way. (Packages with such dependencies live in contrib, since they don’t live up to Debian’s quality standards, which is fine.)
Fnoord 3y ago
Any reproducible package manager is suffice. Debian aims for that, too, AFAIK. With Homebrew you end up having to link the binary, and you cannot link different binaries with the same name. I just use venvs for Python, and pipx takes care of that, together with topgrade which ensures all software repos remain updated, Homebrew or Nix or not. For Rust I use Cargo, etc etc. I'd agree Nix does it more elegantly but it is a very different beast to handle. Anything I want to do with Nix(OS) I have to look up how they do it.
- nerdponx 3y ago
  The whole point of the thread is that Debian doesn't currently vendor dependencies, so you need to package every single dependency. Reproducibly, but fully packaged and maintained nonetheless, and necessitating careful testing to avoid conflicts between versions. My point is that Homebrew takes the vendoring approach instead.
mikemcquaid 3y ago
As the person who has probably influenced Homebrew's policy on this the most: I'm very pleased to read this, thanks for saying it! I was also reading this link today to think more about how Homebrew can handle this stuff better. To me the main differentiation is "is the package for an application or a library?". If it's a library: it's a trickier balance. If it's an application: vendoring the e.g. NPM/RubyGems modules is most likely to create the best, most useful experience for the user, upstream creator and Homebrew maintainers.
- nerdponx 3y ago
  As a very very occasional and light-duty packaging contributor, IMO the best thing Homebrew can do at this point is start writing much better docs for packagers. The Ruby API doc is very difficult to navigate if you aren't already a Ruby programmer. And it's very hard to know what the "right way" is for doing anything, other than browsing other formulas and hoping you land on a useful example. This is another area where Homebrew has an opportunity to do better than the alternatives. Making it faster wouldn't hurt either, but that's another story ;)

lloydatkinson 3y ago

Nix/NixOS too

gcr 3y ago

I didn't know this about homebrew. Is this only true for installing homebrew python packages? To be honest, I would `brew install python3` or whatever, then manage my own per-project virtual environments with direnv, so I'd bypass all of homebrew's efforts to encapsulate and package python packages...

nerdponx 3y ago
It's at the discretion of the package maintainer, but it's pretty common. Just look at the `Formula.rb` for any standalone app in Python or Lua. Example: https://github.com/Homebrew/homebrew-core/blob/66c42fbd814b3a0c291fb872a569abc441747685/Formula/l/locust.rb#L26-L129 https://github.com/Homebrew/homebrew-core/blob/66c42fbd814b3... I think using per-project venv is still the sensible thing to do, not just in Python but using the equivalent in Lua (hererocks), Ruby (Bundler), etc. This is for system-wide installations of standalone CLI tools and other "apps". However if you do find something on PyPI that is a standalone app, but isn't in Homebrew, you can create an isolated env for it automatically using Pipx https://pypa.github.io/pipx/ https://pypa.github.io/pipx/.

cogman10 3y ago

You are basically describing appimage, flatpak, and to a lesser extent, snap and docker. AppImage is the closest analog as it will reuse files across images.

nerdponx 3y ago
I'm not sure about that. I think it's more akin to vendoring libraries in a C application by unpacking someone else's tarball into your own source tree. The overall philosophy is that of AppImage I suppose. Flatpak doesn't really count because you still have "runtimes" that other apps depend on.

hedora 3y ago

The article suggests using debian’s contrib package category in this way. (Packages with such dependencies live in contrib, since they don’t live up to Debian’s quality standards, which is fine.)

Fnoord 3y ago

Any reproducible package manager is suffice. Debian aims for that, too, AFAIK. With Homebrew you end up having to link the binary, and you cannot link different binaries with the same name. I just use venvs for Python, and pipx takes care of that, together with topgrade which ensures all software repos remain updated, Homebrew or Nix or not. For Rust I use Cargo, etc etc. I'd agree Nix does it more elegantly but it is a very different beast to handle. Anything I want to do with Nix(OS) I have to look up how they do it.

nerdponx 3y ago
The whole point of the thread is that Debian doesn't currently vendor dependencies, so you need to package every single dependency. Reproducibly, but fully packaged and maintained nonetheless, and necessitating careful testing to avoid conflicts between versions. My point is that Homebrew takes the vendoring approach instead.

mikemcquaid 3y ago

As the person who has probably influenced Homebrew's policy on this the most: I'm very pleased to read this, thanks for saying it! I was also reading this link today to think more about how Homebrew can handle this stuff better. To me the main differentiation is "is the package for an application or a library?". If it's a library: it's a trickier balance. If it's an application: vendoring the e.g. NPM/RubyGems modules is most likely to create the best, most useful experience for the user, upstream creator and Homebrew maintainers.

nerdponx 3y ago
As a very very occasional and light-duty packaging contributor, IMO the best thing Homebrew can do at this point is start writing much better docs for packagers. The Ruby API doc is very difficult to navigate if you aren't already a Ruby programmer. And it's very hard to know what the "right way" is for doing anything, other than browsing other formulas and hoping you land on a useful example. This is another area where Homebrew has an opportunity to do better than the alternatives. Making it faster wouldn't hurt either, but that's another story ;)