4 ms·

Why I made pkgr.io – digressions on software packaging

25 points by crohr 13y ago

mwcampbell 13y ago
A problem with the Heroku/pkgr approach to handling dependencies is that applications using this approach can never fully follow the advice provided under "Dependencies" in the 12-factor guidelines (http://12factor.net/dependencies http://12factor.net/dependencies), because some libraries are always outside the scope of RubyGems, pip, npm, or whatever language-specific package system the app is using. Heroku works around this problem, rather than dealing with it directly, by including a hodgepodge of packages in the Cedar base system. If you have a Heroku account, create an app, run a one-off shell process, and look at all the packages that are installed (e.g. by looking at /var/lib/dpkg/info/*.list). Or look at how Dokku and Flynn approximate this poorly defined base system by looking at https://github.com/progrium/cedarish https://github.com/progrium/cedarish. I suspect that pkgr will need to do the same. EDIT: It's particularly ironic that the aformentioned section of the 12-factor guidelines says an app shouldn't assume the existence of commands like curl and the ImageMagick utilities, yet the Cedar base system includes both. So it seems to me that containerization, with the whole container under the app developer's control (unlike on Heroku), is the only way to fully implement the 12-factor approach to dependencies. The downside is that the app developer is now responsible for the whole stack except for the kernel, including security-sensitive libraries such as OpenSSL. So I guess we'll need tools to help app developers keep up with changes in the lower layers of the stack. Another problem, for now, is that Docker isn't yet ready for production, and isn't even targeting more conservative distros like CentOS and Debian yet.
- crohr 13y ago
  OP here. It's true that Heroku were required to pre-install a lot of dependencies, because they don't offer a way for developers to specify additional dependencies to be installed on their servers. On the contrary, with pkgr.io, additional build and runtime dependencies can/must be added by the package maintainer if anything other than the base stuff is needed. See https://pkgr.io/doc#dependencies https://pkgr.io/doc#dependencies and, as an example, the .pkgr.yml file for a fairly complex project such as Gitlab: https://github.com/pkgr/gitlabhq/blob/pkgr/.pkgr.yml https://github.com/pkgr/gitlabhq/blob/pkgr/.pkgr.yml.
  - mwcampbell 13y ago
    Ah, OK, thanks for pointing out that feature of pkgr. I think Heroku would do well, in a hypothetical Cedar+1, to allow both buildpacks and apps to specify additional Ubuntu packages to be installed in the container, and then dramatically strip down the base system.
- zimbatm 13y ago
  Ideally all your project's dependencies are described in a DSL. A second DSL is involved for dependency resolution and version pinning. Unfortunately each language seem to want to implement their own system.
deleted 13y ago
[deleted]

25 points by crohr 13y ago

A problem with the Heroku/pkgr approach to handling dependencies is that applications using this approach can never fully follow the advice provided under "Dependencies" in the 12-factor guidelines (http://12factor.net/dependencies http://12factor.net/dependencies), because some libraries are always outside the scope of RubyGems, pip, npm, or whatever language-specific package system the app is using. Heroku works around this problem, rather than dealing with it directly, by including a hodgepodge of packages in the Cedar base system. If you have a Heroku account, create an app, run a one-off shell process, and look at all the packages that are installed (e.g. by looking at /var/lib/dpkg/info/*.list). Or look at how Dokku and Flynn approximate this poorly defined base system by looking at https://github.com/progrium/cedarish https://github.com/progrium/cedarish. I suspect that pkgr will need to do the same. EDIT: It's particularly ironic that the aformentioned section of the 12-factor guidelines says an app shouldn't assume the existence of commands like curl and the ImageMagick utilities, yet the Cedar base system includes both. So it seems to me that containerization, with the whole container under the app developer's control (unlike on Heroku), is the only way to fully implement the 12-factor approach to dependencies. The downside is that the app developer is now responsible for the whole stack except for the kernel, including security-sensitive libraries such as OpenSSL. So I guess we'll need tools to help app developers keep up with changes in the lower layers of the stack. Another problem, for now, is that Docker isn't yet ready for production, and isn't even targeting more conservative distros like CentOS and Debian yet.

crohr 13y ago
OP here. It's true that Heroku were required to pre-install a lot of dependencies, because they don't offer a way for developers to specify additional dependencies to be installed on their servers. On the contrary, with pkgr.io, additional build and runtime dependencies can/must be added by the package maintainer if anything other than the base stuff is needed. See https://pkgr.io/doc#dependencies https://pkgr.io/doc#dependencies and, as an example, the .pkgr.yml file for a fairly complex project such as Gitlab: https://github.com/pkgr/gitlabhq/blob/pkgr/.pkgr.yml https://github.com/pkgr/gitlabhq/blob/pkgr/.pkgr.yml.
- mwcampbell 13y ago
  Ah, OK, thanks for pointing out that feature of pkgr. I think Heroku would do well, in a hypothetical Cedar+1, to allow both buildpacks and apps to specify additional Ubuntu packages to be installed in the container, and then dramatically strip down the base system.
zimbatm 13y ago
Ideally all your project's dependencies are described in a DSL. A second DSL is involved for dependency resolution and version pinning. Unfortunately each language seem to want to implement their own system.

deleted 13y ago

[deleted]