5 ms·

This guy doesn't seem to understand the real differences between java/clojure packaging, and scripting language packaging. Ruby's "gems" are source packages.

by hapless 13y ago

This guy doesn't seem to understand the real differences between java/clojure packaging, and scripting language packaging. Ruby's "gems" are source packages. When a gem is installed, arbitrary code is executed, C stubs (or even C libraries!) are compiled, and the resulting artifacts can be very different on different systems. Some Ruby gems even modify the Ruby code at install-time. The same is true for Perl (CPAN) and Python (setuptools, pip, etc) Java and Clojure packages are artifacts. Each jar is a finished, built product. If it has JNI stubs, those stubs were pre-built against a specific version. The artifact is identical on every installed system. No arbitrary code is executed. Reasoning about the contents of a Clojure package does not require me to solve the halting problem. The artifact approach is unambiguously better for any production deployment. The source-based approach found in Ruby, Perl, and Python is a problem for me more often than a solution. It is hypothetically great that I can use the same "gem" on OSX and Linux, but it is more important to me that I get consistent deployments on two different Linux systems.

npsimons 13y ago
The artifact approach is unambiguously better for any production deployment. Which is precisely why many Linux distributions use them, and "app stores" were already a thing in Debian in the mid-90s. Every time I read about a programming language's "package manager" I can't help but feel that the creators of such are reinventing the wheel, poorly, and mostly to paper over issues in backwards OSes that don't have proper package management.
- Anderkent 13y ago
  .. or they don't like the limitation of only having 1 version of any artifact installed at a time, artifacts being installed globally, etc. etc. Of course they could target Nix instead, but how many people use Nix?
  - dllthomas 13y ago
    "Of course they could target Nix instead, but how many people use Nix?" More than use the new system they developed instead, at the outset.
- FooBarWidget 13y ago
  You make it sound like "papering over issues in backward OSes" is a bad thing. The fact is, those "backward OSes" have a HUGE user base, no matter how much you may judge them for being inferior. There is clearly demand for a tool that: 1. Allows developers to distribute their stuff to users on those OSes. 2. Requires the least amount of effort as possible on part of the developer. 3. Does not require the developer to custom-package their stuff multiple times, once for each OS. Face it, developers are lazy (not meant in a derogatory way). Most solutions do not meet requirements 2 and 3, which is why developers will largely choose to avoid them until they have no choice. And also, what matthewmacleod said in this thread is very true.
- matthewmacleod 13y ago
  Hmm. This is an appropriate implementation for an operating system, but I don't think it's suitable for a programming language. To take Ruby as an example - I have lots of different interpreter versions installed. 1.8.7, 1.9.3, 2.0, 2.1, jruby, rubinius. There's some scope for managing this in OS package managers, but it's already complex. Then for each of these environments, I want a different set of packages with different versions (some are compatible with older versions, and some aren't, for example). And finally, I want different versions and combinations of these packages for different apps that I might want to run - again, there's limited scope to implement this in an OS package manager.
  - regularfry 13y ago
    You ideally want both. Source packages for development work, and artifacts for deployment. I've switched to .debs for deployment. Oh so simple.
    - matthewmacleod 13y ago
      Oh yeah, don't get me wrong - that's lovely. There's nothing more frustrating that trying to get a mildly complex Rails app deployed on a mutable server.
  - bryanlarsen 13y ago
    It's only a problem because the versioning on most ruby gems is broken, because Ruby package managers enable such breakage. The big difference between dpkg and gem is that gem allows you to install more than one version of a gem at a time. With dpkg you overcome this by creating a whole new package when the major version changes. Thus I can install libcap and libcap2 simultaneously. Any breakages in minor version changes are considered significant problems. This is the right thing to do in a production environment: minor & patch level increments are very likely bug fixes and security fixes, and should be applied to all packages that use that library. But it's a PITA for development. And since package manager developers are mostly developers, they did what was right for them, and allowed multiple versions to be installed simultaneously. Today, the right answer is to use something like Vagrant or Docker so that development environments are much closer to production environments.
- klibertp 13y ago
  > creators of such are reinventing the wheel It's kinda funny how they named the latest and greatest in the Python world because of this: http://wheel.readthedocs.org/en/latest/ http://wheel.readthedocs.org/en/latest/
jamesbritt 13y ago
What gems modify the Ruby code?
- mbillie1 13y ago
  I assume he means: any gems that monkey-patch any of the built-in classes. I believe ActiveSupport modifies things like Fixnum (1.days.from_now, for example).
  - reiz 13y ago
    Yes. That's what I meant.
    - jamesbritt 13y ago
      Ah. Ok. I was thinking something changing core files on disk. I bet that's doable ... Change string.c, recompile, if a gem can execute arbitray code.
  - klibertp 13y ago
    Does it really happen at install time or (like in Python) at runtime everytime the package/module/gem is loaded/required?
    - evilduck 13y ago
      ActiveSupport doesn't modify the base Ruby classes until it's been loaded. Merely installing gems will not impact anything except disk space.
- hapless 13y ago
  puppet, facter, rake, mongrel use 'sed' to edit the code on the fly at install time, to set shebang lines etc There are surely many others, these are just the ones I have at hand as examples.
gmjosack 13y ago
Python has wheel, which is a binary package format that pip supports. It is something I've been looking into for deploying Python packages in production. Edit: Link to PEP for Wheel: http://www.python.org/dev/peps/pep-0427/ http://www.python.org/dev/peps/pep-0427/
nailer 13y ago
If anyone else was wondering: http://en.wikipedia.org/wiki/Halting_problem http://en.wikipedia.org/wiki/Halting_problem
- hapless 13y ago
  Specifically, much gem metadata is stored as a ruby script. To get some information about a gem, I have to execute the ruby code. Python libraries that use setuptools are the same way.
1qaz2wsx3edc 13y ago
I think virtualization can take care of consistent deployments. And developing under a virtualized container ensures consistency from multiple platforms. My point is: a source-based approach is not the tool to blame for consistency, it's used with lazy concern, and that's a mistake onto the crafter. I work on two different platforms as one. Virtualization achieves that.
- hapless 13y ago
  Yes, freezing particular OS configurations into golden images on a virt platform can make a source-based approach repeatable. Unfortunately, this just moves the problem: I now have to maintain the golden images. Relying on virtualization for repeatable source-based deployment is like screwing in a lightbulb by rotating the entire building around the bulb.
spellboots 13y ago
I don't think the artifact approach is unambiguously better. Sure, "consistent deployments on two different Linux systems" is important but you can achieve that with rubygems along with the extra flexibility that distributing the packages as source provides. Two approaches you can take: 1. bundle package will package up all of your installed gems, which you can then deploy with your project. 2. Your build server can install all of your gems, and then you can package up the resulting directory and deploy it - you could also build e.g. an rpm package.
- hapless 13y ago
  "bundle package" still just bundles the source packages. Installing all of the gems then packaging the resulting directory is an example of an artifact-based approach. I prefer to do it gem-by-gem using 'fpm' in order to maintain better dependency data -- fpm will invoke rpmbuild with the right options to find shared objects and automatically attach correct dependency data for the ruby runtime, C libraries etc
kosinus 13y ago
I used to be in the Ruby / Rails / Bundler camp, and this was indeed an annoyance. But with Node.js, the typical deployment scenario for us, and I believe others as well, is to simply tar up the entire app including dependencies. I'm not sure if this is possible with the other scripting languages you mention. I also don't believe this is as big an issue in development for Node.js, because most packages are pure-JS, and don't run any build steps whatsoever. Finally, I think it's also very convenient to have my dependencies' sources available in development. I trace into them quite often.
- evilduck 13y ago
  Bundler offers similar functionality, but it seems like most people don't use it. http://bundler.io/v1.5/bundle_package.html http://bundler.io/v1.5/bundle_package.html
  - hapless 13y ago
    "bundle package" caches the source packages, the gems themselves. not the artifacts produced by installing the gems.
curun1r 13y ago
The artifact approach is unambiguously better for any production deployment. Which is only a problem if you're still doing application deployments directly to production machines, which is becoming less and less common. Tools like Docker, Packer and Aminator allow you to do the package install at build time.

by hapless 13y ago

npsimons 13y ago

The artifact approach is unambiguously better for any production deployment. Which is precisely why many Linux distributions use them, and "app stores" were already a thing in Debian in the mid-90s. Every time I read about a programming language's "package manager" I can't help but feel that the creators of such are reinventing the wheel, poorly, and mostly to paper over issues in backwards OSes that don't have proper package management.

Anderkent 13y ago
.. or they don't like the limitation of only having 1 version of any artifact installed at a time, artifacts being installed globally, etc. etc. Of course they could target Nix instead, but how many people use Nix?
- dllthomas 13y ago
  "Of course they could target Nix instead, but how many people use Nix?" More than use the new system they developed instead, at the outset.
FooBarWidget 13y ago
You make it sound like "papering over issues in backward OSes" is a bad thing. The fact is, those "backward OSes" have a HUGE user base, no matter how much you may judge them for being inferior. There is clearly demand for a tool that: 1. Allows developers to distribute their stuff to users on those OSes. 2. Requires the least amount of effort as possible on part of the developer. 3. Does not require the developer to custom-package their stuff multiple times, once for each OS. Face it, developers are lazy (not meant in a derogatory way). Most solutions do not meet requirements 2 and 3, which is why developers will largely choose to avoid them until they have no choice. And also, what matthewmacleod said in this thread is very true.
matthewmacleod 13y ago
Hmm. This is an appropriate implementation for an operating system, but I don't think it's suitable for a programming language. To take Ruby as an example - I have lots of different interpreter versions installed. 1.8.7, 1.9.3, 2.0, 2.1, jruby, rubinius. There's some scope for managing this in OS package managers, but it's already complex. Then for each of these environments, I want a different set of packages with different versions (some are compatible with older versions, and some aren't, for example). And finally, I want different versions and combinations of these packages for different apps that I might want to run - again, there's limited scope to implement this in an OS package manager.
- regularfry 13y ago
  You ideally want both. Source packages for development work, and artifacts for deployment. I've switched to .debs for deployment. Oh so simple.
  - matthewmacleod 13y ago
    Oh yeah, don't get me wrong - that's lovely. There's nothing more frustrating that trying to get a mildly complex Rails app deployed on a mutable server.
- bryanlarsen 13y ago
  It's only a problem because the versioning on most ruby gems is broken, because Ruby package managers enable such breakage. The big difference between dpkg and gem is that gem allows you to install more than one version of a gem at a time. With dpkg you overcome this by creating a whole new package when the major version changes. Thus I can install libcap and libcap2 simultaneously. Any breakages in minor version changes are considered significant problems. This is the right thing to do in a production environment: minor & patch level increments are very likely bug fixes and security fixes, and should be applied to all packages that use that library. But it's a PITA for development. And since package manager developers are mostly developers, they did what was right for them, and allowed multiple versions to be installed simultaneously. Today, the right answer is to use something like Vagrant or Docker so that development environments are much closer to production environments.
klibertp 13y ago
> creators of such are reinventing the wheel It's kinda funny how they named the latest and greatest in the Python world because of this: http://wheel.readthedocs.org/en/latest/ http://wheel.readthedocs.org/en/latest/

jamesbritt 13y ago

What gems modify the Ruby code?

mbillie1 13y ago
I assume he means: any gems that monkey-patch any of the built-in classes. I believe ActiveSupport modifies things like Fixnum (1.days.from_now, for example).
- reiz 13y ago
  Yes. That's what I meant.
  - jamesbritt 13y ago
    Ah. Ok. I was thinking something changing core files on disk. I bet that's doable ... Change string.c, recompile, if a gem can execute arbitray code.
- klibertp 13y ago
  Does it really happen at install time or (like in Python) at runtime everytime the package/module/gem is loaded/required?
  - evilduck 13y ago
    ActiveSupport doesn't modify the base Ruby classes until it's been loaded. Merely installing gems will not impact anything except disk space.
hapless 13y ago
puppet, facter, rake, mongrel use 'sed' to edit the code on the fly at install time, to set shebang lines etc There are surely many others, these are just the ones I have at hand as examples.

gmjosack 13y ago

Python has wheel, which is a binary package format that pip supports. It is something I've been looking into for deploying Python packages in production. Edit: Link to PEP for Wheel: http://www.python.org/dev/peps/pep-0427/ http://www.python.org/dev/peps/pep-0427/

nailer 13y ago

If anyone else was wondering: http://en.wikipedia.org/wiki/Halting_problem http://en.wikipedia.org/wiki/Halting_problem

hapless 13y ago
Specifically, much gem metadata is stored as a ruby script. To get some information about a gem, I have to execute the ruby code. Python libraries that use setuptools are the same way.

1qaz2wsx3edc 13y ago

I think virtualization can take care of consistent deployments. And developing under a virtualized container ensures consistency from multiple platforms. My point is: a source-based approach is not the tool to blame for consistency, it's used with lazy concern, and that's a mistake onto the crafter. I work on two different platforms as one. Virtualization achieves that.

hapless 13y ago
Yes, freezing particular OS configurations into golden images on a virt platform can make a source-based approach repeatable. Unfortunately, this just moves the problem: I now have to maintain the golden images. Relying on virtualization for repeatable source-based deployment is like screwing in a lightbulb by rotating the entire building around the bulb.

spellboots 13y ago

I don't think the artifact approach is unambiguously better. Sure, "consistent deployments on two different Linux systems" is important but you can achieve that with rubygems along with the extra flexibility that distributing the packages as source provides. Two approaches you can take: 1. bundle package will package up all of your installed gems, which you can then deploy with your project. 2. Your build server can install all of your gems, and then you can package up the resulting directory and deploy it - you could also build e.g. an rpm package.

hapless 13y ago
"bundle package" still just bundles the source packages. Installing all of the gems then packaging the resulting directory is an example of an artifact-based approach. I prefer to do it gem-by-gem using 'fpm' in order to maintain better dependency data -- fpm will invoke rpmbuild with the right options to find shared objects and automatically attach correct dependency data for the ruby runtime, C libraries etc

kosinus 13y ago

I used to be in the Ruby / Rails / Bundler camp, and this was indeed an annoyance. But with Node.js, the typical deployment scenario for us, and I believe others as well, is to simply tar up the entire app including dependencies. I'm not sure if this is possible with the other scripting languages you mention. I also don't believe this is as big an issue in development for Node.js, because most packages are pure-JS, and don't run any build steps whatsoever. Finally, I think it's also very convenient to have my dependencies' sources available in development. I trace into them quite often.

evilduck 13y ago
Bundler offers similar functionality, but it seems like most people don't use it. http://bundler.io/v1.5/bundle_package.html http://bundler.io/v1.5/bundle_package.html
- hapless 13y ago
  "bundle package" caches the source packages, the gems themselves. not the artifacts produced by installing the gems.

curun1r 13y ago

The artifact approach is unambiguously better for any production deployment. Which is only a problem if you're still doing application deployments directly to production machines, which is becoming less and less common. Tools like Docker, Packer and Aminator allow you to do the package install at build time.