10 ms·

On Distributing Command Line Applications: Why I Switched From Ruby to Go

96 points by reactor 13y ago

delinka 13y ago
He leaves out discussion of a crucial part of app distribution: dynamic libraries. While you can't create and link with dynamic libraries in Go, you can call into C or C++ libraries. Go binaries are self-contained as long as all your code is Go. But if you're depending on a C or C++ library, you still have to worry about this aspect of distribution.
- rwcarlsen 13y ago
  It is now possible to statically compile+link C libraries into your Go binary. The sqlite3 package (http://godoc.org/code.google.com/p/go-sqlite/go1/sqlite3 http://godoc.org/code.google.com/p/go-sqlite/go1/sqlite3) is a good example of this.
  - archgoon 13y ago
    Another aspect is that some opensource licensing schemes may require you to link dynamically. The LGPL, for example, requires you to do this unless you release the rest of your source code.
- octo_t 13y ago
  I'd have thought you could just statically link with the C and C++ libraries, creating a static binary? edit: beaten by rwcarlsen
avtar 13y ago
After attempting the distribution of a command line application via Rubygems I came to the following conclusion: Any application not intended exculsively for Rubyists should not be distributed via Rubygems. Another option could also be to convert gems (and dependency gems) to deb or rpm packages https://github.com/jordansissel/fpm/wiki/ConvertingGems https://github.com/jordansissel/fpm/wiki/ConvertingGems As far as I know, the Foreman project utilizes this approach http://deb.theforeman.org/pool/precise/stable/f/foreman/ http://deb.theforeman.org/pool/precise/stable/f/foreman/
- codegangsta 13y ago
  That is definitely an option, and it is essentially what we did for our linux support when we were building out a cross platform toolchain for mobile games. The tricky part was supporting Windows, Linux, and Mac all at once. Again, not suprised that distribution was hard, there were just a couple things that made distribution in Ruby a more painful experience
izietto 13y ago
Why don't compile Ruby using f.e. LLVM?
- TylerE 13y ago
  Because it's kinda pointless. The thing that makes Ruby (very) slow isn't the runtime (there might be at most a 1.5x - 2x speedup there), it's the language design. All that dynamism and runtime reflection is really a performance killer.
  - gngeal 13y ago
    it's the language design. All that dynamism and runtime reflection is really a performance killer. So why are Lisp and Smalltalk, which are equally dynamic and reflective, even faster?
    - greiskul 13y ago
      Ruby does have support for callcc, which most lisps, specially the fast ones like Common Lisp, don't, since it is a pain to implement a fast one without implications in your whole language. Can't comment on smalltalk, since I never programmed in it.
      - vidarh 13y ago
        But nobody uses call/cc in Ruby in practice. The issue with Ruby performance is immature VM's/compilers. There are tons of performance killers in Ruby, but most of them can be "worked around" by making assumptions about the common case and "just" spend some cycles verifying when the assumptions are violated. E.g. 99.99% of the time, noone are stupid enough to redefine integer addition (because, besides the fact there aren't many good reasons to, all kinds of hell breaks loose if you do, for obvious reasons), so you can usually check the type tag on integers and inline the addition. The problem is that edge case. But the edge case can be detected when someone tries to redefine '+' and you'd still save massive amounts of time if you were to set a flag at that point and still inline the additions, just with a test and branch to a slow path if people does something stupid (or in the case of JIT's, "just" re-jit everything at that point - yes, then overriding '+' will be slow, but common code will be much faster).
_pmf_ 13y ago
How many architectures are officially supported by Go?
- codegangsta 13y ago
  Supported architectures listed here: http://golang.org/doc/install#requirements http://golang.org/doc/install#requirements
  - dmm 13y ago
    There are two implementations of go: gc and gccgo. The link above describes gc. gccgo is built on gcc and supports many more environments, at least in theory. gccgo will run on addtional processors like powerpc, mips, and sparc and other operating systems like solaris. These aren't necessarily well tested though.
- MetaCosm 13y ago
  Straight forward, easy builtin cross compiling is supported from<=>to: darwin/386 darwin/amd64 freebsd/386 freebsd/amd64 linux/386 linux/amd64 linux/arm windows/386 windows/amd64 But, once you start going down the path of other implementations of go (gccgo, gollvm, gollum, goios, etc) the number of platforms supported goes up, but not sure how well supposed / stable the massive list of things go has been ported to is...
kinofcain 13y ago
This also makes go applications trivial to deploy. What used to be a massive amount of plumbing and bootstrapping for a ruby environment with multiple app server instances behind a reverse proxy with all the associated configuration and dependencies can now be little more than pushing a binary to a (nearly) stock machine instance. It really simplifies ops.
- FooBarWidget 13y ago
  I really don't see what having multiple app server instances behind reverse proxy has anything to do with Ruby vs Go. App servers behind reverse proxy is an architectural decision. You can write a Go web app that must be put behind a reverse proxy, or you can write a Go web app that is supposed to act as its own web server. Likewise, your Ruby app does not have to run multiple app server instances behind a reverse proxy. You can run it together with your main web server (Apache or Nginx) and you can run a single instance instead. See https://www.phusionpassenger.com/ https://www.phusionpassenger.com/ Of course, installing dependencies still matters. For Ruby, you can just use the Brightbox Debian/Ubuntu packages.
  - kinofcain 13y ago
    All the mainstream ruby web frameworks require something like nginx in front of multiple instances of something like thin. You can use something like phusion, unicorn or rainbows that do forking for you instead of launching multiple processes by hand, but you still have to configure them and they're still running multiple app instances. Go's built-in http library, and all the associated frameworks, run multi-threaded out of the box, and are fast enough that you don't need to put nginx or haproxy in front of them. I love ruby, I've been building and deploying rails apps since 1.0, the ease of deploying go apps is precisely the type of thing the Ruby/Rails community cares a lot about: making developers' lives easier.
    - FooBarWidget 13y ago
      It is not true that they must run multiple instances. Rails has been multithreading-capable for years. Other Ruby frameworks have been multithreading-capable for much longer. With Phusion Passenger Enterprise, not only do you not have to setup reverse proxying, the sole instance can run multithreaded. As for "fast enough that you don't need to put nginx or haproxy in front of them" - the point of reverse proxying is not performance, it's security. If anything, reverse proxying makes things slower (theoretically) because the kernel has to make one extra copy of the data. You reverse proxy stuff so that nginx or haproxy can properly handle I/O management and concurrency, and so they can sanitize HTTP headers, not because it gives you a performance boost. If Go frameworks are multithreaded by default, instead of events, then Go too can benefit from reverse proxying. If you don't want to think about any kind of reverse proxy setup, there's Passenger Standalone in the Ruby world. You type 'passenger start', and you have a fully-functional, production ready server listening on any port you, that does not require reverse proxying.
codegangsta 13y ago
Blog author here. Long story short I got frustrated with distributing Ruby cli apps and decided to switch to Go for speed and distribution. I also created a nice cli library built on top of the Go flag package. I've been having a lot of fun in Go so far and encourage anyone who hasn't given it a shot to try it out. https://github.com/codegangsta/cli https://github.com/codegangsta/cli
- johnbellone 13y ago
  Made the same decision as you recently. The only pep-peeve of mine is that I'm not a huge fan of using JSON as a configuration mechanism. I much preferred having a Ruby file for configuration. My middle ground here is using a system ruby via /usr/bin/env ruby and generating the necessary JSON output for people that want the nice syntax.
rlpb 13y ago
As a non-Rubyist, I share your pain of having to use Rubygems (and pypi, and CPAN, and all the others). However: "Since Go is a compiled language, binaries of your app can be precompiled for each platform that you wish to distribute for." For me, this is worse. There's no audit trail. No way for me to verify that the binary of your app is the same as the source you provide. So there's no way I'm running your binary on my system. I simply cannot trust it with my user account. Not only is this bad from a security perspective, it's also bad for maintenance. Over time, many end up in a situation where others cannot rebuild binaries from source at all, since binary distribution becomes the norm and the source->binary mechanism isn't maintained except on the developer's own system. Look at the Java community for an example of this. This is a problem because it means that others cannot easy be a member of the community working on the source. If you want to promote a community around the source (like Github does), you must also provide an easy way to rebuild binaries from the source. Unfortunately, there's no good answer here. Distributions like Debian have solved this problem - there are source packages, and there's a standard way of building them to get the binary packages. But there's quite a bit of arcane crud that's built up over the years, making it difficult to learn and understand Debian packaging. And the solution isn't cross platform. Each language community has produced their own solution (Rubygems, Python eggs, CPAN, etc) but none of them work with the packaging provided by any other language. To me, this may be cross platform, but not being cross language is just as big a problem.
- j_s 13y ago
  Hacker News hosted the following discussion regarding problems that have already occurred maintaining Go software due to reliance on binary distribution: "Golang dependency hell in Haunts The Manse game" https://news.ycombinator.com/item?id=5796597 https://news.ycombinator.com/item?id=5796597 In this specific instance it was more of an incorrect understanding of Go dependencies and misuse/lack of source control, but it seemed to at least come close to an echo of the issues you are raising. The specific quote from the comments: "There is no version control for dependencies".
  - tptacek 13y ago
    That wasn't a binary distribution problem or anything like one, was it? That was as I remember it a development team that simply hadn't created a known-good repository of their third-party dependencies. While I grant you that's a common pitfall in the rubygem era, it's hard to feel much sympathy as someone who came up building commercial C code, where not having a one-command third-party build-world system was a cardinal sin.
_sabe_ 13y ago
I think todays programmers is doing everything back ways. First they think about what language they want to use, then what libraries do they need, and then finally on what platform will it run and how will it be deployed. In the old days everything was much easier. First you chose a platform, depending on budget, support and so on. When you decided what platform to use, that then dictated what libraries you had at hands, and what package manager you needed for deployment. That maybe sounds very limiting, but on the other hand you don't always have to come up with new solutions how to bundle and deploy stuff all the time...
- FooBarWidget 13y ago
  Programmers "today"? Maybe I'm not old enough but I cannot remember a time when programmers would choose a platform based on their target userbase, unless they are forced to do so (e.g. iOS). Even back in 2000, given the choice, programmers would always choose to use their favorite language if they can. I remember that back in the days I was very much concerned about distribution. If I write an app in Java, users needed to download a 20 MB runtime (mind you, circa 2000 dialup was common). Visual Basic, Visual C++, etc all required some form of runtime. .NET was in its infancy and not widely distributed. I chose to write as much as possible in Delphi so that users only had 1 executable to worry about, even when Delphi wasn't the most productive platform. I asked questions on various developer forums, and I was being ridiculed for thinking so much about user distribution; the consensus from the community was: pick a language they like, and the user will just have to put up with whatever additional multi-megabyte runtime it requires.
  - _sabe_ 13y ago
    Back in 2000? Good tools like Make where built in the 70s that's utilized by Autotools and other great build systems to bridge the gap between different platforms, so that you don't have to distribute as the author of the article we're discussing, a whole runtime with all dependencies.
    - TeMPOraL 13y ago
      But the main point of the article was (which I strongly agree with) that running stuff like Autotools, gems, or package manager kind of sucks when you just want to install a console tool. At least back in the "old times" we were distributing program + additional runtime, and not requiring a random unexpected runtime in proper version + whole supporting infrastructure + program source code to build. The former option greatly simplified installation.
riobard 13y ago
Not just command line apps. Any time you need to deploy on multiple machines, Go saves you a lot of trouble. At work we use primarily `buildout` [1] to deploy Python apps with complex dependencies. It is rather slow and fragile. Additionally, PyPi and various external repositories are not very reliable, and we ended up setting up local mirrors. Nevertheless, the process is not very pleasant. We are experimenting with Go in some smaller projects, and so far the experience is great! We can build on a development machine (OS X) with all the dependencies version controlled (using git-subtree) along with our code, and cross-compile a static binary for Linux for deployment. Deploying is simply copying the binary to the production machines (usual configuration management is done via Puppet). Besides, Go can efficiently make use of multiple cores (remember to set GOMAXPROCS correctly), and we can just deploy one instance of an app per server, unlike one instance per core as in Python's case (I know there's multiprocessing, but it's bad for operation and management). This greatly reduces memory overhead if the app is big (e.g. consumes a couple hundred of MB upfront and you have 32 cores per machine = a few GB's memory is wasted). Only problem is that Go's 3rd-party libs are not as comprehensive as Python's yet, and many packages we rely on do not have mature/reliable counterparts in Go. So we are still using Python for our core stuff, but smaller projects that do not require many external packages can be done in Go elegantly. [1]: http://www.buildout.org/en/latest/ http://www.buildout.org/en/latest/
earlz 13y ago
I've written about this before, but this guy definitely wrote it better. I experience a very close relation between this and Ruby vs C#(Mono). I love Ruby as a language, but the pain of using Rubygems and random things requiring Ruby <1.9 and <1.8 (And setting up rvm to manage that) is an absolute pain. Python is slightly better, but I still have a python2 and python3 installed. This is why I love mono. Things may not be backwards compatible, but I've never had a problem where something wasn't forwards compatible. And I think this is a result of it being compiled. The compilation step decouples the language and syntax from the actual execution of the code. This means that compiled applications can have drastic and breaking syntax improvements, but still be forward compatible on future runtimes. The same can not be said of interpreted languages unless you somehow work a version number into the execution, and even then it makes implementation of the languages much harder because future versions must support every version of the syntax
pnathan 13y ago
In a prior job, I did a decent amount of Python distributions. I had it simple: 1-2 python libraries plugging into a single app which was distributed on a .deb server. Even that was not particularly fun. I can definitely understand the appeal of a build n run system. As a user, I find rubygem based installs rather horrific (same for pypi & cpan). n.b.: I'm sure there are tools out there that will appropriately massage your app and put together a app.zip package for you. But IME those sorts of tools are a lot of work to get going.
mwcampbell 13y ago
I agree that the self-contained nature of compiled Go programs is one of Go's most appealing characteristics. In fact, I think you were too diplomatic toward Ruby, and most other popular languages. In my opinion, to insist that distributing a program should not be easy is to expect too little of our tools. I can identify two main deficiencies that make it hard to distribute programs: 1. Lack of a module system that scales well to large programs while still using static linking. According to the talk "Go at Google: Language design in the service of software engineering" (http://talks.golang.org/2012/splash.article http://talks.golang.org/2012/splash.article), this is one of the main problems that the Go developers aimed to solve. 2. Too much dynamism in the language, such that a packaging tool can't tell with certainty what a program needs, and just as important, what it doesn't need. Go is static enough that this simply doesn't apply. Dart is more static than JavaScript, but still allows some dynamism. The Google Closure tools solve the problem by subsetting JavaScript. Is there any such subset for Python or Ruby? (I don't count RPython, since AFAIK that's only intended for PyPy itself.) So when choosing a language for a program that one expects to distribute to many users, I think it's a good idea to take these things into account. This may mean choosing a language that lacks some dynamic features that many of us find convenient. (Compile-time metaprogramming can compensate; too bad neither Go nor Dart has it.) Go looks like a pretty good language for servers and command-line utilities.
shurcooL 13y ago
Related to cli and Go, if you ever want to quickly execute Go funcs from a terminal [1]: $ goe go/parser 'ParseExpr("5 + 7")' [1] https://github.com/shurcooL/goe#examples https://github.com/shurcooL/goe#examples
- astrodust 13y ago
  What it needs is a `-e` flag: ruby -e 'puts 5 + 7'
  - shurcooL 13y ago
    Can you please elaborate on how that would help? Is it just for familiarity/similarity? I kinda see the 'e' from ruby -e going after the go to form goe. goe -e would be just redundant, wouldn't it?
    - astrodust 13y ago
      Perl, Ruby, Python, and many others have a succinct "evaluate this string" command-line flag. It's so commonplace it's to be expected.
conroy 13y ago
One feature I miss from distributing on RubyGems or PyPI is easy update functionality. Thankfully, go-update[1] looks to solve the problem by making it easy to create self-updating Go binaries. [1]: https://github.com/inconshreveable/go-update https://github.com/inconshreveable/go-update
stephth 13y ago
What is the DSL used in the Ruby example?
drcube 13y ago
What's wrong with distributing source + makefile?

96 points by reactor 13y ago

He leaves out discussion of a crucial part of app distribution: dynamic libraries. While you can't create and link with dynamic libraries in Go, you can call into C or C++ libraries. Go binaries are self-contained as long as all your code is Go. But if you're depending on a C or C++ library, you still have to worry about this aspect of distribution.

rwcarlsen 13y ago
It is now possible to statically compile+link C libraries into your Go binary. The sqlite3 package (http://godoc.org/code.google.com/p/go-sqlite/go1/sqlite3 http://godoc.org/code.google.com/p/go-sqlite/go1/sqlite3) is a good example of this.
- archgoon 13y ago
  Another aspect is that some opensource licensing schemes may require you to link dynamically. The LGPL, for example, requires you to do this unless you release the rest of your source code.
octo_t 13y ago
I'd have thought you could just statically link with the C and C++ libraries, creating a static binary? edit: beaten by rwcarlsen

avtar 13y ago

After attempting the distribution of a command line application via Rubygems I came to the following conclusion: Any application not intended exculsively for Rubyists should not be distributed via Rubygems. Another option could also be to convert gems (and dependency gems) to deb or rpm packages https://github.com/jordansissel/fpm/wiki/ConvertingGems https://github.com/jordansissel/fpm/wiki/ConvertingGems As far as I know, the Foreman project utilizes this approach http://deb.theforeman.org/pool/precise/stable/f/foreman/ http://deb.theforeman.org/pool/precise/stable/f/foreman/

codegangsta 13y ago
That is definitely an option, and it is essentially what we did for our linux support when we were building out a cross platform toolchain for mobile games. The tricky part was supporting Windows, Linux, and Mac all at once. Again, not suprised that distribution was hard, there were just a couple things that made distribution in Ruby a more painful experience

izietto 13y ago

Why don't compile Ruby using f.e. LLVM?

TylerE 13y ago
Because it's kinda pointless. The thing that makes Ruby (very) slow isn't the runtime (there might be at most a 1.5x - 2x speedup there), it's the language design. All that dynamism and runtime reflection is really a performance killer.
- gngeal 13y ago
  it's the language design. All that dynamism and runtime reflection is really a performance killer. So why are Lisp and Smalltalk, which are equally dynamic and reflective, even faster?
  - greiskul 13y ago
    Ruby does have support for callcc, which most lisps, specially the fast ones like Common Lisp, don't, since it is a pain to implement a fast one without implications in your whole language. Can't comment on smalltalk, since I never programmed in it.
    - vidarh 13y ago
      But nobody uses call/cc in Ruby in practice. The issue with Ruby performance is immature VM's/compilers. There are tons of performance killers in Ruby, but most of them can be "worked around" by making assumptions about the common case and "just" spend some cycles verifying when the assumptions are violated. E.g. 99.99% of the time, noone are stupid enough to redefine integer addition (because, besides the fact there aren't many good reasons to, all kinds of hell breaks loose if you do, for obvious reasons), so you can usually check the type tag on integers and inline the addition. The problem is that edge case. But the edge case can be detected when someone tries to redefine '+' and you'd still save massive amounts of time if you were to set a flag at that point and still inline the additions, just with a test and branch to a slow path if people does something stupid (or in the case of JIT's, "just" re-jit everything at that point - yes, then overriding '+' will be slow, but common code will be much faster).

_pmf_ 13y ago

How many architectures are officially supported by Go?

codegangsta 13y ago
Supported architectures listed here: http://golang.org/doc/install#requirements http://golang.org/doc/install#requirements
- dmm 13y ago
  There are two implementations of go: gc and gccgo. The link above describes gc. gccgo is built on gcc and supports many more environments, at least in theory. gccgo will run on addtional processors like powerpc, mips, and sparc and other operating systems like solaris. These aren't necessarily well tested though.
MetaCosm 13y ago
Straight forward, easy builtin cross compiling is supported from<=>to: darwin/386 darwin/amd64 freebsd/386 freebsd/amd64 linux/386 linux/amd64 linux/arm windows/386 windows/amd64 But, once you start going down the path of other implementations of go (gccgo, gollvm, gollum, goios, etc) the number of platforms supported goes up, but not sure how well supposed / stable the massive list of things go has been ported to is...

kinofcain 13y ago

This also makes go applications trivial to deploy. What used to be a massive amount of plumbing and bootstrapping for a ruby environment with multiple app server instances behind a reverse proxy with all the associated configuration and dependencies can now be little more than pushing a binary to a (nearly) stock machine instance. It really simplifies ops.

FooBarWidget 13y ago
I really don't see what having multiple app server instances behind reverse proxy has anything to do with Ruby vs Go. App servers behind reverse proxy is an architectural decision. You can write a Go web app that must be put behind a reverse proxy, or you can write a Go web app that is supposed to act as its own web server. Likewise, your Ruby app does not have to run multiple app server instances behind a reverse proxy. You can run it together with your main web server (Apache or Nginx) and you can run a single instance instead. See https://www.phusionpassenger.com/ https://www.phusionpassenger.com/ Of course, installing dependencies still matters. For Ruby, you can just use the Brightbox Debian/Ubuntu packages.
- kinofcain 13y ago
  All the mainstream ruby web frameworks require something like nginx in front of multiple instances of something like thin. You can use something like phusion, unicorn or rainbows that do forking for you instead of launching multiple processes by hand, but you still have to configure them and they're still running multiple app instances. Go's built-in http library, and all the associated frameworks, run multi-threaded out of the box, and are fast enough that you don't need to put nginx or haproxy in front of them. I love ruby, I've been building and deploying rails apps since 1.0, the ease of deploying go apps is precisely the type of thing the Ruby/Rails community cares a lot about: making developers' lives easier.
  - FooBarWidget 13y ago
    It is not true that they must run multiple instances. Rails has been multithreading-capable for years. Other Ruby frameworks have been multithreading-capable for much longer. With Phusion Passenger Enterprise, not only do you not have to setup reverse proxying, the sole instance can run multithreaded. As for "fast enough that you don't need to put nginx or haproxy in front of them" - the point of reverse proxying is not performance, it's security. If anything, reverse proxying makes things slower (theoretically) because the kernel has to make one extra copy of the data. You reverse proxy stuff so that nginx or haproxy can properly handle I/O management and concurrency, and so they can sanitize HTTP headers, not because it gives you a performance boost. If Go frameworks are multithreaded by default, instead of events, then Go too can benefit from reverse proxying. If you don't want to think about any kind of reverse proxy setup, there's Passenger Standalone in the Ruby world. You type 'passenger start', and you have a fully-functional, production ready server listening on any port you, that does not require reverse proxying.

codegangsta 13y ago

Blog author here. Long story short I got frustrated with distributing Ruby cli apps and decided to switch to Go for speed and distribution. I also created a nice cli library built on top of the Go flag package. I've been having a lot of fun in Go so far and encourage anyone who hasn't given it a shot to try it out. https://github.com/codegangsta/cli https://github.com/codegangsta/cli

johnbellone 13y ago
Made the same decision as you recently. The only pep-peeve of mine is that I'm not a huge fan of using JSON as a configuration mechanism. I much preferred having a Ruby file for configuration. My middle ground here is using a system ruby via /usr/bin/env ruby and generating the necessary JSON output for people that want the nice syntax.

rlpb 13y ago

As a non-Rubyist, I share your pain of having to use Rubygems (and pypi, and CPAN, and all the others). However: "Since Go is a compiled language, binaries of your app can be precompiled for each platform that you wish to distribute for." For me, this is worse. There's no audit trail. No way for me to verify that the binary of your app is the same as the source you provide. So there's no way I'm running your binary on my system. I simply cannot trust it with my user account. Not only is this bad from a security perspective, it's also bad for maintenance. Over time, many end up in a situation where others cannot rebuild binaries from source at all, since binary distribution becomes the norm and the source->binary mechanism isn't maintained except on the developer's own system. Look at the Java community for an example of this. This is a problem because it means that others cannot easy be a member of the community working on the source. If you want to promote a community around the source (like Github does), you must also provide an easy way to rebuild binaries from the source. Unfortunately, there's no good answer here. Distributions like Debian have solved this problem - there are source packages, and there's a standard way of building them to get the binary packages. But there's quite a bit of arcane crud that's built up over the years, making it difficult to learn and understand Debian packaging. And the solution isn't cross platform. Each language community has produced their own solution (Rubygems, Python eggs, CPAN, etc) but none of them work with the packaging provided by any other language. To me, this may be cross platform, but not being cross language is just as big a problem.

j_s 13y ago
Hacker News hosted the following discussion regarding problems that have already occurred maintaining Go software due to reliance on binary distribution: "Golang dependency hell in Haunts The Manse game" https://news.ycombinator.com/item?id=5796597 https://news.ycombinator.com/item?id=5796597 In this specific instance it was more of an incorrect understanding of Go dependencies and misuse/lack of source control, but it seemed to at least come close to an echo of the issues you are raising. The specific quote from the comments: "There is no version control for dependencies".
- tptacek 13y ago
  That wasn't a binary distribution problem or anything like one, was it? That was as I remember it a development team that simply hadn't created a known-good repository of their third-party dependencies. While I grant you that's a common pitfall in the rubygem era, it's hard to feel much sympathy as someone who came up building commercial C code, where not having a one-command third-party build-world system was a cardinal sin.

_sabe_ 13y ago

I think todays programmers is doing everything back ways. First they think about what language they want to use, then what libraries do they need, and then finally on what platform will it run and how will it be deployed. In the old days everything was much easier. First you chose a platform, depending on budget, support and so on. When you decided what platform to use, that then dictated what libraries you had at hands, and what package manager you needed for deployment. That maybe sounds very limiting, but on the other hand you don't always have to come up with new solutions how to bundle and deploy stuff all the time...

FooBarWidget 13y ago
Programmers "today"? Maybe I'm not old enough but I cannot remember a time when programmers would choose a platform based on their target userbase, unless they are forced to do so (e.g. iOS). Even back in 2000, given the choice, programmers would always choose to use their favorite language if they can. I remember that back in the days I was very much concerned about distribution. If I write an app in Java, users needed to download a 20 MB runtime (mind you, circa 2000 dialup was common). Visual Basic, Visual C++, etc all required some form of runtime. .NET was in its infancy and not widely distributed. I chose to write as much as possible in Delphi so that users only had 1 executable to worry about, even when Delphi wasn't the most productive platform. I asked questions on various developer forums, and I was being ridiculed for thinking so much about user distribution; the consensus from the community was: pick a language they like, and the user will just have to put up with whatever additional multi-megabyte runtime it requires.
- _sabe_ 13y ago
  Back in 2000? Good tools like Make where built in the 70s that's utilized by Autotools and other great build systems to bridge the gap between different platforms, so that you don't have to distribute as the author of the article we're discussing, a whole runtime with all dependencies.
  - TeMPOraL 13y ago
    But the main point of the article was (which I strongly agree with) that running stuff like Autotools, gems, or package manager kind of sucks when you just want to install a console tool. At least back in the "old times" we were distributing program + additional runtime, and not requiring a random unexpected runtime in proper version + whole supporting infrastructure + program source code to build. The former option greatly simplified installation.

riobard 13y ago

Not just command line apps. Any time you need to deploy on multiple machines, Go saves you a lot of trouble. At work we use primarily `buildout` [1] to deploy Python apps with complex dependencies. It is rather slow and fragile. Additionally, PyPi and various external repositories are not very reliable, and we ended up setting up local mirrors. Nevertheless, the process is not very pleasant. We are experimenting with Go in some smaller projects, and so far the experience is great! We can build on a development machine (OS X) with all the dependencies version controlled (using git-subtree) along with our code, and cross-compile a static binary for Linux for deployment. Deploying is simply copying the binary to the production machines (usual configuration management is done via Puppet). Besides, Go can efficiently make use of multiple cores (remember to set GOMAXPROCS correctly), and we can just deploy one instance of an app per server, unlike one instance per core as in Python's case (I know there's multiprocessing, but it's bad for operation and management). This greatly reduces memory overhead if the app is big (e.g. consumes a couple hundred of MB upfront and you have 32 cores per machine = a few GB's memory is wasted). Only problem is that Go's 3rd-party libs are not as comprehensive as Python's yet, and many packages we rely on do not have mature/reliable counterparts in Go. So we are still using Python for our core stuff, but smaller projects that do not require many external packages can be done in Go elegantly. [1]: http://www.buildout.org/en/latest/ http://www.buildout.org/en/latest/

earlz 13y ago

I've written about this before, but this guy definitely wrote it better. I experience a very close relation between this and Ruby vs C#(Mono). I love Ruby as a language, but the pain of using Rubygems and random things requiring Ruby <1.9 and <1.8 (And setting up rvm to manage that) is an absolute pain. Python is slightly better, but I still have a python2 and python3 installed. This is why I love mono. Things may not be backwards compatible, but I've never had a problem where something wasn't forwards compatible. And I think this is a result of it being compiled. The compilation step decouples the language and syntax from the actual execution of the code. This means that compiled applications can have drastic and breaking syntax improvements, but still be forward compatible on future runtimes. The same can not be said of interpreted languages unless you somehow work a version number into the execution, and even then it makes implementation of the languages much harder because future versions must support every version of the syntax

pnathan 13y ago

In a prior job, I did a decent amount of Python distributions. I had it simple: 1-2 python libraries plugging into a single app which was distributed on a .deb server. Even that was not particularly fun. I can definitely understand the appeal of a build n run system. As a user, I find rubygem based installs rather horrific (same for pypi & cpan). n.b.: I'm sure there are tools out there that will appropriately massage your app and put together a app.zip package for you. But IME those sorts of tools are a lot of work to get going.

mwcampbell 13y ago

I agree that the self-contained nature of compiled Go programs is one of Go's most appealing characteristics. In fact, I think you were too diplomatic toward Ruby, and most other popular languages. In my opinion, to insist that distributing a program should not be easy is to expect too little of our tools. I can identify two main deficiencies that make it hard to distribute programs: 1. Lack of a module system that scales well to large programs while still using static linking. According to the talk "Go at Google: Language design in the service of software engineering" (http://talks.golang.org/2012/splash.article http://talks.golang.org/2012/splash.article), this is one of the main problems that the Go developers aimed to solve. 2. Too much dynamism in the language, such that a packaging tool can't tell with certainty what a program needs, and just as important, what it doesn't need. Go is static enough that this simply doesn't apply. Dart is more static than JavaScript, but still allows some dynamism. The Google Closure tools solve the problem by subsetting JavaScript. Is there any such subset for Python or Ruby? (I don't count RPython, since AFAIK that's only intended for PyPy itself.) So when choosing a language for a program that one expects to distribute to many users, I think it's a good idea to take these things into account. This may mean choosing a language that lacks some dynamic features that many of us find convenient. (Compile-time metaprogramming can compensate; too bad neither Go nor Dart has it.) Go looks like a pretty good language for servers and command-line utilities.

shurcooL 13y ago

Related to cli and Go, if you ever want to quickly execute Go funcs from a terminal [1]: $ goe go/parser 'ParseExpr("5 + 7")' [1] https://github.com/shurcooL/goe#examples https://github.com/shurcooL/goe#examples

astrodust 13y ago
What it needs is a `-e` flag: ruby -e 'puts 5 + 7'
- shurcooL 13y ago
  Can you please elaborate on how that would help? Is it just for familiarity/similarity? I kinda see the 'e' from ruby -e going after the go to form goe. goe -e would be just redundant, wouldn't it?
  - astrodust 13y ago
    Perl, Ruby, Python, and many others have a succinct "evaluate this string" command-line flag. It's so commonplace it's to be expected.

conroy 13y ago

One feature I miss from distributing on RubyGems or PyPI is easy update functionality. Thankfully, go-update[1] looks to solve the problem by making it easy to create self-updating Go binaries. [1]: https://github.com/inconshreveable/go-update https://github.com/inconshreveable/go-update

stephth 13y ago

What is the DSL used in the Ruby example?

drcube 13y ago

What's wrong with distributing source + makefile?