6 ms·

As a non-Rubyist, I share your pain of having to use Rubygems (and pypi, and CPAN, and all the others). However: "Since Go is a compiled language, binaries of

by rlpb 13y ago

As a non-Rubyist, I share your pain of having to use Rubygems (and pypi, and CPAN, and all the others). However: "Since Go is a compiled language, binaries of your app can be precompiled for each platform that you wish to distribute for." For me, this is worse. There's no audit trail. No way for me to verify that the binary of your app is the same as the source you provide. So there's no way I'm running your binary on my system. I simply cannot trust it with my user account. Not only is this bad from a security perspective, it's also bad for maintenance. Over time, many end up in a situation where others cannot rebuild binaries from source at all, since binary distribution becomes the norm and the source->binary mechanism isn't maintained except on the developer's own system. Look at the Java community for an example of this. This is a problem because it means that others cannot easy be a member of the community working on the source. If you want to promote a community around the source (like Github does), you must also provide an easy way to rebuild binaries from the source. Unfortunately, there's no good answer here. Distributions like Debian have solved this problem - there are source packages, and there's a standard way of building them to get the binary packages. But there's quite a bit of arcane crud that's built up over the years, making it difficult to learn and understand Debian packaging. And the solution isn't cross platform. Each language community has produced their own solution (Rubygems, Python eggs, CPAN, etc) but none of them work with the packaging provided by any other language. To me, this may be cross platform, but not being cross language is just as big a problem.

j_s 13y ago
Hacker News hosted the following discussion regarding problems that have already occurred maintaining Go software due to reliance on binary distribution: "Golang dependency hell in Haunts The Manse game" https://news.ycombinator.com/item?id=5796597 https://news.ycombinator.com/item?id=5796597 In this specific instance it was more of an incorrect understanding of Go dependencies and misuse/lack of source control, but it seemed to at least come close to an echo of the issues you are raising. The specific quote from the comments: "There is no version control for dependencies".
- tptacek 13y ago
  That wasn't a binary distribution problem or anything like one, was it? That was as I remember it a development team that simply hadn't created a known-good repository of their third-party dependencies. While I grant you that's a common pitfall in the rubygem era, it's hard to feel much sympathy as someone who came up building commercial C code, where not having a one-command third-party build-world system was a cardinal sin.
  - conroy 13y ago
    Create a known-good repository of third-party dependencies is so easy in Go. Change the $GOPATH, `go get` all the dependencies and add that folder to version control.
    - tptacek 13y ago
      I think I agree. Also, for all the complaints about "magic" in Rails, there's nothing more magical in both Rubyland and Pythonistan than package management, and the explicitness of Golang's "package" system is a welcome contrast.
      - steveklabnik 13y ago
        > there's nothing more magical in both Rubyland and Pythonistan than package management, Could you expand on this a bit? I'm interested in your opinions, as it's hard to see from the inside. In Ruby, it's just things modifying the $LOAD_PATH, which should be basically the same as changing your $GOPATH, no?
        philsnow 13y ago
        I don't know much / anything about ruby package management, but if it's commonplace for ruby packages to change the LOAD_PATH (which I guess is similar to PYTHON_PATH / sys.path in python ?), that is really quite magical (by which I mean, causes unexpected behavior). What conroy is talking about is creating a static set of dependencies that you check into your own source control, because go doesn't let you specify e.g. github commit hashes to "pin" a package dependency to a certain revision. So a workaround for that is to (as a one-off) change your GOPATH ($GOPATH/src and $GOPATH/bin is where the standard "go get" / "go install" etc put things) to something temporary, "go get" your dependences (which will fetch them from github or wherever at the HEAD revision). Now run your test suite to make sure everything passes. If it does, great, take the current $GOPATH/src and check the contents of that into your own revision control. Et voila, ghetto-"pin"ning.
pkulak 13y ago
Nothing is stopping you from building the app from source instead, just like every other open source project. But at least with Go you have the _option_ of just downloading and running a binary if you trust the author.
- rlpb 13y ago
  This is true. I do have that option. But I very rarely trust the author. Most of the time I've never heard of the author. This is a good thing, and exactly why Github has been so successful. By nature, if there's something obscure that I need doing, and I find some software or software module that lets me do it, then chances are I won't know the author. This means that downloading and running the binary is suddenly a very obscure use case. It should be for you, too.
  - mdwrigh2 13y ago
    > This means that downloading and running the binary is suddenly a very obscure use case. It should be for you, too. A rare use case for you, but definitely not an obscure one in general. Besides, trusting the author, in this case, only matters if you're willing to audit the source to verify that nothing malicious is occurring -- something I'm not going to do in general. I mean, realistically, when was the last time you audited your network drivers? your OS? your browser?
    - rlpb 13y ago
      You don't have to verify the source yourself. Others can do it. I trust the Debian maintainer for my package. He's a real world person who has had his key signed by other Debian developers. The fact that the source can be verified also reduces the risk of malicious modification. It's not necessarily just the author who might be malicious, either. The author's computer where he runs his compiler might be compromised.
  - chimeracoder 13y ago
    > This means that downloading and running the binary is suddenly a very obscure use case. It should be for you, too. That's very much not true, especially when you start dealing with corporate/enterprise systems (and remember, Go is intended as a systems programming language). Just last week I heard Eliot Horowitz (CTO of 10Gen) talking about how it's advantageous for them to use Go for parts of their product, because they can provide guarantees about the software that they're running ("Hey, you're running the exact same file as what we're providing you, not fetching dependencies separately and interpreting things on-the-fly, etc.") From a security perspective, these corporate clients have already entered into contractual agreements where they don't really care to verify that the source and binary refer to the same code. If I'm paying a third-party company to provide me with software and services, I'm trusting them to provide security at multiple levels. I'm probably not even auditing their source code myself, so the idea that they would ship us a binary that didn't match the source code is something that wouldn't even enter my mind. What I've described above is not an obscure use case. Yes, there are other things that could go wrong that statically compiled binaries can't solve, but from the point of view of a bank or a large, non-tech corporation, it's one less link that could fail in the process.
coldtea 13y ago
>For me, this is worse. There's no audit trail. No way for me to verify that the binary of your app is the same as the source you provide. So there's no way I'm running your binary on my system. I simply cannot trust it with my user account. It's not for you then. For one, billions of people run tons of binary third party apps all day long. Second, when you deploy to people, they have to trust you. Whether they trust your code is irrelevant. Most of them are non-programmers anyway and wouldn't even check if you gave them a source code printout.

by rlpb 13y ago

j_s 13y ago

Hacker News hosted the following discussion regarding problems that have already occurred maintaining Go software due to reliance on binary distribution: "Golang dependency hell in Haunts The Manse game" https://news.ycombinator.com/item?id=5796597 https://news.ycombinator.com/item?id=5796597 In this specific instance it was more of an incorrect understanding of Go dependencies and misuse/lack of source control, but it seemed to at least come close to an echo of the issues you are raising. The specific quote from the comments: "There is no version control for dependencies".

tptacek 13y ago
That wasn't a binary distribution problem or anything like one, was it? That was as I remember it a development team that simply hadn't created a known-good repository of their third-party dependencies. While I grant you that's a common pitfall in the rubygem era, it's hard to feel much sympathy as someone who came up building commercial C code, where not having a one-command third-party build-world system was a cardinal sin.
- conroy 13y ago
  Create a known-good repository of third-party dependencies is so easy in Go. Change the $GOPATH, `go get` all the dependencies and add that folder to version control.
  - tptacek 13y ago
    I think I agree. Also, for all the complaints about "magic" in Rails, there's nothing more magical in both Rubyland and Pythonistan than package management, and the explicitness of Golang's "package" system is a welcome contrast.
    - steveklabnik 13y ago
      > there's nothing more magical in both Rubyland and Pythonistan than package management, Could you expand on this a bit? I'm interested in your opinions, as it's hard to see from the inside. In Ruby, it's just things modifying the $LOAD_PATH, which should be basically the same as changing your $GOPATH, no?
      - philsnow 13y ago
        I don't know much / anything about ruby package management, but if it's commonplace for ruby packages to change the LOAD_PATH (which I guess is similar to PYTHON_PATH / sys.path in python ?), that is really quite magical (by which I mean, causes unexpected behavior). What conroy is talking about is creating a static set of dependencies that you check into your own source control, because go doesn't let you specify e.g. github commit hashes to "pin" a package dependency to a certain revision. So a workaround for that is to (as a one-off) change your GOPATH ($GOPATH/src and $GOPATH/bin is where the standard "go get" / "go install" etc put things) to something temporary, "go get" your dependences (which will fetch them from github or wherever at the HEAD revision). Now run your test suite to make sure everything passes. If it does, great, take the current $GOPATH/src and check the contents of that into your own revision control. Et voila, ghetto-"pin"ning.

pkulak 13y ago

Nothing is stopping you from building the app from source instead, just like every other open source project. But at least with Go you have the _option_ of just downloading and running a binary if you trust the author.

rlpb 13y ago
This is true. I do have that option. But I very rarely trust the author. Most of the time I've never heard of the author. This is a good thing, and exactly why Github has been so successful. By nature, if there's something obscure that I need doing, and I find some software or software module that lets me do it, then chances are I won't know the author. This means that downloading and running the binary is suddenly a very obscure use case. It should be for you, too.
- mdwrigh2 13y ago
  > This means that downloading and running the binary is suddenly a very obscure use case. It should be for you, too. A rare use case for you, but definitely not an obscure one in general. Besides, trusting the author, in this case, only matters if you're willing to audit the source to verify that nothing malicious is occurring -- something I'm not going to do in general. I mean, realistically, when was the last time you audited your network drivers? your OS? your browser?
  - rlpb 13y ago
    You don't have to verify the source yourself. Others can do it. I trust the Debian maintainer for my package. He's a real world person who has had his key signed by other Debian developers. The fact that the source can be verified also reduces the risk of malicious modification. It's not necessarily just the author who might be malicious, either. The author's computer where he runs his compiler might be compromised.
- chimeracoder 13y ago
  > This means that downloading and running the binary is suddenly a very obscure use case. It should be for you, too. That's very much not true, especially when you start dealing with corporate/enterprise systems (and remember, Go is intended as a systems programming language). Just last week I heard Eliot Horowitz (CTO of 10Gen) talking about how it's advantageous for them to use Go for parts of their product, because they can provide guarantees about the software that they're running ("Hey, you're running the exact same file as what we're providing you, not fetching dependencies separately and interpreting things on-the-fly, etc.") From a security perspective, these corporate clients have already entered into contractual agreements where they don't really care to verify that the source and binary refer to the same code. If I'm paying a third-party company to provide me with software and services, I'm trusting them to provide security at multiple levels. I'm probably not even auditing their source code myself, so the idea that they would ship us a binary that didn't match the source code is something that wouldn't even enter my mind. What I've described above is not an obscure use case. Yes, there are other things that could go wrong that statically compiled binaries can't solve, but from the point of view of a bank or a large, non-tech corporation, it's one less link that could fail in the process.

coldtea 13y ago

>For me, this is worse. There's no audit trail. No way for me to verify that the binary of your app is the same as the source you provide. So there's no way I'm running your binary on my system. I simply cannot trust it with my user account. It's not for you then. For one, billions of people run tons of binary third party apps all day long. Second, when you deploy to people, they have to trust you. Whether they trust your code is irrelevant. Most of them are non-programmers anyway and wouldn't even check if you gave them a source code printout.