7 ms·

How is this Go's fault? Just because you can pull code automatically from a remote repository doesn't mean you always should. It seems to me that keeping the en

by Udo 13y ago

How is this Go's fault? Just because you can pull code automatically from a remote repository doesn't mean you always should. It seems to me that keeping the entirety of the code necessary to compile a project in a local build directory is a very good idea if you don't want to compile against moving targets.

nilsbunger 13y ago
or at least make sure a remote Github reference is for a specific checkout!
- dkulchenko 13y ago
  IIRC, Go does not support importing anything but HEAD (yet, at least).
  - thepicard 13y ago
    Yeah, but you can use any number of systems to keep track of it if needed. I'm using git submodules in my current project, which keeps track of what commit you are using. Submodules are flawed in a lot of ways, but as a simple way to to keep a pointer to the appropriate version of an external project they are great. It's a little extra work to set them up, but it's fairly marginal. Check them out with go get, add the repo after the fact with git submodule, and after that just use git submodule to always grab the correct versions.
  - ominous_prime 13y ago
    Go supports some versioning. It will checkout from: 1. A tag named for the version of go used, or 2. A branch named for the version of go used, or 3. The master branch But no, you can't spec a version to import explicitly.
    - MetaCosm 13y ago
      Transitive dependencies (pkgA imports pkgB@v1 but my code need pkgB@v2 which is incompatible with v1) are the thing of nightmares in large systems development. This is a decided on feature by the golang team, not an oversite or something beyond their technical capabilities.
- Udo 13y ago
  Exactly! But trusting a remote repo to contain an essential part required to build the project is just extremely short-sighted, no matter what language is being used.
  - robryan 13y ago
    I think it is okay, as long as you have a backup plan. Can be advantageous to keep the code required to build the project small and have people pull dependencies as required.
- mercurial 13y ago
  Github is not a substitute for a packaging system.
tiredofcareer 13y ago
You are correct. Go's remote imports are dangerous for long-term project maintenance, but the feature is still useful for quick, throwaway projects. Go badly needs two things: 1. A best practice that dictates that you never import from remote repositories in production, long-term code; the feature is fine for one-offs and experimentation, but the article summarizes only one way this style of work can lead you in to a maintenance world of pain. What happens if the repo you're importing from Github is deleted? What do you do for fresh clones? You're going to end up changing the URL anyway. I feel the Go community has kind of glossed over this (and I like Go). 2. An equivalent of CPAN or PyPI, which you could then import from in concert with a tool to manage those dependencies, a la: import ( "cgan/video/graphics/opengl" ) This model works for CPAN, PyPI, and so on for a reason, and that reason is avoiding several of the dependency/merge hells that remote repos can create. CPAN provides Perl a lot, such as distributed testing in a variety of environments. I personally think such a thing is necessary for long-term maintenance of any software project that utilizes third-party libraries. This is one of Google's oversights in Go, because they have an (obviously) different take on third-party code. Here's a good case: Developer A checks out the code clean. Five minutes later, developer B checks out the code clean. In both cases, your "go get" bootstrap script fetches two different commits, because in that five minutes, upstream committed a bug. Developer B cannot build or, worse, can build but has several tests fail for unknown reasons or, even worse, the program no longer functions properly. Developer A has none of those problems. In a world with a CPAN-like, developer B can see that he has 0.9.1 and developer A has 0.9.0, developer B can commit "foo/bar: =0.9.0" to the project's dependency file, then everybody else doesn't suffer the same fate. In the current world, you're either massaging your local fork to keep the new commit out, or any other troublesome, non-scalable approach to this. Building large software projects against a repository never works. You need tested, versioned, cut releases to build against, not master HEAD. It only takes one bad upstream commit to entirely torpedo your build, and you've now completely removed the ability to qualify a new library version against the rest of your code base. Other people are suggesting "well, maintain your own forks," so you're basically moving merge hell from one place to another. I, personally, have better things to do with my time; I've seen (Java) apps with dozens of dependencies before, and keeping dozens of repositories remotely stable for a team of people will rapidly turn into multiple full-time jobs. Do you want to hire two maintenance monkeys[0] to constantly keep your build green by massaging upstream repositories, or do you want to hire two feature developers? Exactly. I've started writing a CPAN-like for Go a couple times but I'm always held back by these threads: https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/c-s4_uxamf8 https://groups.google.com/forum/?fromgroups=#!topic/golang-n... https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/-UG-lF_Ey-o https://groups.google.com/forum/?fromgroups=#!topic/golang-n... The second one highlighting how difficult Go is to package as a language -- my personal opinion is treat Go just like C and distribute binary libraries in libpackage, then the source in libpackage-src. If one message in that thread is true and binaries refuse to compile with different version compilers, I'm troubled about Go long-term. [0]: I'm not calling all build engineers maintenance monkeys. I'm saying the hypothetical job we just created is a monkey job. I love you, build engineers, you keep me green.
wereHamster 13y ago
Maintain your own forks of the libraries you need. Commit your changes there, and also submit upstream. Pull upstream changes back to your fork when you are ready. You have this problem with any language or tool that uses github directly as the registry.
- npsimons 13y ago
  Maintain your own forks of the libraries you need. Exactly this. This is just one reason I really like to "git clone --mirror" onto my server, then clone from that, and work from there.
  - beagle3 13y ago
    What's the benefit of that, as opposed to cloning upstream and working on that clone? How does the intermediate mirror help?
    - npsimons 13y ago
      I've run into the problem (with some Emacs add-ons in particular), of changing something on one machine, only to miss that change in another. So, if I've got my own "central" repo, I can push changes to that and pull them down to other machines. Since I site the repos on a server I have online already, it is also trivially easy to make it publicly "pullable" for others if need be.
azernik 13y ago
It sounds like it's even worse than that - the developer was maintaining a fork of the upstream projects, but didn't put it in version control. That is, he/she used Go's github import mechanism to get a copy of a project onto their own machine, and edited the locally cached copy. So of course, when other people tried to build, the libraries didn't match up. The real issue here seems to be that this was a half-finished project, at a very early stage of development, and the developer probably never intended their local scratch copy to be used in production [http://www.joystiq.com/2012/10/24/haunts-anatomy-of-a-kickstarter-collapse/ http://www.joystiq.com/2012/10/24/haunts-anatomy-of-a-kickst...]

by Udo 13y ago

nilsbunger 13y ago

or at least make sure a remote Github reference is for a specific checkout!

dkulchenko 13y ago
IIRC, Go does not support importing anything but HEAD (yet, at least).
- thepicard 13y ago
  Yeah, but you can use any number of systems to keep track of it if needed. I'm using git submodules in my current project, which keeps track of what commit you are using. Submodules are flawed in a lot of ways, but as a simple way to to keep a pointer to the appropriate version of an external project they are great. It's a little extra work to set them up, but it's fairly marginal. Check them out with go get, add the repo after the fact with git submodule, and after that just use git submodule to always grab the correct versions.
- ominous_prime 13y ago
  Go supports some versioning. It will checkout from: 1. A tag named for the version of go used, or 2. A branch named for the version of go used, or 3. The master branch But no, you can't spec a version to import explicitly.
  - MetaCosm 13y ago
    Transitive dependencies (pkgA imports pkgB@v1 but my code need pkgB@v2 which is incompatible with v1) are the thing of nightmares in large systems development. This is a decided on feature by the golang team, not an oversite or something beyond their technical capabilities.
Udo 13y ago
Exactly! But trusting a remote repo to contain an essential part required to build the project is just extremely short-sighted, no matter what language is being used.
- robryan 13y ago
  I think it is okay, as long as you have a backup plan. Can be advantageous to keep the code required to build the project small and have people pull dependencies as required.
mercurial 13y ago
Github is not a substitute for a packaging system.

tiredofcareer 13y ago

You are correct. Go's remote imports are dangerous for long-term project maintenance, but the feature is still useful for quick, throwaway projects. Go badly needs two things: 1. A best practice that dictates that you never import from remote repositories in production, long-term code; the feature is fine for one-offs and experimentation, but the article summarizes only one way this style of work can lead you in to a maintenance world of pain. What happens if the repo you're importing from Github is deleted? What do you do for fresh clones? You're going to end up changing the URL anyway. I feel the Go community has kind of glossed over this (and I like Go). 2. An equivalent of CPAN or PyPI, which you could then import from in concert with a tool to manage those dependencies, a la: import ( "cgan/video/graphics/opengl" ) This model works for CPAN, PyPI, and so on for a reason, and that reason is avoiding several of the dependency/merge hells that remote repos can create. CPAN provides Perl a lot, such as distributed testing in a variety of environments. I personally think such a thing is necessary for long-term maintenance of any software project that utilizes third-party libraries. This is one of Google's oversights in Go, because they have an (obviously) different take on third-party code. Here's a good case: Developer A checks out the code clean. Five minutes later, developer B checks out the code clean. In both cases, your "go get" bootstrap script fetches two different commits, because in that five minutes, upstream committed a bug. Developer B cannot build or, worse, can build but has several tests fail for unknown reasons or, even worse, the program no longer functions properly. Developer A has none of those problems. In a world with a CPAN-like, developer B can see that he has 0.9.1 and developer A has 0.9.0, developer B can commit "foo/bar: =0.9.0" to the project's dependency file, then everybody else doesn't suffer the same fate. In the current world, you're either massaging your local fork to keep the new commit out, or any other troublesome, non-scalable approach to this. Building large software projects against a repository never works. You need tested, versioned, cut releases to build against, not master HEAD. It only takes one bad upstream commit to entirely torpedo your build, and you've now completely removed the ability to qualify a new library version against the rest of your code base. Other people are suggesting "well, maintain your own forks," so you're basically moving merge hell from one place to another. I, personally, have better things to do with my time; I've seen (Java) apps with dozens of dependencies before, and keeping dozens of repositories remotely stable for a team of people will rapidly turn into multiple full-time jobs. Do you want to hire two maintenance monkeys[0] to constantly keep your build green by massaging upstream repositories, or do you want to hire two feature developers? Exactly. I've started writing a CPAN-like for Go a couple times but I'm always held back by these threads: https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/c-s4_uxamf8 https://groups.google.com/forum/?fromgroups=#!topic/golang-n... https://groups.google.com/forum/?fromgroups=#!topic/golang-nuts/-UG-lF_Ey-o https://groups.google.com/forum/?fromgroups=#!topic/golang-n... The second one highlighting how difficult Go is to package as a language -- my personal opinion is treat Go just like C and distribute binary libraries in libpackage, then the source in libpackage-src. If one message in that thread is true and binaries refuse to compile with different version compilers, I'm troubled about Go long-term. [0]: I'm not calling all build engineers maintenance monkeys. I'm saying the hypothetical job we just created is a monkey job. I love you, build engineers, you keep me green.

wereHamster 13y ago

Maintain your own forks of the libraries you need. Commit your changes there, and also submit upstream. Pull upstream changes back to your fork when you are ready. You have this problem with any language or tool that uses github directly as the registry.

npsimons 13y ago
Maintain your own forks of the libraries you need. Exactly this. This is just one reason I really like to "git clone --mirror" onto my server, then clone from that, and work from there.
- beagle3 13y ago
  What's the benefit of that, as opposed to cloning upstream and working on that clone? How does the intermediate mirror help?
  - npsimons 13y ago
    I've run into the problem (with some Emacs add-ons in particular), of changing something on one machine, only to miss that change in another. So, if I've got my own "central" repo, I can push changes to that and pull them down to other machines. Since I site the repos on a server I have online already, it is also trivially easy to make it publicly "pullable" for others if need be.

azernik 13y ago

It sounds like it's even worse than that - the developer was maintaining a fork of the upstream projects, but didn't put it in version control. That is, he/she used Go's github import mechanism to get a copy of a project onto their own machine, and edited the locally cached copy. So of course, when other people tried to build, the libraries didn't match up. The real issue here seems to be that this was a half-finished project, at a very early stage of development, and the developer probably never intended their local scratch copy to be used in production [http://www.joystiq.com/2012/10/24/haunts-anatomy-of-a-kickstarter-collapse/ http://www.joystiq.com/2012/10/24/haunts-anatomy-of-a-kickst...]