7 ms·

Why are you putting build artifacts into source control?

Xylakant 12y ago
Because they're intermediary steps of a process, regenerating them takes an hour and I don't feel like setting up everybody's environment to build them. Most people don't need the capability, but they need the result. Why are you asking?
- tedunangst 12y ago
  People get pretty religious about what should (or not) be checked in to version control.
  - Xylakant 12y ago
    I can tell from the downvotes :)
  - pyre 12y ago
    It's designed with a particular use-case in mind. When people complain that square pegs don't fit into round holes, it makes more sense for them to step back and evaluate what they are trying to accomplish, and the tools they are using the accomplish it.
- namdnay 12y ago
  I think the question asked was closer to "why aren't you using an artifact repository?" Nexus is pretty good, but if the language you are using isn't integrated well with gradle/maven you can always just use a shared drive fed by jenkins builds.
  - Xylakant 12y ago
    Valid question: Because the tooling does not think in terms of artifacts that are versioned in repositories, but rather in terms of "files" that are in a given location. I'm using a shared location, but every build requires modifying another file to point to the now current version. It's all solvable, but the easiest solution would be to just version the result. I could fix the tooling, but alas, I have other yaks to shave. It's an imperfect world.
    - pyre 12y ago
      > but every build requires modifying another file to point to the now current version If you could version the file in git, you would have to check in the new version of the file, so it's not like you're adding a step to have to update (e.g.) a symlink.
      - Xylakant 12y ago
        But I have a lot of files lying in a shared location that are named build-<datetime> and I can accidentally break revisions in the repo by moving/renaming/deleting any of these. That may be a feature to some people, but that's something I consider a weak spot. It's brittle and prone to breakage and I dislike brittle.
  - TeMPOraL 12y ago
    > Nexus is pretty good, but if the language you are using isn't integrated well with gradle/maven you can always just use a shared drive fed by jenkins builds. Here's where I start to have problems with contemporary development culture. You mentioned using Nexus, Gradle, Maven and Jenkins where the guy just want to keep some binaries along with the source code they're generated from. We're complicating things beyond reason nowadays.
    - angersock 12y ago
      Agreed. That said--and this is without knowing the exact build and tooling environment, so I may well be giving advice inappropriate to the situation at hand!--the second part of that "keep some binaries along with the source files they're generated from" is kind of an antipattern. If it takes too long to generate them from source, each time erry time, they need to fix that issue--least of all because slow builds mean slow testing, and slow testing means no testing. That's why I spent two days earlier this year moving a 30-minute build down to a 2.5 minute build.
      - Xylakant 12y ago
        Yes, your advice is sound for the general case but inapplicable in this specific incarnation of the problem.
        angersock 12y ago
        Out of curiosity, what is the gist of your setup? Why is this incarnation such a departure from the general case?
        Xylakant 12y ago
        The output of the build is basically a tool in itself. So most people don't need the build process, just the resulting tool. The input changes on pretty much a monthly basis and is not easily versioned. I could set up all dev machines to support the build and everybody could build it themselves from sources, but that would require me to * install all the required instruments for the ritual rain dance * teach the whole team how to do the ritual rain dance * support the people that break an arm or a leg doing the ritual rain dance So I prefer perform the dance on my machine, collect the tool and point the main dev environment to the right location. It's all scripted, so I kick off the job and grab a coffee. However, I want to keep old instances around so I can track when bugs crept in, so I can't just go overwrite the result, so I need to adjust the pointer every time. If I could just check the tool in with the regular dev setup that would be much easier, but - since we're using git - that would blow up quite quickly and overwhelm my disk space. (And folks would kill me for filling up their disks as well - rightfully so). That's something SVN or another centralized VCS would handle much more gracefully. In a gist, I have a use-case for fairly stupid versioned file storage with a push/pull api. No complicated merging, no branching, nothing. git-annex could do, but is overkill. There are some better solutions to what we're currently doing, but there's so many yaks to shave and so few razors.
- phkahler 12y ago
  >> Because they're intermediary steps of a process, regenerating them takes an hour and I don't feel like setting up everybody's environment to build them. Most people don't need the capability, but they need the result. Put your build tools in a repo. It should be easy to set up a new developer with a complete build system by getting it from the a repo. Now use make or anything that can check dependencies so those things are not regenerated unless they need to be. Always check in a freshly generated file along with its source.
  - Too 12y ago
    Setting up a build system can be a bitch. Not every developer needs to be able to build every obscure module of all your companies tooling. A .net developer shouldn't have to rebuild c++ boost just because he relies on a small native dll. In some cases the tooling of one developer can even conflict with the one of another, one requires python 2.7 to be in your PATH while the other requires 3.3, etc. I fully agree that it should be easy to set up but the reality is different, especially when you deal with legacy, rely on third party libraries or build open source projects from source.
deleted 12y ago
[deleted]
icefox 12y ago
What is annoying is when build artifacts are checked into source control systems along side of the source. There is very little way to exclude them or only check out the binaries, you can only have both. This happens all too often and I don't care if you want to keep using subversion you really should stop doing this. /src/foo.c /src/foo.o
Gorkys 12y ago
Not the OP, but in our product we have very large binaries that are the output of the product itself. Each time a change is made to the program, the output needs to be compared to the existing output first automatically and if the output isn't byte-by-byte identical then manually (visually). So essentially, without large binaries in source control we'd lose testing.

Because they're intermediary steps of a process, regenerating them takes an hour and I don't feel like setting up everybody's environment to build them. Most people don't need the capability, but they need the result. Why are you asking?

tedunangst 12y ago
People get pretty religious about what should (or not) be checked in to version control.
- Xylakant 12y ago
  I can tell from the downvotes :)
- pyre 12y ago
  It's designed with a particular use-case in mind. When people complain that square pegs don't fit into round holes, it makes more sense for them to step back and evaluate what they are trying to accomplish, and the tools they are using the accomplish it.
namdnay 12y ago
I think the question asked was closer to "why aren't you using an artifact repository?" Nexus is pretty good, but if the language you are using isn't integrated well with gradle/maven you can always just use a shared drive fed by jenkins builds.
- Xylakant 12y ago
  Valid question: Because the tooling does not think in terms of artifacts that are versioned in repositories, but rather in terms of "files" that are in a given location. I'm using a shared location, but every build requires modifying another file to point to the now current version. It's all solvable, but the easiest solution would be to just version the result. I could fix the tooling, but alas, I have other yaks to shave. It's an imperfect world.
  - pyre 12y ago
    > but every build requires modifying another file to point to the now current version If you could version the file in git, you would have to check in the new version of the file, so it's not like you're adding a step to have to update (e.g.) a symlink.
    - Xylakant 12y ago
      But I have a lot of files lying in a shared location that are named build-<datetime> and I can accidentally break revisions in the repo by moving/renaming/deleting any of these. That may be a feature to some people, but that's something I consider a weak spot. It's brittle and prone to breakage and I dislike brittle.
- TeMPOraL 12y ago
  > Nexus is pretty good, but if the language you are using isn't integrated well with gradle/maven you can always just use a shared drive fed by jenkins builds. Here's where I start to have problems with contemporary development culture. You mentioned using Nexus, Gradle, Maven and Jenkins where the guy just want to keep some binaries along with the source code they're generated from. We're complicating things beyond reason nowadays.
  - angersock 12y ago
    Agreed. That said--and this is without knowing the exact build and tooling environment, so I may well be giving advice inappropriate to the situation at hand!--the second part of that "keep some binaries along with the source files they're generated from" is kind of an antipattern. If it takes too long to generate them from source, each time erry time, they need to fix that issue--least of all because slow builds mean slow testing, and slow testing means no testing. That's why I spent two days earlier this year moving a 30-minute build down to a 2.5 minute build.
    - Xylakant 12y ago
      Yes, your advice is sound for the general case but inapplicable in this specific incarnation of the problem.
      - angersock 12y ago
        Out of curiosity, what is the gist of your setup? Why is this incarnation such a departure from the general case?
        Xylakant 12y ago
        The output of the build is basically a tool in itself. So most people don't need the build process, just the resulting tool. The input changes on pretty much a monthly basis and is not easily versioned. I could set up all dev machines to support the build and everybody could build it themselves from sources, but that would require me to * install all the required instruments for the ritual rain dance * teach the whole team how to do the ritual rain dance * support the people that break an arm or a leg doing the ritual rain dance So I prefer perform the dance on my machine, collect the tool and point the main dev environment to the right location. It's all scripted, so I kick off the job and grab a coffee. However, I want to keep old instances around so I can track when bugs crept in, so I can't just go overwrite the result, so I need to adjust the pointer every time. If I could just check the tool in with the regular dev setup that would be much easier, but - since we're using git - that would blow up quite quickly and overwhelm my disk space. (And folks would kill me for filling up their disks as well - rightfully so). That's something SVN or another centralized VCS would handle much more gracefully. In a gist, I have a use-case for fairly stupid versioned file storage with a push/pull api. No complicated merging, no branching, nothing. git-annex could do, but is overkill. There are some better solutions to what we're currently doing, but there's so many yaks to shave and so few razors.
phkahler 12y ago
>> Because they're intermediary steps of a process, regenerating them takes an hour and I don't feel like setting up everybody's environment to build them. Most people don't need the capability, but they need the result. Put your build tools in a repo. It should be easy to set up a new developer with a complete build system by getting it from the a repo. Now use make or anything that can check dependencies so those things are not regenerated unless they need to be. Always check in a freshly generated file along with its source.
- Too 12y ago
  Setting up a build system can be a bitch. Not every developer needs to be able to build every obscure module of all your companies tooling. A .net developer shouldn't have to rebuild c++ boost just because he relies on a small native dll. In some cases the tooling of one developer can even conflict with the one of another, one requires python 2.7 to be in your PATH while the other requires 3.3, etc. I fully agree that it should be easy to set up but the reality is different, especially when you deal with legacy, rely on third party libraries or build open source projects from source.

deleted 12y ago

[deleted]

icefox 12y ago

What is annoying is when build artifacts are checked into source control systems along side of the source. There is very little way to exclude them or only check out the binaries, you can only have both. This happens all too often and I don't care if you want to keep using subversion you really should stop doing this. /src/foo.c /src/foo.o

Gorkys 12y ago

Not the OP, but in our product we have very large binaries that are the output of the product itself. Each time a change is made to the program, the output needs to be compared to the existing output first automatically and if the output isn't byte-by-byte identical then manually (visually). So essentially, without large binaries in source control we'd lose testing.