5 ms·

In short: not many. There is a serious tension between branching workflows and continuous integration (I mean the technique of merge-early-merge-often, not "th

In short: not many. There is a serious tension between branching workflows and continuous integration (I mean the technique of merge-early-merge-often, not "the Jenkins server that runs the tests"). The whole point of continuous integration is that you do not wait until you are done to try to merge your changes. Merges in themselves are painful and not only are there fewer conflicts but they are less serious when everyone is (with few exceptions) just doing little merges all the time. Moreover, having your changes merged in quick also means that you will get feedback about your changes before you're "done". If your change doesn't break a test but breaks functionality in a subtle way there is a much greater chance that someone will notice it and if you're doing something wrong that you don't know about, again, it's much more likely that someone will tell you. In bigger teams or in complicated domains this becomes more and more important. Decent and related article: http://martinfowler.com/bliki/FeatureBranch.html http://martinfowler.com/bliki/FeatureBranch.html

flexd 13y ago
Why can't you just run continuous integration on all the branches?
- baq 13y ago
  have fun handling 100x build volume...
  - acdha 13y ago
    Builds are one of the easiest tasks to parallelize since they're supposed to be completely isolated by design. If you have that many active developers, you also have enough budget for extra hardware – even if you went with e.g. EC2 on-demand, you can buy a lot of capacity for what you spend on a single engineer-hour.
- mullr 13y ago
  Because that would be continuous builds on unintegrated-branches. The point of continuous integration is that everybody is reconciling their work with everybody else's all the time, thus helping you find nasty integration surprises early and while you can still do something about them.
  - calpaterson 13y ago
    I tried to explain that running tests on something is not CI, but I apparently failed :(
  - drunkenfly 13y ago
    Not if you're rebasing your branch on every change in master.
    - mullr 13y ago
      A good point. Or merging in changes from master, if you like it that way.
    - calpaterson 13y ago
      Not even then - because everyone else on the team is doing the same
- aidenn0 13y ago
  That's not CI. Example with 3 branches: Trunk Feature1 Feature2 If you nightly merge Trunk into Feature1 and nightly merge Trunk into Feature2 and test all 3 branches, you still haven't tested the integration of Feature1 and Feature2. With enough developers working on enough features the odds that at least one pair of features won't work together approaches unity. People have tried all sorts of methodology and design, &ct. to try and keep this from happening, and they haven't succeeded yet. On the other hand, if feature branches get merged into trunk as soon as they aren't completely broken, and with a way to disable them, then you will very quickly discover that the features don't play well together.
sgk284 13y ago
You seem to imply that working on a feature branch means you wait until you're finished the feature to try and merge your changes, which would be an absolutely insane workflow in any sizable org. While working on a feature branch, you should be pulling in the latest from origin/master on a pretty regular basis such that you never have a big merge. You should be merging early and merging often. The benefit here is that you get to work on functionality without hindering anyone else (by putting the codebase in a functioning, but transitional state), while getting to integrate on your own schedule. By the time it's ready to push to everyone else, there should be no merge. It'll be a fast-forward. Feature branches, with a policy of merge early and often, make life substantially easier than traditional continuous integration workflows. Having everybody checking in in-progress work to the same place is a nightmare. When I'm working on a feature, I want to be able to commit in small bite-sized chunks that allow me to explore solutions, rollback changes, rebase on-top of whatever I want, and ultimately squash my work into a single cleanly merged chunk of work. Any other workflow is limiting and restricting many of the benefits of git.
- calpaterson 13y ago
  I think we have a language difference: when I say "merging" I don't mean, "incorporate changes from trunk" I mean "you branch and trunk become the same". > getting to integrate on your own schedule Essentially, this is what I oppose, for the reasons I described originally. > Any other workflow is limiting and restricting many of the benefits of git. The point of git is to make your workflow better, the point of your workflow is not to make git better. Yeah "branching is easy", and that's good, but it doesn't necessarily mean that you want to start using branches heavily (especially long-lived ones). Bad version control systems were only part of the reason to avoid branches.
lmm 13y ago
The problem I have with the CI approach is that it means you have to push a lot more of your program's integration to runtime, which is a worse place to detect failures. When adding a new feature ideally one writes the use case, which won't compile, and then implements the things it calls, which won't compile, and so on - if you're using types properly, then once you've added the front of a new feature your code won't compile until the entire feature implementation is correct. This is a wonderful way to code, eliminating huge classes of potential bugs, but CI seems fundamentally incompatible with it.
- calpaterson 13y ago
  Ok, so first of all, not everyone uses types and I for one think it is ok to use dynamically typed languages. Second, I think you have a different vision of CI from me (maybe more frequent). I'm not advocating pushing code that is broken, I'm advocating pushing code that is working (but maybe is incomplete).
alinajaf 13y ago
Quick question, have you ever personally experienced this work on a team, or is this just theory? I'm genuinely interested in collecting good examples of it working. The other article about flickr doing this with feature toggles seemed to be a good fit. <Rant> The reason I ask is that earlier this year, I was working with a client who spent upwards of $3 million USD on a content managed website with a huge consultancy that I can neither confirm nor deny is heavily affiliated with the author of the article you may or may not have posted a link to. They were left with a config-driven feature-toggle system that can only be described as a Rube-Goldberg monstrosity. If statements from old feature toggles littering the codebase and different features toggled on or off in environments that no one knew the meaning of. My favourite was when we discovered after days of trawling through the code that a feature toggle was being used to switch a feature off rather than on. There was also a team-wide ban on branching of any kind, including local branching. This belies a misunderstanding of how git works more than anything else. I went ahead and kept as many local branches as I bloody well felt like, because there's no way they'd know the difference. </Rant>
- deleted 13y ago
  [deleted]
- Chris_Newton 13y ago
  I sympathise with that concern, and anecdotally I’m still waiting to encounter a feature-toggled project in my own work where the maintenance overheads didn’t quickly become painful out of all proportion to the benefits. It was bad enough in the days of #ifdef driven development in C, where the meaning of your code and even your entire software design could change based on an arbitrary set of macros that didn’t respect scope, didn’t offer any type safety, and might be defined anywhere: a random header file, a makefile setting some command line option, or just some environment variable on one particular developer’s PC that they set three years ago and forgot about. Making a controlled, reproducible build that you actually trust to be what you think it is is very difficult in that kind of environment. It’s even worse with dynamically typed languages, because now you don’t even have compile-time warnings about redundant code or type safety guarantees on the code inside the feature checks to prove it would run sensibly. To me, it feels like the whole strategy plays right into the major downsides of using dynamic types. No doubt some people would disagree with me, and would never dream of writing code without comprehensive test suites that back it up and are known to have 100% coverage with every possible combination of features toggled in or out. (If you really think you are in this position, I know a guy who’d like to offer you an exclusive financial deal guaranteed to make you rich, if you just send a 25K deposit up-front in unmarked, non-sequential notes to his mailbox in Nigeria.) What really bugs me is that this whole feature toggling premise seems to be caused by the tail wagging the dog. Feature toggling is advocated as a way to do continuous integration. CI is advocated as a way to avoid nightmare merges. However, the existence of nightmare merges presupposes that you can have multiple developers simultaneously working on the same part of your code, making sufficiently intricate changes that merging will cause headaches, but for different reasons and without being aware of each other. Otherwise, they could just communicate effectively, and merge intelligently as they go along if that makes sense. How can any development team possibly get into that situation without some spectacular, fundamental failure of project management and technical leadership? That kind of failure is essentially a social problem, so it’s doubtful that any kind of technical solution could solve it reliably, whether a branching model, or a CI-not-branching model, or any other model for that matter.
kristoffer 13y ago
For me the optimal (current best thinking) way of doing continuous integration is to consider the "master" branch to be the integration branch. All tests should always be working here. It should not be possible to deliver a feature branch which breaks the regression tests on master. Thus I don't like unnecessary merges of unfinished work, it increases the risk for breaking the tests, which will slow down development. If you rebase your feature branch continuously then you achieve what you are asking for without merging unfinished work.

flexd 13y ago

Why can't you just run continuous integration on all the branches?

baq 13y ago
have fun handling 100x build volume...
- acdha 13y ago
  Builds are one of the easiest tasks to parallelize since they're supposed to be completely isolated by design. If you have that many active developers, you also have enough budget for extra hardware – even if you went with e.g. EC2 on-demand, you can buy a lot of capacity for what you spend on a single engineer-hour.
mullr 13y ago
Because that would be continuous builds on unintegrated-branches. The point of continuous integration is that everybody is reconciling their work with everybody else's all the time, thus helping you find nasty integration surprises early and while you can still do something about them.
- calpaterson 13y ago
  I tried to explain that running tests on something is not CI, but I apparently failed :(
- drunkenfly 13y ago
  Not if you're rebasing your branch on every change in master.
  - mullr 13y ago
    A good point. Or merging in changes from master, if you like it that way.
  - calpaterson 13y ago
    Not even then - because everyone else on the team is doing the same
aidenn0 13y ago
That's not CI. Example with 3 branches: Trunk Feature1 Feature2 If you nightly merge Trunk into Feature1 and nightly merge Trunk into Feature2 and test all 3 branches, you still haven't tested the integration of Feature1 and Feature2. With enough developers working on enough features the odds that at least one pair of features won't work together approaches unity. People have tried all sorts of methodology and design, &ct. to try and keep this from happening, and they haven't succeeded yet. On the other hand, if feature branches get merged into trunk as soon as they aren't completely broken, and with a way to disable them, then you will very quickly discover that the features don't play well together.

sgk284 13y ago

You seem to imply that working on a feature branch means you wait until you're finished the feature to try and merge your changes, which would be an absolutely insane workflow in any sizable org. While working on a feature branch, you should be pulling in the latest from origin/master on a pretty regular basis such that you never have a big merge. You should be merging early and merging often. The benefit here is that you get to work on functionality without hindering anyone else (by putting the codebase in a functioning, but transitional state), while getting to integrate on your own schedule. By the time it's ready to push to everyone else, there should be no merge. It'll be a fast-forward. Feature branches, with a policy of merge early and often, make life substantially easier than traditional continuous integration workflows. Having everybody checking in in-progress work to the same place is a nightmare. When I'm working on a feature, I want to be able to commit in small bite-sized chunks that allow me to explore solutions, rollback changes, rebase on-top of whatever I want, and ultimately squash my work into a single cleanly merged chunk of work. Any other workflow is limiting and restricting many of the benefits of git.

calpaterson 13y ago
I think we have a language difference: when I say "merging" I don't mean, "incorporate changes from trunk" I mean "you branch and trunk become the same". > getting to integrate on your own schedule Essentially, this is what I oppose, for the reasons I described originally. > Any other workflow is limiting and restricting many of the benefits of git. The point of git is to make your workflow better, the point of your workflow is not to make git better. Yeah "branching is easy", and that's good, but it doesn't necessarily mean that you want to start using branches heavily (especially long-lived ones). Bad version control systems were only part of the reason to avoid branches.

lmm 13y ago

The problem I have with the CI approach is that it means you have to push a lot more of your program's integration to runtime, which is a worse place to detect failures. When adding a new feature ideally one writes the use case, which won't compile, and then implements the things it calls, which won't compile, and so on - if you're using types properly, then once you've added the front of a new feature your code won't compile until the entire feature implementation is correct. This is a wonderful way to code, eliminating huge classes of potential bugs, but CI seems fundamentally incompatible with it.

calpaterson 13y ago
Ok, so first of all, not everyone uses types and I for one think it is ok to use dynamically typed languages. Second, I think you have a different vision of CI from me (maybe more frequent). I'm not advocating pushing code that is broken, I'm advocating pushing code that is working (but maybe is incomplete).

alinajaf 13y ago

Quick question, have you ever personally experienced this work on a team, or is this just theory? I'm genuinely interested in collecting good examples of it working. The other article about flickr doing this with feature toggles seemed to be a good fit. <Rant> The reason I ask is that earlier this year, I was working with a client who spent upwards of $3 million USD on a content managed website with a huge consultancy that I can neither confirm nor deny is heavily affiliated with the author of the article you may or may not have posted a link to. They were left with a config-driven feature-toggle system that can only be described as a Rube-Goldberg monstrosity. If statements from old feature toggles littering the codebase and different features toggled on or off in environments that no one knew the meaning of. My favourite was when we discovered after days of trawling through the code that a feature toggle was being used to switch a feature off rather than on. There was also a team-wide ban on branching of any kind, including local branching. This belies a misunderstanding of how git works more than anything else. I went ahead and kept as many local branches as I bloody well felt like, because there's no way they'd know the difference. </Rant>

deleted 13y ago
[deleted]
Chris_Newton 13y ago
I sympathise with that concern, and anecdotally I’m still waiting to encounter a feature-toggled project in my own work where the maintenance overheads didn’t quickly become painful out of all proportion to the benefits. It was bad enough in the days of #ifdef driven development in C, where the meaning of your code and even your entire software design could change based on an arbitrary set of macros that didn’t respect scope, didn’t offer any type safety, and might be defined anywhere: a random header file, a makefile setting some command line option, or just some environment variable on one particular developer’s PC that they set three years ago and forgot about. Making a controlled, reproducible build that you actually trust to be what you think it is is very difficult in that kind of environment. It’s even worse with dynamically typed languages, because now you don’t even have compile-time warnings about redundant code or type safety guarantees on the code inside the feature checks to prove it would run sensibly. To me, it feels like the whole strategy plays right into the major downsides of using dynamic types. No doubt some people would disagree with me, and would never dream of writing code without comprehensive test suites that back it up and are known to have 100% coverage with every possible combination of features toggled in or out. (If you really think you are in this position, I know a guy who’d like to offer you an exclusive financial deal guaranteed to make you rich, if you just send a 25K deposit up-front in unmarked, non-sequential notes to his mailbox in Nigeria.) What really bugs me is that this whole feature toggling premise seems to be caused by the tail wagging the dog. Feature toggling is advocated as a way to do continuous integration. CI is advocated as a way to avoid nightmare merges. However, the existence of nightmare merges presupposes that you can have multiple developers simultaneously working on the same part of your code, making sufficiently intricate changes that merging will cause headaches, but for different reasons and without being aware of each other. Otherwise, they could just communicate effectively, and merge intelligently as they go along if that makes sense. How can any development team possibly get into that situation without some spectacular, fundamental failure of project management and technical leadership? That kind of failure is essentially a social problem, so it’s doubtful that any kind of technical solution could solve it reliably, whether a branching model, or a CI-not-branching model, or any other model for that matter.

kristoffer 13y ago

For me the optimal (current best thinking) way of doing continuous integration is to consider the "master" branch to be the integration branch. All tests should always be working here. It should not be possible to deliver a feature branch which breaks the regression tests on master. Thus I don't like unnecessary merges of unfinished work, it increases the risk for breaking the tests, which will slow down development. If you rebase your feature branch continuously then you achieve what you are asking for without merging unfinished work.