3 ms·

Architecture people tend to speak in trivial examples. Swapping out an algorithm with a backwards compatible one? Good for you! It doesn't matter how you do it

by xorcist 11y ago

Architecture people tend to speak in trivial examples. Swapping out an algorithm with a backwards compatible one? Good for you! It doesn't matter how you do it because it's the most trivial example one could think of. (A more realistic example would be new functionality that touches large parts of the code base, or compliance with a new API that the payment processor switches that has more data on every payment as it crosses our systems.) I never understood why some people are scared of branching. It seems to work pretty well for Linux development. Is your project really that much more complex? As long as you merge continously from upstream, you're golden. What's the problem you're trying to solve, exactly? I _do_ have some reservations against #ifdef-sprinkled code. And it's not an unlikely scenario that your feature toggles turn into something very similar, if conditions change and maybe they're not that temporary anymore.

samwgoldman 11y ago
I think the calculus changes when you consider teams with 1000s of programmers committing 100s of changes per day. Keeping feature branches up-to-date can pretty quickly adds up to a lot of time and creates a lot of friction for refactoring/cleanup-type changes. Feature toggles enable some additional cool stuff like partial/incremental rollout and A/B testing, which really pay dividends.
- lucaspiller 11y ago
  I've never worked on codebases with so many programmers, but I kind of feel that's if that's how you work, 'you're doing it wrong'. Why would you have so many programmers working on a single codebase, why not split it up in distinct parts? If that's not possible because so much stuff is shared, how can you possibly work effectively with so many people changing so many things?
  - geocar 11y ago
    1000 programmers on a single codebase means: • "master" always builds • "master" is always stable Any testing has to be done with in-code branches, and has to be built in-place incrementally so that other developers can see you touching stuff (and can be defensive about their tasks).
qwer 11y ago
> I never understood why some people are scared of branching. Well this is branching (in the same way that if..else is branching), just not in the VCS. What's the advantage? This methodology can allow code to go to production for some or all users, either as dark launches ( http://agiletesting.blogspot.com/2009/07/dark-launching-and-other-lessons-from.html http://agiletesting.blogspot.com/2009/07/dark-launching-and-... ) or as production code. VCS branching gives you none of that, and sometimes you need to roll things out sloooowly, either for quality control or for capacity monitoring. > It seems to work pretty well for Linux development. Well for example, if you're following the linux development model for webops, you're almost certainly doing it wrong. I'm sure your website doesn't have release candidates and LTS versions all at different URLs. > As long as you merge continuously from upstream, you're golden. This is not really true either. It's fine if you're the only one on a team following that model, but if everyone on your team follows that model, you're delaying collaboration and causing merge debt. Upstream ends up like a ghost town until someone like you dumps a week worth of work on it just before you're about to dump your week's worth too. > I _do_ have some reservations against #ifdef-sprinkled code. This is definitely a downside! You have to use a lot of discipline to keep this minimal and well-organized.
mikekchar 11y ago
I'm not quite sure I understand all of your comment, but I can respond to why one might want feature toggles rather than branches. One of the activities that can really improve the quality of code on a team is continuous integration. Now, I don't mean setting up a "CI" server to run tests for you when you push your code. I mean actually merging your code with your fellow programmers every 20 minutes or so. The reason this is such a powerful technique is that it gets people to see what you are doing pretty much as soon as you do it. If you do something stupid (as we all do), they complain instantly because they will merge in your code every 20 minutes. If you do CI well, there are no lingering surprises or hard decisions of "Well, maybe you should start all over again". The problem is that all the code is mixed up. This is completely fine if you have 1 or 2 week sprints and deploy at the end of the sprint. The problem is that this kills another really power technique called "continuous deployment". Ideally, we would like to deploy as soon as a story is finished. But if it's mixed up with a whole bunch of other code, then you need to make sure that the other code isn't going to break something. Enter feature toggles. It's a challenge to support both CI and CD at the same time. You can never break any code. It takes a lot of discipline, but if you can manage it, I guarantee pretty enormous improvements in both productivity and code quality (and even programmer happiness). And before you ask, yes I have had teams which did CI well, but no, we didn't do CD. It's not for everyone. But as I said, it's pretty powerful if you manage it.
- kbenson 11y ago
  But that's the point of a branch, a place to put your code before it's integrated into the whole as an "atomic operation" so: You aren't possibly breaking live code You making lots of changes to code other people might be changing, causing conflicts You don't need to either go back and remove the old branches after the deployment or leave them in to clutter the code. You don't have to worry as much about overlooking some portion of the codebase using the old code (if you change the API enough to make it incompatible), since you can literally remove old APIs and cause compile errors or easily detectable runtime errors. You can change data structures with conversion scripts where it's not feasible to overlap functionality (whether those be code constructs, or disk/DB format).
  - 11y ago
Twisell 11y ago
Actually I would not necessarily oppose toggling and branching. These two method remains tools that can be used separately or together to achieve some goal. I don't see why one shall not use branching to implement a toggling API.
alkonaut 11y ago
> Architecture people tend to speak in trivial examples. Even just having a single deployment (such as a web app) is a huge simplification of "deployment" compared to other software such as a driver, firmware, desktop application or just about any other piece of software.
jon-wood 11y ago
We recently feature toggled exactly your use case - switching to a new API from our payment provider which needs more information. You seem to be working on the assumption that everything has to be toggled or nothing. What we did was to collect the additional information for everyone (its a low risk change after all), and then slowly roll out the new payment API, starting with staff, then friends & family, and finally doing a phased rollout to customers. Feature toggling not only allowed us to roll it out, but also to rapidly roll it back to deal with issues, and resulted in a very smooth implementation of a feature that had potential to directly affect revenue.
- misterkwon 11y ago
  Has anyone tried Launchdarkly for this?
  - jon-wood 11y ago
    While it looks pretty slick, I wouldn't personally pay a monthly fee for feature flagging. Our (pretty basic) implementation is just over 50 lines of code, and allows us to do all the targeting we need - either individual users, or particular groups of users we've tagged as having the feature. It just doesn't seem worth shelling out extra for that.
LoSboccacc 11y ago
> I never understood why some people are scared of branching. we are using a similar pattern to add advanced features to certain plan tiers, can't do that with branching on saas deployments - the project isn't complex at all. we also use it to do progressive release, so that a feature get tested on, say, 10% of the userbase and we improve on their feedback before releasing it at large, but that's secondary to our use case
einrealist 11y ago
Branching adds too much "meta complexity". That is complexity that you cannot track in actual deployed code. You need processes to manage the branches and to determine when and how to merge. And merging branches is always an error prone process itself. And if you have one deployment target, like a web application, you probably need several staging environments for your branches, so branching does not become the bottleneck towards production. Also if you have test code that is maintained elsewhere (e.g. UI tests), that code cannot reflect your feature changes transparently. In comparison to that: feature toggles. You need one branch (developers can still can have private branches). And the transitions are transparent in your code. Actually, all the complexity is forwarded to the code and toggle management. Most people understand the concept of toggles better than branching workflows / processes. Toggles are easier to test and your tests do reflect that features are changing. I made my decision. I will never have official branches again in any of the projects I supervise.
UK-AL 11y ago
Well your merging a lot earlier which means less conflicts. Other benefits include, being able to release to a select group of people, and being turn off features easily if something goes wrong

by xorcist 11y ago

samwgoldman 11y ago

I think the calculus changes when you consider teams with 1000s of programmers committing 100s of changes per day. Keeping feature branches up-to-date can pretty quickly adds up to a lot of time and creates a lot of friction for refactoring/cleanup-type changes. Feature toggles enable some additional cool stuff like partial/incremental rollout and A/B testing, which really pay dividends.

lucaspiller 11y ago
I've never worked on codebases with so many programmers, but I kind of feel that's if that's how you work, 'you're doing it wrong'. Why would you have so many programmers working on a single codebase, why not split it up in distinct parts? If that's not possible because so much stuff is shared, how can you possibly work effectively with so many people changing so many things?
- geocar 11y ago
  1000 programmers on a single codebase means: • "master" always builds • "master" is always stable Any testing has to be done with in-code branches, and has to be built in-place incrementally so that other developers can see you touching stuff (and can be defensive about their tasks).

qwer 11y ago

> I never understood why some people are scared of branching. Well this is branching (in the same way that if..else is branching), just not in the VCS. What's the advantage? This methodology can allow code to go to production for some or all users, either as dark launches ( http://agiletesting.blogspot.com/2009/07/dark-launching-and-other-lessons-from.html http://agiletesting.blogspot.com/2009/07/dark-launching-and-... ) or as production code. VCS branching gives you none of that, and sometimes you need to roll things out sloooowly, either for quality control or for capacity monitoring. > It seems to work pretty well for Linux development. Well for example, if you're following the linux development model for webops, you're almost certainly doing it wrong. I'm sure your website doesn't have release candidates and LTS versions all at different URLs. > As long as you merge continuously from upstream, you're golden. This is not really true either. It's fine if you're the only one on a team following that model, but if everyone on your team follows that model, you're delaying collaboration and causing merge debt. Upstream ends up like a ghost town until someone like you dumps a week worth of work on it just before you're about to dump your week's worth too. > I _do_ have some reservations against #ifdef-sprinkled code. This is definitely a downside! You have to use a lot of discipline to keep this minimal and well-organized.

mikekchar 11y ago

I'm not quite sure I understand all of your comment, but I can respond to why one might want feature toggles rather than branches. One of the activities that can really improve the quality of code on a team is continuous integration. Now, I don't mean setting up a "CI" server to run tests for you when you push your code. I mean actually merging your code with your fellow programmers every 20 minutes or so. The reason this is such a powerful technique is that it gets people to see what you are doing pretty much as soon as you do it. If you do something stupid (as we all do), they complain instantly because they will merge in your code every 20 minutes. If you do CI well, there are no lingering surprises or hard decisions of "Well, maybe you should start all over again". The problem is that all the code is mixed up. This is completely fine if you have 1 or 2 week sprints and deploy at the end of the sprint. The problem is that this kills another really power technique called "continuous deployment". Ideally, we would like to deploy as soon as a story is finished. But if it's mixed up with a whole bunch of other code, then you need to make sure that the other code isn't going to break something. Enter feature toggles. It's a challenge to support both CI and CD at the same time. You can never break any code. It takes a lot of discipline, but if you can manage it, I guarantee pretty enormous improvements in both productivity and code quality (and even programmer happiness). And before you ask, yes I have had teams which did CI well, but no, we didn't do CD. It's not for everyone. But as I said, it's pretty powerful if you manage it.

kbenson 11y ago
But that's the point of a branch, a place to put your code before it's integrated into the whole as an "atomic operation" so: You aren't possibly breaking live code You making lots of changes to code other people might be changing, causing conflicts You don't need to either go back and remove the old branches after the deployment or leave them in to clutter the code. You don't have to worry as much about overlooking some portion of the codebase using the old code (if you change the API enough to make it incompatible), since you can literally remove old APIs and cause compile errors or easily detectable runtime errors. You can change data structures with conversion scripts where it's not feasible to overlap functionality (whether those be code constructs, or disk/DB format).
- 11y ago

Twisell 11y ago

Actually I would not necessarily oppose toggling and branching. These two method remains tools that can be used separately or together to achieve some goal. I don't see why one shall not use branching to implement a toggling API.

alkonaut 11y ago

> Architecture people tend to speak in trivial examples. Even just having a single deployment (such as a web app) is a huge simplification of "deployment" compared to other software such as a driver, firmware, desktop application or just about any other piece of software.

jon-wood 11y ago

We recently feature toggled exactly your use case - switching to a new API from our payment provider which needs more information. You seem to be working on the assumption that everything has to be toggled or nothing. What we did was to collect the additional information for everyone (its a low risk change after all), and then slowly roll out the new payment API, starting with staff, then friends & family, and finally doing a phased rollout to customers. Feature toggling not only allowed us to roll it out, but also to rapidly roll it back to deal with issues, and resulted in a very smooth implementation of a feature that had potential to directly affect revenue.

misterkwon 11y ago
Has anyone tried Launchdarkly for this?
- jon-wood 11y ago
  While it looks pretty slick, I wouldn't personally pay a monthly fee for feature flagging. Our (pretty basic) implementation is just over 50 lines of code, and allows us to do all the targeting we need - either individual users, or particular groups of users we've tagged as having the feature. It just doesn't seem worth shelling out extra for that.

LoSboccacc 11y ago

> I never understood why some people are scared of branching. we are using a similar pattern to add advanced features to certain plan tiers, can't do that with branching on saas deployments - the project isn't complex at all. we also use it to do progressive release, so that a feature get tested on, say, 10% of the userbase and we improve on their feedback before releasing it at large, but that's secondary to our use case

einrealist 11y ago

Branching adds too much "meta complexity". That is complexity that you cannot track in actual deployed code. You need processes to manage the branches and to determine when and how to merge. And merging branches is always an error prone process itself. And if you have one deployment target, like a web application, you probably need several staging environments for your branches, so branching does not become the bottleneck towards production. Also if you have test code that is maintained elsewhere (e.g. UI tests), that code cannot reflect your feature changes transparently. In comparison to that: feature toggles. You need one branch (developers can still can have private branches). And the transitions are transparent in your code. Actually, all the complexity is forwarded to the code and toggle management. Most people understand the concept of toggles better than branching workflows / processes. Toggles are easier to test and your tests do reflect that features are changing. I made my decision. I will never have official branches again in any of the projects I supervise.

UK-AL 11y ago

Well your merging a lot earlier which means less conflicts. Other benefits include, being able to release to a select group of people, and being turn off features easily if something goes wrong