6 ms·

Maybe this works in the type of work this guy does, but I've been horribly burned by feature flags. How do you test for the combinatorial explosion of feature i

by roel_v 4y ago

Maybe this works in the type of work this guy does, but I've been horribly burned by feature flags. How do you test for the combinatorial explosion of feature interactions? Even if you separate into 'silos' that pinky promise do not affect anything beyond some 'defined' boundary, interactions are still too numerous to test as different products very quickly. And now you have person A using 'feature flag combination' A' and person B using feature flag combination B', and they ask you a question about how this or that is supposed to work, and you won't even know until you are able to exhaustively reason through what the effect of every feature flag this person has en/disabled is - if you even know that at all. Yes in the end it's all in the process, but feature flags are not the end-all be-all these few paragraphs posit them to be.

capableweb 4y ago
They are supposed to be removed quickly. The biggest issue I've seen with implementations of feature flags in products are that they are not removed quickly enough. They should be removed as quickly as possible, to avoid what you're saying. In some places, I've even advocated and implemented strategies for not considering something actually done until the feature flag has been removed. So it can be deployed and activated in production, but in terms of project management, the overall task of the feature shouldn't be considered done until cleanup has been done. This process has been the most successful. Another strategy that kind of worked but not as well, is requiring the cleanup of the feature flag to happen in the next sprint. But it tends to be de-prioritized, and sometimes even pushed to the next sprint as other things are always more important. Another strategy that I've discussed and thought about but never actually implemented or advocated for, is limiting the max amount of active feature flags in the specific project to N numbers (ideally 1). So if you want to activate another feature flag, you have to first cleanup the previous one.
- capn_duck 4y ago
  If they are limited to 1, then they are not appropriate to use as a deployment mechanism. Do you only have one change being worked at a time? On my team, there's typically at least have a dozen different stories being worked in parallel.
  - capableweb 4y ago
    I corrected my comment, it's supposed to be "active" feature flags, so only one enabled one. Thinking is that once you've enabled one in production, it should be stable enough that you should be able to clean it up. Would still enable multiple flags existing so different features can be in progress (and locally you'd only have the one you're working on active).
- roel_v 4y ago
  Yah 'supposed', but that's (to me) in the same bucket as 'different features behind flags aren't supposed to interact' and 'we're supposed to have detailed documentation of who is/was using what set of features at time x'. That's also what I meant with 'it's all in the process' - if there is good management of all of this, it may work. And I'm sure that for some people, it does actually work. Meanwhile for me, I'm staring at build scripts with 15+ year old feature flags that at this point are realistically never going to be removed, and people still sometimes suggest 'yeah but this one little thing we can put in a custom build until it works for everyone else too...' So now not having feature flags is a hill I'll die on.
  - capableweb 4y ago
    Yeah, but there are lots of things that can be footguns if you use them incorrectly, that's simply a part of the job as a software engineer. Otherwise you could argue that `rm -rf` should be removed as it could potentially delete files if you're not careful, but I haven't seen anyone argue for that instead of arguing for "Do be careful when using rm -rf". Same goes for feature flags, and a ton of other things.
  - Hermitian909 4y ago
    Feature flag systems should have the following features: - Ability in code to set the default value in non-production environments such that if a developer never set the flag, it assumes this default value. When a feature is fully released the default should be "on for everyone", generally with the expectation it will be retired soon. - A required field designating an owner usually a team, but can be a person if your company is small enough - A field in the flag definition for an expected retirement date The defaults keep combinatorial explosion down and the other practices make it easy for one responsible person to audit flags and track down responsible parties.

by roel_v 4y ago

capableweb 4y ago

They are supposed to be removed quickly. The biggest issue I've seen with implementations of feature flags in products are that they are not removed quickly enough. They should be removed as quickly as possible, to avoid what you're saying. In some places, I've even advocated and implemented strategies for not considering something actually done until the feature flag has been removed. So it can be deployed and activated in production, but in terms of project management, the overall task of the feature shouldn't be considered done until cleanup has been done. This process has been the most successful. Another strategy that kind of worked but not as well, is requiring the cleanup of the feature flag to happen in the next sprint. But it tends to be de-prioritized, and sometimes even pushed to the next sprint as other things are always more important. Another strategy that I've discussed and thought about but never actually implemented or advocated for, is limiting the max amount of active feature flags in the specific project to N numbers (ideally 1). So if you want to activate another feature flag, you have to first cleanup the previous one.

capn_duck 4y ago
If they are limited to 1, then they are not appropriate to use as a deployment mechanism. Do you only have one change being worked at a time? On my team, there's typically at least have a dozen different stories being worked in parallel.
- capableweb 4y ago
  I corrected my comment, it's supposed to be "active" feature flags, so only one enabled one. Thinking is that once you've enabled one in production, it should be stable enough that you should be able to clean it up. Would still enable multiple flags existing so different features can be in progress (and locally you'd only have the one you're working on active).
roel_v 4y ago
Yah 'supposed', but that's (to me) in the same bucket as 'different features behind flags aren't supposed to interact' and 'we're supposed to have detailed documentation of who is/was using what set of features at time x'. That's also what I meant with 'it's all in the process' - if there is good management of all of this, it may work. And I'm sure that for some people, it does actually work. Meanwhile for me, I'm staring at build scripts with 15+ year old feature flags that at this point are realistically never going to be removed, and people still sometimes suggest 'yeah but this one little thing we can put in a custom build until it works for everyone else too...' So now not having feature flags is a hill I'll die on.
- capableweb 4y ago
  Yeah, but there are lots of things that can be footguns if you use them incorrectly, that's simply a part of the job as a software engineer. Otherwise you could argue that `rm -rf` should be removed as it could potentially delete files if you're not careful, but I haven't seen anyone argue for that instead of arguing for "Do be careful when using rm -rf". Same goes for feature flags, and a ton of other things.
- Hermitian909 4y ago
  Feature flag systems should have the following features: - Ability in code to set the default value in non-production environments such that if a developer never set the flag, it assumes this default value. When a feature is fully released the default should be "on for everyone", generally with the expectation it will be retired soon. - A required field designating an owner usually a team, but can be a person if your company is small enough - A field in the flag definition for an expected retirement date The defaults keep combinatorial explosion down and the other practices make it easy for one responsible person to audit flags and track down responsible parties.