3 ms·

In my experience these (configuration based feature flags and feature flag services) are actually two complimentary capabilities that solve two completely diffe

In my experience these (configuration based feature flags and feature flag services) are actually two complimentary capabilities that solve two completely different problems but that happen to share the same name: feature flags. The config approach is critical for using feature flags as a software development lifecycle tool. It is how you manage having a codebase which contains the unfinished code for new feature x, but can still be deployed and pass all tests without feature x being turned on. In this model you need a mechanism which allows a developer who is working on feature x to enable it for local testing, and for your CI system to be able to interact with the flag system to test that the application works in both states - with x turned on and off. This is ideal for trunk based development models; feature branches are an alternative approach that doesn’t really benefit from this (indeed it adds complexity to working in feature branches). Meanwhile feature flag services are to solve the problem that different people using the same software need different features turned on. That can be as simple as internal testers or beta users, it can be holding features to roll out in fixed update windows per tenant, or it can be part of a risk management strategy where features are rolled out through progressive exposure. It can also be tempting to mix up your feature flags system with an A/B testing system - you can use a feature flag service to expose a feature to a test cohort and measure performance changes. There can be reasons for doing that but it’s really important not to tie all these things together: not every development lifecycle change is an ab test hypothesis. Not every ab test hypothesis is a development lifecycle change (often it is really about testing changes in data, and feature config is just one piece of data you might want to change). Similarly some other data changes than code changes need to roll out progressively to mitigate risk. So all these things might be feature-flag shaped, but that doesn’t mean you can substitute different feature flag solutions in and solve the same problems. This post is saying ‘it’s okay not to have an exposure control solution; you can have a config file’ - which is obviously true, if what you have is a config management problem, not an exposure control problem.

jdwyah 1mo ago
Strongly agree that Config & FeatureFlags are two different things. But as I've tried building tools in the space: man is it difficult to really nail down what the difference is. For Quonfig I landed on: - Data model wise they are identical. Flags and configs can both be targeted. They can both use segments. They can both do partial rollouts. They can both have the same range of values (bool/number/duration/json/json-w-schema/etc) - Its fine to use Flags for your Experiments/AB Tests, but that's just the "allocation engine". The rest of experimentation is the exposure tracking and goal tracking. Those ought live in your product analytics stack, because they are really just events. But its helpful to have a single place (flags) for the experiment allocation because then you can re-use segments for things like hold out groups / and just general targeting. - The only real difference is that Flags are intended to be ephemeral and configs are intended to be permanent. A good UI should show you how long a flag has been alive and help you clean it up (or convert it to a config) if it's been true for everyone for too long. - The use cases are different enough that it's worth keeping them in two separate UI.
- scott_w 1mo ago
  One way I’ve thought of it is to ask what happens if the function is blocked, how is it enabled mechanically and why would that be changed? Once you answer these, you start to see the groupings more clearly, I’ve found.
- alyssanicoll 1mo ago
  [flagged]
scott_w 1mo ago
I 100% agree with everything you said and more. I spent a significant amount of time at my job arguing constantly with the entire engineering department that “Feature Flags” are not all the same thing and had to push back on efforts to mix them all up. Examples: Experiment flags for A/B testing Permission checks at a user level Product flags for feature gating by plan Rollout flags to launch a new feature gradually Configuration for an account/system/feature The number of times I had to push against the “just put it behind hasFeatureFlag(user, flag)” is more than I can count at this point! I think it comes from a misunderstanding of the DRY principle, to be honest.
- esafak 1mo ago
  Why would you not use the same service to handle them? Less is more.
  - hasyimibhar 1mo ago
    Having worked at a large company that misuses feature flag service for everything, here are some examples why you shouldn’t: - one team uses feature flag for product gating. Feature flag service goes down. Users temporarily got locked out of the features they paid for. - one team uses feature flag for dynamic pricing by leveraging targeting rules (how hard is it to write a bunch of if else in code?). It’s evaluated against all users, even if they are not active (for analysis reasons). Feature flag service charges by MAU. We have millions of users. Our feature flag service bill is now 6 digits per year. - one team uses feature flag as literal json store instead of a proper db (god knows why). Someone updated the value but the “schema” is wrong. Shit breaks.
    - esafak 1mo ago
      > - one team uses feature flag for product gating. Feature flag service goes down. Users temporarily got locked out of the features they paid for. That could happen if they used a separate service. In this case you need to build defaults; open or closed, as the case demands. > - one team uses feature flag for dynamic pricing by leveraging targeting rules (how hard is it to write a bunch of if else in code?). It’s evaluated against all users, even if they are not active (for analysis reasons). Feature flag service charges by MAU. We have millions of users. Our feature flag service bill is now 6 digits per year. This is a valid reason if you don't own the service, as it seems you don't. If you did, you should have asked your internal customers what their requirements were. > - one team uses feature flag as literal json store instead of a proper db (god knows why). Someone updated the value but the “schema” is wrong. Shit breaks. Separating the services would not help here. They are simply using the wrong tool.

jdwyah 1mo ago

Strongly agree that Config & FeatureFlags are two different things. But as I've tried building tools in the space: man is it difficult to really nail down what the difference is. For Quonfig I landed on: - Data model wise they are identical. Flags and configs can both be targeted. They can both use segments. They can both do partial rollouts. They can both have the same range of values (bool/number/duration/json/json-w-schema/etc) - Its fine to use Flags for your Experiments/AB Tests, but that's just the "allocation engine". The rest of experimentation is the exposure tracking and goal tracking. Those ought live in your product analytics stack, because they are really just events. But its helpful to have a single place (flags) for the experiment allocation because then you can re-use segments for things like hold out groups / and just general targeting. - The only real difference is that Flags are intended to be ephemeral and configs are intended to be permanent. A good UI should show you how long a flag has been alive and help you clean it up (or convert it to a config) if it's been true for everyone for too long. - The use cases are different enough that it's worth keeping them in two separate UI.

scott_w 1mo ago
One way I’ve thought of it is to ask what happens if the function is blocked, how is it enabled mechanically and why would that be changed? Once you answer these, you start to see the groupings more clearly, I’ve found.
alyssanicoll 1mo ago
[flagged]

scott_w 1mo ago

I 100% agree with everything you said and more. I spent a significant amount of time at my job arguing constantly with the entire engineering department that “Feature Flags” are not all the same thing and had to push back on efforts to mix them all up. Examples: Experiment flags for A/B testing Permission checks at a user level Product flags for feature gating by plan Rollout flags to launch a new feature gradually Configuration for an account/system/feature The number of times I had to push against the “just put it behind hasFeatureFlag(user, flag)” is more than I can count at this point! I think it comes from a misunderstanding of the DRY principle, to be honest.

esafak 1mo ago
Why would you not use the same service to handle them? Less is more.
- hasyimibhar 1mo ago
  Having worked at a large company that misuses feature flag service for everything, here are some examples why you shouldn’t: - one team uses feature flag for product gating. Feature flag service goes down. Users temporarily got locked out of the features they paid for. - one team uses feature flag for dynamic pricing by leveraging targeting rules (how hard is it to write a bunch of if else in code?). It’s evaluated against all users, even if they are not active (for analysis reasons). Feature flag service charges by MAU. We have millions of users. Our feature flag service bill is now 6 digits per year. - one team uses feature flag as literal json store instead of a proper db (god knows why). Someone updated the value but the “schema” is wrong. Shit breaks.
  - esafak 1mo ago
    > - one team uses feature flag for product gating. Feature flag service goes down. Users temporarily got locked out of the features they paid for. That could happen if they used a separate service. In this case you need to build defaults; open or closed, as the case demands. > - one team uses feature flag for dynamic pricing by leveraging targeting rules (how hard is it to write a bunch of if else in code?). It’s evaluated against all users, even if they are not active (for analysis reasons). Feature flag service charges by MAU. We have millions of users. Our feature flag service bill is now 6 digits per year. This is a valid reason if you don't own the service, as it seems you don't. If you did, you should have asked your internal customers what their requirements were. > - one team uses feature flag as literal json store instead of a proper db (god knows why). Someone updated the value but the “schema” is wrong. Shit breaks. Separating the services would not help here. They are simply using the wrong tool.