11 ms·

feature flags are supposed to be simple, this is not. Am I thinking of the wrong there here? feature flags are often used to release a feature and then removed

feature flags are supposed to be simple, this is not. Am I thinking of the wrong there here? feature flags are often used to release a feature and then removed once that feature is in production. Why do we even need an API?

brandall10 4y ago
At a basic level, yes. When you think about things like LaunchDarkly they tend to also be used to conduct A/B tests where logic is used to determine when they should be toggled on/off.
DJBunnies 4y ago
What, you don't want an external dependency for toggles?
moltar 4y ago
Yes and no. Simple on the surface. But complex under the hood. A good example of the managed complexity is AWS AppConfig. It helps with feature value validations to prevent bad input, it helps with gradual roll out and roll backs if alarm is triggered. This product is based on years of internal use and experimentation.
deleted 4y ago
[deleted]
vouwfietsman 4y ago
I was equally confused, and found: https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/feature-flags https://docs.microsoft.com/en-us/dotnet/architecture/cloud-n... It says: Feature flags are built upon conditional logic that control visibility of functionality for users at run time. In modern cloud-native systems, it's common to deploy new features into production early, but test them with a limited audience. As confidence increases, the feature can be incrementally rolled out to wider audiences. Looks like this supports an advanced form of, let's be generous, integration testing. Other uses mentioned by the article: Restrict premium functionality to specific customer groups willing to pay higher subscription fees. Stabilize a system by quickly deactivating a problem feature, avoiding the risks of a rollback or immediate hotfix. Disable an optional feature with high resource consumption during peak usage periods. Conduct experimental feature releases to small user segments to validate feasibility and popularity. All of this sounds to me like an urge to release before something is technologically mature. (note, not functionally mature, that's just A/B testing). Sometimes I wonder that if you need these things, are you really sure about anything you're doing at all? I don't mean the individual developer, I mean the software industry at large. Imagine a civil engineer saying: yeah we're gonna build a new bridge here, and we're gonna let in only like 10 cars a minute, just to see if it holds. Then later, as we get more confidence, we can open it up more. or We're gonna add a way to temporarily block this off ramp from the highway to the city. If it gets too busy we can just block the off ramp, and everything will be fine. I feel that most of the reasons mentioned above, except for A/B testing, are kind of bogus. Maybe somebody with more experience can share a counter view.
- abridgett 4y ago
  You comparison with building bridges without adequate testing is unfair. It's about increasing control, not giving permission to lower quality. In terms of rollout it's more like releasing new drugs - initially it's very small groups that will be tightly studied, over time these groups are increased in size (and monitoring is probably reduced in scope). The use of feature flags to control rollout and "blast damage" from unexpected behaviour or bugs is really useful. Let me give a real example I was involved with - mobile app development. You can never test everything (not every Android phone variant, nor every 3rd party app which may interfere). Being able to roll out a new feature gradually and then revert (or in our case even blacklist certain models) is key to increasing quality and reducing problems. It also accelerates development and (in my mind a _key_ benefit) reduces stress/anxiety for the developers.
  - vouwfietsman 4y ago
    Yeah that makes sense as a use case, I agree. I would still argue that it doesn't speak in favor of software development in general that we need this though, but I see that in your situation it would be useful. But to further my point: why not solve this with traditional versioning? The user can just pick the version of the app that is compatible with his device, and report bugs about the newer version. I suppose the culprit there is the app store approval process that blocks you from doing fast releases? And you just have a single native app wrapping a web-app that loads from the server? Or perhaps the user cannot roll back the versions easily? If so I think that is a manufactured problem. Not a problem that the developer can solve or be blamed for, but ultimately a problem that should not exist/should have been solved, but was invented/grown/ignored by circumstance. I mean that there is nothing in the problem space of the end-user that requires you to solve this problem. By extension, the solution sounds like a band-aid to me, someone not familiar with the space. The user is no longer in control over the software he runs, and applications may break and recover at any time. To me that does not sound like an increase in quality. Thats not to say that I don't see why you have problems, or that I think they can be completely solved by better quality control. What I mean to say is that the issues you describe sound like issues software developers have faced for years, and tackled with application versions.
nerdponx 4y ago
This is a classic "sounds easy but is actually a bit tricky" situation. What if you have 1m LoC and 5k feature flags covering features over a 6-12 month development cycle? What if you have a combination of "serverless" cloud stuff and long-running server apps? What about feature toggles on the client side, or in your GraphQL layer? What if you have applications multiple programming languages? Etc etc. As with a lot of things in programming, returns to scale actually go negative after a certain size.
- vouwfietsman 4y ago
  I think the confusion here is that traditionally a feature flag is part of the source code, or potentially deployment configuration. Not something that should be modified during runtime of production code, but only during deployments/releases. It seems that the modern interpretation is much more fluid, where a feature flag can be toggled on and off during runtime over multiple services. It seems logical that you would then want a "command and control" API that flips these things on a larger scale. However, to me this still sounds very strange. If you have 5k feature flags, do you really have any idea of what is running at all, at any time, anyway? What if you get a bug ticket from last week? Is there such a thing as a "deployment" if the code you execute is dependent on 5k bits that can be flipped and alter execution at any time? If you really need this much runtime configuration shouldn't it be part of some admin workflow that is just inherent to your application? Why would you even risk synchronizing another piece of state over a distributed system that alters the code? Wouldn't you want this stuff to just be part of your data model? Seems its no longer about feature flags, but about having a global table of bits you can abuse from any and all elements of your architecture, as a shortcut to actual good data design.
  - nerdponx 4y ago
    As an addendum, I think it's very unlikely that any one team is concerned with a large number of feature flags at any given time. But if you have multiple teams working on different parts of a big application, complexity starts to add up.

brandall10 4y ago

At a basic level, yes. When you think about things like LaunchDarkly they tend to also be used to conduct A/B tests where logic is used to determine when they should be toggled on/off.

DJBunnies 4y ago

What, you don't want an external dependency for toggles?

moltar 4y ago

Yes and no. Simple on the surface. But complex under the hood. A good example of the managed complexity is AWS AppConfig. It helps with feature value validations to prevent bad input, it helps with gradual roll out and roll backs if alarm is triggered. This product is based on years of internal use and experimentation.

deleted 4y ago

[deleted]

vouwfietsman 4y ago

I was equally confused, and found: https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/feature-flags https://docs.microsoft.com/en-us/dotnet/architecture/cloud-n... It says: Feature flags are built upon conditional logic that control visibility of functionality for users at run time. In modern cloud-native systems, it's common to deploy new features into production early, but test them with a limited audience. As confidence increases, the feature can be incrementally rolled out to wider audiences. Looks like this supports an advanced form of, let's be generous, integration testing. Other uses mentioned by the article: Restrict premium functionality to specific customer groups willing to pay higher subscription fees. Stabilize a system by quickly deactivating a problem feature, avoiding the risks of a rollback or immediate hotfix. Disable an optional feature with high resource consumption during peak usage periods. Conduct experimental feature releases to small user segments to validate feasibility and popularity. All of this sounds to me like an urge to release before something is technologically mature. (note, not functionally mature, that's just A/B testing). Sometimes I wonder that if you need these things, are you really sure about anything you're doing at all? I don't mean the individual developer, I mean the software industry at large. Imagine a civil engineer saying: yeah we're gonna build a new bridge here, and we're gonna let in only like 10 cars a minute, just to see if it holds. Then later, as we get more confidence, we can open it up more. or We're gonna add a way to temporarily block this off ramp from the highway to the city. If it gets too busy we can just block the off ramp, and everything will be fine. I feel that most of the reasons mentioned above, except for A/B testing, are kind of bogus. Maybe somebody with more experience can share a counter view.

abridgett 4y ago
You comparison with building bridges without adequate testing is unfair. It's about increasing control, not giving permission to lower quality. In terms of rollout it's more like releasing new drugs - initially it's very small groups that will be tightly studied, over time these groups are increased in size (and monitoring is probably reduced in scope). The use of feature flags to control rollout and "blast damage" from unexpected behaviour or bugs is really useful. Let me give a real example I was involved with - mobile app development. You can never test everything (not every Android phone variant, nor every 3rd party app which may interfere). Being able to roll out a new feature gradually and then revert (or in our case even blacklist certain models) is key to increasing quality and reducing problems. It also accelerates development and (in my mind a _key_ benefit) reduces stress/anxiety for the developers.
- vouwfietsman 4y ago
  Yeah that makes sense as a use case, I agree. I would still argue that it doesn't speak in favor of software development in general that we need this though, but I see that in your situation it would be useful. But to further my point: why not solve this with traditional versioning? The user can just pick the version of the app that is compatible with his device, and report bugs about the newer version. I suppose the culprit there is the app store approval process that blocks you from doing fast releases? And you just have a single native app wrapping a web-app that loads from the server? Or perhaps the user cannot roll back the versions easily? If so I think that is a manufactured problem. Not a problem that the developer can solve or be blamed for, but ultimately a problem that should not exist/should have been solved, but was invented/grown/ignored by circumstance. I mean that there is nothing in the problem space of the end-user that requires you to solve this problem. By extension, the solution sounds like a band-aid to me, someone not familiar with the space. The user is no longer in control over the software he runs, and applications may break and recover at any time. To me that does not sound like an increase in quality. Thats not to say that I don't see why you have problems, or that I think they can be completely solved by better quality control. What I mean to say is that the issues you describe sound like issues software developers have faced for years, and tackled with application versions.

nerdponx 4y ago

This is a classic "sounds easy but is actually a bit tricky" situation. What if you have 1m LoC and 5k feature flags covering features over a 6-12 month development cycle? What if you have a combination of "serverless" cloud stuff and long-running server apps? What about feature toggles on the client side, or in your GraphQL layer? What if you have applications multiple programming languages? Etc etc. As with a lot of things in programming, returns to scale actually go negative after a certain size.

vouwfietsman 4y ago
I think the confusion here is that traditionally a feature flag is part of the source code, or potentially deployment configuration. Not something that should be modified during runtime of production code, but only during deployments/releases. It seems that the modern interpretation is much more fluid, where a feature flag can be toggled on and off during runtime over multiple services. It seems logical that you would then want a "command and control" API that flips these things on a larger scale. However, to me this still sounds very strange. If you have 5k feature flags, do you really have any idea of what is running at all, at any time, anyway? What if you get a bug ticket from last week? Is there such a thing as a "deployment" if the code you execute is dependent on 5k bits that can be flipped and alter execution at any time? If you really need this much runtime configuration shouldn't it be part of some admin workflow that is just inherent to your application? Why would you even risk synchronizing another piece of state over a distributed system that alters the code? Wouldn't you want this stuff to just be part of your data model? Seems its no longer about feature flags, but about having a global table of bits you can abuse from any and all elements of your architecture, as a shortcut to actual good data design.
- nerdponx 4y ago
  As an addendum, I think it's very unlikely that any one team is concerned with a large number of feature flags at any given time. But if you have multiple teams working on different parts of a big application, complexity starts to add up.