4 ms·

> A better approach is to store configuration in environment variables. It's about damn time, Google! I've been working on a number of GCP projects this year a

> A better approach is to store configuration in environment variables. It's about damn time, Google! I've been working on a number of GCP projects this year and have consistently bumped up against the awkwardness that was Google's (former?) preference for configuration files over environment variables.

emmelaich 7y ago
This is the weakest or least well-defined aspect of 12factor. How can you 'store' anything in something as ephemeral as environment variables? Where do these environment variables come from? When are they set? You should of course store everything in version control; the point of this guideline is that the scope should not be the same as the app or an instance. There is particular tension between this guideline and X. environments should be a similar as possible and I. there should be a single codebase for all deployments.
- emmelaich 7y ago
  Also ... it's a bit circuitous. > requires strict separation of config from code. Config varies substantially across deploys, code does not. What's the difference between config and code? Config is that which changes between deployments / environments.
  - pushrax 7y ago
    Config also benefits from review and version control, and it's inconvenient to use multiple systems for these. It's been working out well for us to store secrets encrypted with public key crypto alongside source, or in a deployment repository using the same source control. The private key is inserted into the infrastructure as a K8s secret.
    - thomascgalvin 7y ago
      One of the concepts twelve factor pushes is that you should be able to release your source code right now without compromising any credentials. Basically, you'd have a git repo for code, and a separate git repo for environment scripts. I've seen organizations use a whole separate infrastructure for config, but that always seemed like overkill to me.
      - pushrax 7y ago
        This is true with encrypted credentials, if security is compromised by release of the repo then the encryption is not secure. I think the spirit of the principle is met, and in cases where you do later release the source code without having planned for it, it's easy enough to move the secrets. It's definitely more convenient to use the same repo, especially in the declarative containerized world; you can make sweeping changes or roll them back in a single atomic commit. That said, if open source is the intent, certainly use multiple repos from the beginning so you avoid git gardening when releasing later.
- unlinked_dll 7y ago
  >How can you 'store' anything in something as ephemeral as environment variables? Where do these environment variables come from? When are they set? Set at build time. They come from a build config file/script as part of the build. They're not perfect, but just about every tool ever made has a way of understanding environment variables - the same can't be said for any given config file. Here's a real world example: I want one source of truth for my build configuration, but my build itself pulls together libraries and binaries in a number of different languages and build systems. How would you handle that with anything but environment variables and a master shell script?
  - emmelaich 7y ago
    > They come from a build config file/script as part of the build. But the script is in VC surely? > How would you handle that with anything but environment variables and a master shell script? Again, where is that master shell script? What args do you give it? My preference is that you have one environment variable, and your config is keyed off that. If that environment variable is not supplied, fail noisily.
- tybit 7y ago
  Another option is rather than storing configuration in config files, is to store it in code. After experimenting with different approaches, storing configuration in code and switching on a single environment variable ENV (or STAGE or whatever you like to call it) this is probably my favourite. This doesn't work for everyones infrastructure setups, but it's worked well for mine.
  - icebraining 7y ago
    A file that stores configuration is a configuration file, even if the encoding format happens to be the same as the main programming language and not toml or yaml :) That works fine if there's a limited number of environments, but it becomes a pain if you're running the same app in too many, or if someone else needs to deploy it independently of the developers.
    - solipsism 7y ago
      it becomes a pain if you're running the same app in too many, No it doesn't. It's trivial to generate the full set of configs using a script, if you have too many to manage manually. or if someone else needs to deploy it independently of the developers And how exactly do envvars help with that? The set of envvars still needs to get pushed with each deployment. envvars are just a special config file.
  - mixedCase 7y ago
    Convenient enough for dynamic languages. Not so much compiled ones.
- alchemism 7y ago
  Kubernetes assists with this aspect of passing environment variables to immutable containers. There you add the variables to the deployment code or edit them live on active deployments. You can take it one step further and define a single ConfigMap resource per environment, containing all of the required variables across your application, and then have all the various containers reference it ‘blindly’ when they are deployed. Said configuration resource would exist in version control but deploy independently from the functional codebase(s).
- snicker7 7y ago
  Configuration isn't stored in environment variables. Rather, they are exposed as environment variables. The idea is that the application environment / infrastructure should own the config data, not the application code itself. 12 factor practices require that devs draw a boundary between app and infrastructure.
lostcolony 7y ago
I actually had this question earlier today. Given a config file per 'environment', with environment being an arbitrary identifier, what benefit do environment variables have over it? I've read some docs on 12 factor apps looking for an answer but didn't really find anything. Just curious what I'm overlooking
- jaysh 7y ago
  As long as your (production) config files are not checked in with the code, then little difference. The challenge then is: where do you store them? Environment variables had the early upside of being first class in any virtually any deployment strategy - from Bash wrappers, supervisord to Docker and Kubernetes - so it's pretty much plug and play. For example, Docker has config files, but they're mounted to a specific place in your container, so that means your application is now married to that location.
- manigandham 7y ago
  Env vars are universally supported, globally available, and simple to manage. Files mean yet another deployment artifact that has to be stored somewhere, and now tied to a location. Sometimes that location is communicated through an env var, and that point why not just set all config that way?
  - AlphaSite 7y ago
    If you have something like k8s it becomes trivial to use files vs end vars.
    - manigandham 7y ago
      How so? You still have to edit the file, save it somewhere, then mount it as a volume and agree on a location.
      - yencabulator 7y ago
        The configuration file can be inline in the yaml, ConfigMaps take care of shuffling bytes around for you, and for a specific app it's config can always be at a fixed path.

emmelaich 7y ago

This is the weakest or least well-defined aspect of 12factor. How can you 'store' anything in something as ephemeral as environment variables? Where do these environment variables come from? When are they set? You should of course store everything in version control; the point of this guideline is that the scope should not be the same as the app or an instance. There is particular tension between this guideline and X. environments should be a similar as possible and I. there should be a single codebase for all deployments.

emmelaich 7y ago
Also ... it's a bit circuitous. > requires strict separation of config from code. Config varies substantially across deploys, code does not. What's the difference between config and code? Config is that which changes between deployments / environments.
- pushrax 7y ago
  Config also benefits from review and version control, and it's inconvenient to use multiple systems for these. It's been working out well for us to store secrets encrypted with public key crypto alongside source, or in a deployment repository using the same source control. The private key is inserted into the infrastructure as a K8s secret.
  - thomascgalvin 7y ago
    One of the concepts twelve factor pushes is that you should be able to release your source code right now without compromising any credentials. Basically, you'd have a git repo for code, and a separate git repo for environment scripts. I've seen organizations use a whole separate infrastructure for config, but that always seemed like overkill to me.
    - pushrax 7y ago
      This is true with encrypted credentials, if security is compromised by release of the repo then the encryption is not secure. I think the spirit of the principle is met, and in cases where you do later release the source code without having planned for it, it's easy enough to move the secrets. It's definitely more convenient to use the same repo, especially in the declarative containerized world; you can make sweeping changes or roll them back in a single atomic commit. That said, if open source is the intent, certainly use multiple repos from the beginning so you avoid git gardening when releasing later.
unlinked_dll 7y ago
>How can you 'store' anything in something as ephemeral as environment variables? Where do these environment variables come from? When are they set? Set at build time. They come from a build config file/script as part of the build. They're not perfect, but just about every tool ever made has a way of understanding environment variables - the same can't be said for any given config file. Here's a real world example: I want one source of truth for my build configuration, but my build itself pulls together libraries and binaries in a number of different languages and build systems. How would you handle that with anything but environment variables and a master shell script?
- emmelaich 7y ago
  > They come from a build config file/script as part of the build. But the script is in VC surely? > How would you handle that with anything but environment variables and a master shell script? Again, where is that master shell script? What args do you give it? My preference is that you have one environment variable, and your config is keyed off that. If that environment variable is not supplied, fail noisily.
tybit 7y ago
Another option is rather than storing configuration in config files, is to store it in code. After experimenting with different approaches, storing configuration in code and switching on a single environment variable ENV (or STAGE or whatever you like to call it) this is probably my favourite. This doesn't work for everyones infrastructure setups, but it's worked well for mine.
- icebraining 7y ago
  A file that stores configuration is a configuration file, even if the encoding format happens to be the same as the main programming language and not toml or yaml :) That works fine if there's a limited number of environments, but it becomes a pain if you're running the same app in too many, or if someone else needs to deploy it independently of the developers.
  - solipsism 7y ago
    it becomes a pain if you're running the same app in too many, No it doesn't. It's trivial to generate the full set of configs using a script, if you have too many to manage manually. or if someone else needs to deploy it independently of the developers And how exactly do envvars help with that? The set of envvars still needs to get pushed with each deployment. envvars are just a special config file.
- mixedCase 7y ago
  Convenient enough for dynamic languages. Not so much compiled ones.
alchemism 7y ago
Kubernetes assists with this aspect of passing environment variables to immutable containers. There you add the variables to the deployment code or edit them live on active deployments. You can take it one step further and define a single ConfigMap resource per environment, containing all of the required variables across your application, and then have all the various containers reference it ‘blindly’ when they are deployed. Said configuration resource would exist in version control but deploy independently from the functional codebase(s).
snicker7 7y ago
Configuration isn't stored in environment variables. Rather, they are exposed as environment variables. The idea is that the application environment / infrastructure should own the config data, not the application code itself. 12 factor practices require that devs draw a boundary between app and infrastructure.

lostcolony 7y ago

I actually had this question earlier today. Given a config file per 'environment', with environment being an arbitrary identifier, what benefit do environment variables have over it? I've read some docs on 12 factor apps looking for an answer but didn't really find anything. Just curious what I'm overlooking

jaysh 7y ago
As long as your (production) config files are not checked in with the code, then little difference. The challenge then is: where do you store them? Environment variables had the early upside of being first class in any virtually any deployment strategy - from Bash wrappers, supervisord to Docker and Kubernetes - so it's pretty much plug and play. For example, Docker has config files, but they're mounted to a specific place in your container, so that means your application is now married to that location.
manigandham 7y ago
Env vars are universally supported, globally available, and simple to manage. Files mean yet another deployment artifact that has to be stored somewhere, and now tied to a location. Sometimes that location is communicated through an env var, and that point why not just set all config that way?
- AlphaSite 7y ago
  If you have something like k8s it becomes trivial to use files vs end vars.
  - manigandham 7y ago
    How so? You still have to edit the file, save it somewhere, then mount it as a volume and agree on a location.
    - yencabulator 7y ago
      The configuration file can be inline in the yaml, ConfigMaps take care of shuffling bytes around for you, and for a specific app it's config can always be at a fixed path.