6 ms·

The whole .env file holding all your configs thing is a disaster. People forget to update the template, it has to be passed around, etc. Configure your secrets

The whole .env file holding all your configs thing is a disaster. People forget to update the template, it has to be passed around, etc. Configure your secrets in Azure Key Vault (AWS Secrets Manager, GCP-something-I-forget) or whatever the non-cloud Vault software is called. Devs only set client_id, client_secret, and env defaulted to dev. On startup, the app fetches configs for the secret store. The needed secrets are in source control. If the needed secret is missing from the secret store, there is a clear error on startup. You are welcome, I just saved you weeks of onboarding, confusion, and devs passing around .envs on Slack.

campbel 3y ago
I agree, sops / ejson / gitcrypt or something else that encrypts a file in vcs, then use your cloud or shared secret vault to disperse the decryption key.
danwee 3y ago
> You are welcome, I just saved you weeks of onboarding, confusion, and devs passing around .envs on Slack. Don't get it. At one company I worked, .env files for the DEV environment were open (the file itself was stored on Vault, so only engineers had access to it), and one could only access DEV resources via VPN. So, starting a service using the DEV .env file was rather easy an uncomplicated.
- throw1234651234 3y ago
  Here is the problem with that. A dev pushes a change that requires a change to the env file. Another dev pulls down the code, but doesn't change the env file. Everything breaks. At really bad companies, DevOps sets env vars from somewhere other than that file passed around by devs - another failure point. My proposal: 1. Key Vault | HashiCorp Vault | whatever is the source of truth for secrets. 2. When a developer adds a key, they add it to Key Vault and update code to use it. 3. On app startup, code checks the dev's local for 2 env vars ONLY, which allow access to key vault. 4. The app then tries to pull in all the env vars it needs by secret name. 5. If any are missing, it fails. This keeps devs from having to pass around .env files outside of source control. Btw, Azure (and all other cloud) builds can be configured to pull env vars directly from Key Vault, so there's that benefit too.
  - parasti 3y ago
    "5. If any are missing, it fails." That's literally all you need to solve the scenario you described with .env files.
    - meowtimemania 3y ago
      The difference is any time the .env config is updated, all engineers have to update their .env file. So with .env file: - developer adds to .env - pushes code - all developers see failures in dev env due to missing env var - all developers search slack/email to figure out what went wrong - all developers update their .env file and things are working again ————- In gp’s comment ———- - one person works on feature that introduces new config - failure happens in their local environment - they update config backend and push code - all other developers get updates with no changes needed
      - throw1234651234 3y ago
        Exactly, thank you for phrasing it much better than I did!
      - Rapzid 3y ago
        You can also just supply defaults that can be locally overriden.. Benefit is they are reviewable in the same PR that adds the code depending on them..
        throw1234651234 3y ago
        You can't keep secrets in code most of the time.
        Rapzid 3y ago
        Usually not big secrets. I do and have done both and a blend. Secrets for external services needed in dev usually in parameter store. Kinda depends on the existing work flow. There is so much to do you should be tackling problems that move the needle and chip away at small stuff, pet peeve stuff in smaller time-boxed increments. It depends, always it depends.
nickzelei 3y ago
I've resulted into a pretty similar setup. We use SOPS and the encrypted .env file is stored in VCS. The key is stored in 1password with instructions on where to stuff it on your machine. Then you just decrypt the .env file and you're off to the races.
- edwinbalani 3y ago
  For anyone new to SOPS like I was - https://github.com/getsops/sops https://github.com/getsops/sops
victorNicollet 3y ago
Hard agree. I think it's easy to get stuck on the mental model that "the configuration value is the secret" (which means the configuration itself needs special handling), and the correct mental model should be "the configuration value is the identifier of the secret", with another system (Azure Key Vault, etc) being responsible for converting the secret-identifier into the secret-value. My company open sourced the .NET library we use for this: https://github.com/Lokad/Secrets https://github.com/Lokad/Secrets
geraldwhen 3y ago
The corporate VPN and proxy setups tend to fail or be slow. This would add non-trivial startup time to any app, and catastrophic startup time to integration tests.

campbel 3y ago

I agree, sops / ejson / gitcrypt or something else that encrypts a file in vcs, then use your cloud or shared secret vault to disperse the decryption key.

danwee 3y ago

> You are welcome, I just saved you weeks of onboarding, confusion, and devs passing around .envs on Slack. Don't get it. At one company I worked, .env files for the DEV environment were open (the file itself was stored on Vault, so only engineers had access to it), and one could only access DEV resources via VPN. So, starting a service using the DEV .env file was rather easy an uncomplicated.

throw1234651234 3y ago
Here is the problem with that. A dev pushes a change that requires a change to the env file. Another dev pulls down the code, but doesn't change the env file. Everything breaks. At really bad companies, DevOps sets env vars from somewhere other than that file passed around by devs - another failure point. My proposal: 1. Key Vault | HashiCorp Vault | whatever is the source of truth for secrets. 2. When a developer adds a key, they add it to Key Vault and update code to use it. 3. On app startup, code checks the dev's local for 2 env vars ONLY, which allow access to key vault. 4. The app then tries to pull in all the env vars it needs by secret name. 5. If any are missing, it fails. This keeps devs from having to pass around .env files outside of source control. Btw, Azure (and all other cloud) builds can be configured to pull env vars directly from Key Vault, so there's that benefit too.
- parasti 3y ago
  "5. If any are missing, it fails." That's literally all you need to solve the scenario you described with .env files.
  - meowtimemania 3y ago
    The difference is any time the .env config is updated, all engineers have to update their .env file. So with .env file: - developer adds to .env - pushes code - all developers see failures in dev env due to missing env var - all developers search slack/email to figure out what went wrong - all developers update their .env file and things are working again ————- In gp’s comment ———- - one person works on feature that introduces new config - failure happens in their local environment - they update config backend and push code - all other developers get updates with no changes needed
    - throw1234651234 3y ago
      Exactly, thank you for phrasing it much better than I did!
    - Rapzid 3y ago
      You can also just supply defaults that can be locally overriden.. Benefit is they are reviewable in the same PR that adds the code depending on them..
      - throw1234651234 3y ago
        You can't keep secrets in code most of the time.
        Rapzid 3y ago
        Usually not big secrets. I do and have done both and a blend. Secrets for external services needed in dev usually in parameter store. Kinda depends on the existing work flow. There is so much to do you should be tackling problems that move the needle and chip away at small stuff, pet peeve stuff in smaller time-boxed increments. It depends, always it depends.

nickzelei 3y ago

I've resulted into a pretty similar setup. We use SOPS and the encrypted .env file is stored in VCS. The key is stored in 1password with instructions on where to stuff it on your machine. Then you just decrypt the .env file and you're off to the races.

edwinbalani 3y ago
For anyone new to SOPS like I was - https://github.com/getsops/sops https://github.com/getsops/sops

victorNicollet 3y ago

Hard agree. I think it's easy to get stuck on the mental model that "the configuration value is the secret" (which means the configuration itself needs special handling), and the correct mental model should be "the configuration value is the identifier of the secret", with another system (Azure Key Vault, etc) being responsible for converting the secret-identifier into the secret-value. My company open sourced the .NET library we use for this: https://github.com/Lokad/Secrets https://github.com/Lokad/Secrets

geraldwhen 3y ago

The corporate VPN and proxy setups tend to fail or be slow. This would add non-trivial startup time to any app, and catastrophic startup time to integration tests.