5 ms·

I don't get `.env` files. Just never made sense that we'd delegate managing environment variables to the processes which consumes them. https://direnv.net http

by jelder 1y ago

I don't get `.env` files. Just never made sense that we'd delegate managing environment variables to the processes which consumes them. https://direnv.net https://direnv.net is a better solution IMO. Once you set it up in your shell, it loads environment variables from the `.envrc` file in whatever the current directory is automatically. It includes a rich standard library (https://direnv.net/man/direnv-stdlib.1.html https://direnv.net/man/direnv-stdlib.1.html) for manipulation of PATH, etc. For secrets, I just add this line to `.envrc`: source_env_if_exists .envrc.private And add `.envrc.private` to `.gitignore`. Now that just works anywhere, whether the authors of whatever tool I'm using official support `.env` files or not.

awestroke 1y ago
This is about formalizing required secrets for dev. Many repos have a ".env.sample", which is better than nothing - you copy it to .env and fill in the blanks. If this is done automatically, even better. If the automatic .env setup also loads secrets from your password manager (with prompting), even better.
- judofyr 1y ago
  With direnv you can also run commands directly in the .envrc: export OPENAPI_API_KEY=$(op read "op://api-prod/openai/api-key") Every time you cd into the directory it will execute the command. Isn’t this even better than copying the secret into a local file?
  - conception 1y ago
    1password actually has environments now so you can just load it directly.
candiddevmike 1y ago
It's such a shame this isn't built into shells. There isn't really a security issue here as you have to trust the directory first before it fires.
deleted 1y ago
[deleted]
__jonas 1y ago
I agree that loading environment variables should not be the responsibility of the process which consumes them, not sure what the benefit of renaming .env to .envrc is. I use mise to load environment variables from .env, since I also use it to manage tool versions. When I don't have that available I just do set -a; source .env; set +a direnv is definitely another good option.
- jelder 1y ago
  direnv also has `dotenv` and `dotenv_if_exists` macros, which nicely gets around the fact that `.env` files typically omit the `export` statement.
- mananaysiempre 1y ago
  Unlike .env, .envrc is not a series of key=value assignments, it’s a full Bash script such that any variables that are changed at the end of its execution are entered into your environment. (The idea is to approximate it being sourced into your shell, except it always uses Bash while you may be using a different shell.) For example, I can write a Nix flake, put “use flake” (and nothing else) in my .envrc in the same directory, and have whatever PATH, PYTHONPATH, etc. changes that are needed to develop against the flake’s dependencies automatically applied when I enter the directory. You could almost certainly use this with virtualenv, nvm, or the like as well, I just haven’t tried.
throw-the-towel 1y ago
And I don't get storing your configuration in env variables, period. They're not discoverable at all, there's no namespacing nor any other isolation, there's no structure to them, and if you want a list you'll have to come up with an ad-hoc encoding. Environment variables are a mess, you'd be better off with a config file.
- w0m 1y ago
  I assume environment variables became popular as it's an 'easy' way to inject secrets without hardcoding them in a config file.
- theozero 1y ago
  Whether you like them or not, using env vars is ubiquitous because it's easy. Anything else ends up tightly coupled within your application code or only within the environment. Often you have a set of config needed both for your application and your surrounding dev tools / scripts. This tries a provide a universal solution. At the end of the day you could imagine varlock loading your config, and injecting it using a method that is not env vars - a file, sidecar process, etc.
0xbadcafebee 1y ago
Most tech people don't grok the abstraction between an executable program and the environment it is executed in. The thing that runs your code is responsible for adding the environment variables from wherever they are kept for that environment. The execution environments running your code are not checking out your Git repository and reading files from it before they execute your program; that would introduce a chicken-and-egg problem (and make it much harder to run your program in new environments). Below is an example of why you can't just load all your variables in a single ".env" file. Environments: - *development*: host: "my laptop" variables: [FOO=dev] - *staging*: host: "a ci/cd server" variables: [FOO=stage] - *production*: host: "a kubernetes server" variables: [FOO=prod] I want to run my program in development! ->> open a terminal on my MacOS laptop ->> change to directory with my code ->> load environment variables into shell $ source .env ->> run my program $ ./MyProgram.EXE ->> the shell takes the variables it has, and adds them to the program at runtime char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL}; char *const envp[] = {"FOO=dev", NULL}; execve("/Users/me/app_src/MyProgram.EXE", argv, envp); ->> the program is loaded, executed, and accesses environment variables passed to it I want to run my program in staging! ->> start a job on a ci/cd server ->> the ci/cd server starts a Docker container ->> the ci/cd server begins running commands in the Docker container ->> checkout latest code and change to new directory ->> load environment variables into shell ->> retrieves environment variables set in ci/cd job configuration ->> sets those variables in the shell ->> run my program $ ./MyProgram.EXE ->> the shell takes the variables it has, and adds them to the program at runtime char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL}; char *const envp[] = {"FOO=stage", NULL}; execve("/Users/me/app_src/MyProgram.EXE", argv, envp); ->> the program is loaded, executed, and accesses environment variables passed to it I want to run my program in production! ->> start a job on a kubernetes server ->> a deployment scheduler schedules a pod to start\ ->> it looks up a configmap with environment variables ->> it looks up a secrets object with secrets ->> it sets environment variables in the pod based on the configmaps and secrets ->> it starts a container in the pod ->> it begins running commands in the container ->> run my program $ ./MyProgram.EXE ->> the container takes the variables it has, and adds them to the program at runtime char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL}; char *const envp[] = {"FOO=prod", NULL}; execve("/Users/me/app_src/MyProgram.EXE", argv, envp); ->> the program is loaded, executed, and accesses environment variables passed to it You have to add the environment variables to the execution environment before your program ever gets run. Otherwise (for example) you could never pull a Git repository to load variables, because where would the credentials to pull your Git repository come from? They have to be added beforehand. So that beforehand step is where you add all your environment variables for that environment. Your program merely reads them when it is executed; no need for your program to read additional files. You should not do something like keep a ".env.dev", ".env.stage", ".env.prod", packaged up in a container. Each execution environment may need slightly different settings at run time. And secrets should not be kept in your Git repo, they should be loaded as-needed, at runtime, by your execution environment. All this is covered neatly in The Twelve Factor App (https://12factor.net/config https://12factor.net/config). As someone who's been doing this for two decades, I highly recommend everyone follow their guide to the letter.

by jelder 1y ago

awestroke 1y ago

This is about formalizing required secrets for dev. Many repos have a ".env.sample", which is better than nothing - you copy it to .env and fill in the blanks. If this is done automatically, even better. If the automatic .env setup also loads secrets from your password manager (with prompting), even better.

judofyr 1y ago
With direnv you can also run commands directly in the .envrc: export OPENAPI_API_KEY=$(op read "op://api-prod/openai/api-key") Every time you cd into the directory it will execute the command. Isn’t this even better than copying the secret into a local file?
- conception 1y ago
  1password actually has environments now so you can just load it directly.

candiddevmike 1y ago

It's such a shame this isn't built into shells. There isn't really a security issue here as you have to trust the directory first before it fires.

deleted 1y ago

[deleted]

__jonas 1y ago

I agree that loading environment variables should not be the responsibility of the process which consumes them, not sure what the benefit of renaming .env to .envrc is. I use mise to load environment variables from .env, since I also use it to manage tool versions. When I don't have that available I just do set -a; source .env; set +a direnv is definitely another good option.

jelder 1y ago
direnv also has `dotenv` and `dotenv_if_exists` macros, which nicely gets around the fact that `.env` files typically omit the `export` statement.
mananaysiempre 1y ago
Unlike .env, .envrc is not a series of key=value assignments, it’s a full Bash script such that any variables that are changed at the end of its execution are entered into your environment. (The idea is to approximate it being sourced into your shell, except it always uses Bash while you may be using a different shell.) For example, I can write a Nix flake, put “use flake” (and nothing else) in my .envrc in the same directory, and have whatever PATH, PYTHONPATH, etc. changes that are needed to develop against the flake’s dependencies automatically applied when I enter the directory. You could almost certainly use this with virtualenv, nvm, or the like as well, I just haven’t tried.

throw-the-towel 1y ago

And I don't get storing your configuration in env variables, period. They're not discoverable at all, there's no namespacing nor any other isolation, there's no structure to them, and if you want a list you'll have to come up with an ad-hoc encoding. Environment variables are a mess, you'd be better off with a config file.

w0m 1y ago
I assume environment variables became popular as it's an 'easy' way to inject secrets without hardcoding them in a config file.
theozero 1y ago
Whether you like them or not, using env vars is ubiquitous because it's easy. Anything else ends up tightly coupled within your application code or only within the environment. Often you have a set of config needed both for your application and your surrounding dev tools / scripts. This tries a provide a universal solution. At the end of the day you could imagine varlock loading your config, and injecting it using a method that is not env vars - a file, sidecar process, etc.

0xbadcafebee 1y ago

Most tech people don't grok the abstraction between an executable program and the environment it is executed in. The thing that runs your code is responsible for adding the environment variables from wherever they are kept for that environment. The execution environments running your code are not checking out your Git repository and reading files from it before they execute your program; that would introduce a chicken-and-egg problem (and make it much harder to run your program in new environments). Below is an example of why you can't just load all your variables in a single ".env" file. Environments: - *development*: host: "my laptop" variables: [FOO=dev] - *staging*: host: "a ci/cd server" variables: [FOO=stage] - *production*: host: "a kubernetes server" variables: [FOO=prod] I want to run my program in development! ->> open a terminal on my MacOS laptop ->> change to directory with my code ->> load environment variables into shell $ source .env ->> run my program $ ./MyProgram.EXE ->> the shell takes the variables it has, and adds them to the program at runtime char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL}; char *const envp[] = {"FOO=dev", NULL}; execve("/Users/me/app_src/MyProgram.EXE", argv, envp); ->> the program is loaded, executed, and accesses environment variables passed to it I want to run my program in staging! ->> start a job on a ci/cd server ->> the ci/cd server starts a Docker container ->> the ci/cd server begins running commands in the Docker container ->> checkout latest code and change to new directory ->> load environment variables into shell ->> retrieves environment variables set in ci/cd job configuration ->> sets those variables in the shell ->> run my program $ ./MyProgram.EXE ->> the shell takes the variables it has, and adds them to the program at runtime char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL}; char *const envp[] = {"FOO=stage", NULL}; execve("/Users/me/app_src/MyProgram.EXE", argv, envp); ->> the program is loaded, executed, and accesses environment variables passed to it I want to run my program in production! ->> start a job on a kubernetes server ->> a deployment scheduler schedules a pod to start\ ->> it looks up a configmap with environment variables ->> it looks up a secrets object with secrets ->> it sets environment variables in the pod based on the configmaps and secrets ->> it starts a container in the pod ->> it begins running commands in the container ->> run my program $ ./MyProgram.EXE ->> the container takes the variables it has, and adds them to the program at runtime char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL}; char *const envp[] = {"FOO=prod", NULL}; execve("/Users/me/app_src/MyProgram.EXE", argv, envp); ->> the program is loaded, executed, and accesses environment variables passed to it You have to add the environment variables to the execution environment before your program ever gets run. Otherwise (for example) you could never pull a Git repository to load variables, because where would the credentials to pull your Git repository come from? They have to be added beforehand. So that beforehand step is where you add all your environment variables for that environment. Your program merely reads them when it is executed; no need for your program to read additional files. You should not do something like keep a ".env.dev", ".env.stage", ".env.prod", packaged up in a container. Each execution environment may need slightly different settings at run time. And secrets should not be kept in your Git repo, they should be loaded as-needed, at runtime, by your execution environment. All this is covered neatly in The Twelve Factor App (https://12factor.net/config https://12factor.net/config). As someone who's been doing this for two decades, I highly recommend everyone follow their guide to the letter.