3 ms·
Most tech people don't grok the abstraction between an executable program and the environment it is executed in. The thing that runs your code is responsible f
by 0xbadcafebee 1y ago
Most tech people don't grok the abstraction between an executable program and the environment it is executed in.
The thing that runs your code is responsible for adding the environment variables from wherever they are kept for that environment. The execution environments running your code are not checking out your Git repository and reading files from it before they execute your program; that would introduce a chicken-and-egg problem (and make it much harder to run your program in new environments).
Below is an example of why you can't just load all your variables in a single ".env" file.
Environments:
- *development*:
host: "my laptop"
variables: [FOO=dev]
- *staging*:
host: "a ci/cd server"
variables: [FOO=stage]
- *production*:
host: "a kubernetes server"
variables: [FOO=prod]
I want to run my program in development!
->> open a terminal on my MacOS laptop
->> change to directory with my code
->> load environment variables into shell
$ source .env
->> run my program
$ ./MyProgram.EXE
->> the shell takes the variables it has, and adds them to the program at runtime
char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL};
char *const envp[] = {"FOO=dev", NULL};
execve("/Users/me/app_src/MyProgram.EXE", argv, envp);
->> the program is loaded, executed, and accesses environment variables passed to it
I want to run my program in staging!
->> start a job on a ci/cd server
->> the ci/cd server starts a Docker container
->> the ci/cd server begins running commands in the Docker container
->> checkout latest code and change to new directory
->> load environment variables into shell
->> retrieves environment variables set in ci/cd job configuration
->> sets those variables in the shell
->> run my program
$ ./MyProgram.EXE
->> the shell takes the variables it has, and adds them to the program at runtime
char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL};
char *const envp[] = {"FOO=stage", NULL};
execve("/Users/me/app_src/MyProgram.EXE", argv, envp);
->> the program is loaded, executed, and accesses environment variables passed to it
I want to run my program in production!
->> start a job on a kubernetes server
->> a deployment scheduler schedules a pod to start\
->> it looks up a configmap with environment variables
->> it looks up a secrets object with secrets
->> it sets environment variables in the pod based on the configmaps and secrets
->> it starts a container in the pod
->> it begins running commands in the container
->> run my program
$ ./MyProgram.EXE
->> the container takes the variables it has, and adds them to the program at runtime
char *const argv[] = {"/Users/me/app_src/MyProgram.EXE", NULL};
char *const envp[] = {"FOO=prod", NULL};
execve("/Users/me/app_src/MyProgram.EXE", argv, envp);
->> the program is loaded, executed, and accesses environment variables passed to it
You have to add the environment variables to the execution environment before your program ever gets run. Otherwise (for example) you could never pull a Git repository to load variables, because where would the credentials to pull your Git repository come from? They have to be added beforehand. So that beforehand step is where you add all your environment variables for that environment. Your program merely reads them when it is executed; no need for your program to read additional files.
You should not do something like keep a ".env.dev", ".env.stage", ".env.prod", packaged up in a container. Each execution environment may need slightly different settings at run time. And secrets should not be kept in your Git repo, they should be loaded as-needed, at runtime, by your execution environment.
All this is covered neatly in The Twelve Factor App (https://12factor.net/config https://12factor.net/config). As someone who's been doing this for two decades, I highly recommend everyone follow their guide to the letter.
- theozero 1y agoEven if you don't set any values in your env files (although why not set some non-sensitive defaults) you can still benefit from the schema and validation that varlock provides. Imagine every public docker container had an .env.schema which was validated on startup, instead of scattered info about the available env vars in their readme.
- 0xbadcafebee 1y agoEnvironment variables are a non-standard, limited form of transferring data. Depending on the platform, they may be case-insensitive, there are reserved words, size limits, naming rules, restricted charsets, etc. And their values are always strings. So having a schema on an environment variable isn't very useful, unless you do things like dynamic typing and type assumptions (which run into bugs). It's not a bad idea to have an opinionated library that defines some restrictions on environment variables, but the applications will always need to add more restrictions that your library has thought of. Schemas are application-specific. Applications all deal with data types differently. Some data types (defined in schemas) even use transforms and complex custom algorithms before their data is validated. So it's better to let the application handle schemas directly on an as-needed basis. All of that can be done independent of environment variables. Just make a library that validates data types, and pass your environment variables (or any data, from anywhere) to it. This is better not only because you can validate any kind of data, but you can load your data from places other than environment variables (from disk, from database, etc). This kind of general abstraction is more useful for general purpose computing, rather than a complex solution tailored for only one use case. Finally: A schema isn't a replacement for documentation. Just because you have a technical document that defines what data is allowed in a variable, doesn't mean that somebody then knows what the hell that thing does - what it affects, when it should or shouldn't be used, etc. Documentation is for humans, schemas are for computers.