3 ms·
The whole idea of using environment variables for configuration information is good, but ultimately flawed, and we are way past the point where this should cont
by 0xbadcafebee 2y ago
The whole idea of using environment variables for configuration information is good, but ultimately flawed, and we are way past the point where this should continue to be the status quo.
Environment variables are great for configuration because:
- you can inherit them from a previous application or application(s)
- you can override them in each environment you run your app in
- you can pass them on to other applications
- they are globals that can be loaded by libraries
- they're not hardcoded in the code, so easier to change things without rebuilding, easier to reuse in different ways/environments/configurations
- the OS has primitives for them
- they're simple
Environment variables are bad for configuration:
- because (by default) when set in application, they are passed on to all future applications/forks/execs
- they are often dumped as part of troubleshooting and aren't considered confidential
- they can often be viewed by external processes/users
- there are restrictions on key names and values and size depending on the platform
- typical "dotenv" solution doesn't necessarily handle things like multi-line strings, has no formal specification
- no types, schemas
What we actually need that environment variables are being used for:
- configuration information passed at execution time that can change per environment
- loading or passing secret values
- development environments
- production environments
So what would be a good alternative?
- an application library ("libconfig") that can load configuration of various types from various sources in various ways
- support for configuration types: key-value, file/blob, integer/float
- support for confidentiality (require specific function to unseal secret values; in programming languages the intent would be you can't just print a stringified version of the variable without an unseal function)
- support for schema (application defines schema, throws exception if value does not match)
- support allowing a configuration to be overloaded by different sources/hierarchies
- support passing a configuration on to other applications
- support tracing, verbose logging
- truly cross-platform and cross-language with one specification, behavior for all
How would it work?
- devs can create a .env file if they want
- devs load 'libconfig' into app, use it to load their configuration values during development. library can have default sources, and even set env vars or an object internally, so no code needs to be written to use it
- in production, same code causes libconfig to look at cloud-native and other sources for configuration
- when debugging, secret confidentiality is maintained, tracing communicates sources of configuration, what was loaded, from where, etc
- theozero 2y agoSuper early days but we're building something very similar to what you are describing over at https://dmno.dev https://dmno.dev Granted our solution is more javascript/typescript focused - and the config schema will be defined using TypeScript and piggyback on npm for sharing plugins/extensions. But the config will be usable in any language (and generate types for multiple languages) with deeper integrations coming soon. The pluggable nature of our architecture also means you can encrypt secrets in your repo if you want to, or sync with other backends. Shouldn't be too hard to keep everything away from env vars either if that's what some folks want. Would love your input, and to hear what you think!
- 0xbadcafebee 2y agoThe feature list looks great, but it looks like it's trying to bite off too much at once. - Supporting a particular repo type is an engineering smell (over-complicated/over-opinionated/tightly-integrated); this should be repo-design-agnostic - The "dmno service" is also biting off too much, this should not be a concept inherent to the configuration library - Schemas should be optional - Data types are fine, but get complicated, especially when mapping between different data formats/containers/transports; it's probably better to start with only a couple types and grow them over time if needed - Inter-service dependency management is also too complex for this solution - Plugins are a good idea - Three different package types? Complexity... - The security features are great I get that you're trying to sell a product, and so having a big kitchen sink makes it more attractive to buy, but it makes for more complicated solutions which then annoy users.