12 ms·

Show HN: Dotenv, if it is a Unix utility

225 points by gyf304 2y ago

I like the idea of using dotenv files, but I dislike having to use different language-specific libraries to read them. To solve this, I created a small utility that lets you prefix any command with "dotenv" to load the ".env" file. This is how I imagine dotenv would work if it had started as a UNIX utility rather than a Node.js library.

kzrdude 2y ago
This idea seems to be cloned everywhere now, so something is causing the popularity
- fyrn_ 2y ago
  Kubernetes / containderd / docker apps are much more convenient to configure through ENV vars, as they easily pass through the sandbox layer (whatever that may be) files are not so easy to make work. Because that's how prod works devs want to be able to recreate prod to run locally, hence the cambrian explosion of tools like this.
  - deleted 2y ago
    [deleted]
  - forrestthewoods 2y ago
    What a tragic state of affairs. It's a shame that running modern software requires carefully packaging a virtual environment and then injecting a bunch of ugly global env vars. I still think Docker shouldn't exist. Programs should simply bundle their dependencies. Running a program should be as simple as download, unzip, run. No complex hierarchical container management needed. Alas I am not King.
    - imtringued 2y ago
      Docker is "programs bundling their dependencies".
      - forrestthewoods 2y ago
        In an extremely heavyweight and needlessly convoluted way.
    - bandie91 2y ago
      > Programs should simply bundle their dependencies. awww. i don't think it's OK in any way to download libc6/msvcrt as many times as I download __any__ software. even more, is there a strong difference between dependency and runtime environment? if sensible people does not bundle the whole python distribution to a "stuff.py" then why bundle libopenssl.so to a webserver application? IMO, a saner approach would be just not to confuse dependencies: appX depends on libY 1.9; appZ depends on libY 2.0; people are quick to declare that appX and appZ are incompatibe as they can not run on the same system due to "conflicting dependencies". but who said you have to seach libY in /usr/lib*/libY.so? if you need different versions of a lib, just install them in separate dirs and make your apps find the right one (eg. by setting RPATH or versioned .so filenames).
      - forrestthewoods 2y ago
        > is there a strong difference between dependency and runtime environment? Programs should rely on the global runtime environment as little as possible > if sensible people does not bundle the whole python distribution to a "stuff.py" Unfortunately Python deployment is such a such an unmitigated disaster that it's a leading cause of Docker images. Deploying a portable copy of Python is about 9 megabytes compressed. This is significantly preferable to multi-gigabyte Docker images. > people are quick to declare that appX and appZ are incompatibe as they can not run on the same system due to "conflicting dependencies". but who said you have to seach libY in /usr/lib*/libY.so? if you need different versions of a lib, just install them in separate dirs and make your apps find the right one (eg. by setting RPATH or versioned .so filenames). You make a strong and compelling argument as to why programs should bundle their dependencies and not rely on the system environment. Users should not have to perform any witchcraft to launch a program. Download and run. No further steps should be necessary.
    - throwaway290232 2y ago
      [dead]
petepete 2y ago
I think direnv already does a good job in this space, and it's already available in your package manager. https://direnv.net/ https://direnv.net/
- kelnos 2y ago
  I've used direnv, but I think a nice property of OP's dotenv is that it's explicit: if I want to pass env vars, I run my program under it. If I don't, then I don't. There's no "hidden behavior" for me to forget about and then get surprised by.
  - TheRoque 2y ago
    As far as I'm aware of, Direnv's behavior is not hidden at all. Whenever you cd into the directory, you get a message listing all the new en var activated. And when you change the .envrc, you get another message saying that direnv has been deactivated. I never had happen to me "oh shoot !! I forgot this env var was activated because I'm in this dir".
    - 3836293648 2y ago
      Oh yeah, that's the default. Everyone I know im ediately disabled that and I even forgot about it till now
    - macintux 2y ago
      I don't think "hidden" and "explicit" are true antonyms. From your description, direnv is implicit and noisy, whereas dotenv seems to be (unless you embed it in a script) explicit and quiet.
      - TheRoque 2y ago
        Well basically, you cd in the shell, and then it shows: "direnv: loading ~/dev/foo/.envrc direnv: export +DATABASE_URL" Is this "implicit" to you ? Because to me it's pretty explicit. But yeah it's automatic, if you don't want this behavior, you don't install direnv. Just to be clear, implicit is "suggested but not communicated directly", and to me, this is communicated directly, so I don't see why it would be implicit...
tester457 2y ago
dotenv started as a ruby library actually. The first implementation inspired the others such as the golang version of the library.
deleted 2y ago
[deleted]
miohtama 2y ago
There is also shdotenv that allows you to load different .env file formats and convert between them, e.g. for UNIX shell. https://github.com/ko1nksm/shdotenv https://github.com/ko1nksm/shdotenv
- gyf304 2y ago
  This is a very neat project that seems to accomplish the same goal and have some extra features.
- sirwitti 2y ago
  I recently discovered shdotenv and I like it a lot!
mongol 2y ago
Would not sh -c '. .env; echo $MY_VAR' do the same thing? (I am not in front of a shell at the moment.)
- mixmastamyk 2y ago
  Seems to work.
- kelnos 2y ago
  That would, but unless each line in your .env file is prefixed with "export", those env vars won't get passed into any subprocesses you run.
  - deleted 2y ago
    [deleted]
- iokanuon 2y ago
  You'd need to `set -a` or pass the `-a` as a flag to have them auto-exported though, so: sh -ac '. ./.env; ./prog' Also if you use the `.` builtin it's a good idea to specify the path with a slash in it, so that `.` doesn't search $PATH first.
- datascienced 2y ago
  Now that is unixy!
- emmanueloga_ 2y ago
  There are like a couple dozen different ways to do this... I have this on my .bashrc: alias loadenv='export $(xargs <.env)' source: [1] -- 1: https://stackoverflow.com/a/60406814/855105 https://stackoverflow.com/a/60406814/855105
  - geysersam 2y ago
    Very nice! Thanks for the suggestion. Seems more Unix-esque. Are there any important drawbacks of this version compared to the dedicated tool? (dotenv or dotenvx)
grounder 2y ago
Compare with dotenvx - https://github.com/dotenvx/dotenvx https://github.com/dotenvx/dotenvx This is my current tool of choice.
spullara 2y ago
[flagged]
- VWWHFSfQ 2y ago
  what are the suggestions
  - spullara 2y ago
    GPT-4: The code provided has a few potential issues, including security vulnerabilities: Buffer Overflow and Memory Allocation Errors: The malloc function in read_file does not check if the memory allocation fails (it checks if buffer is NULL instead of buffer). This can lead to a null pointer dereference if malloc fails and returns NULL. There's a possibility of buffer overflow or improper handling if the file size read by ftell is exactly MAX_FILE_SIZE, because an additional byte is added (buffer = malloc(file_size + 1);). Improper Input Validation: The skip_until_charset and skip_charset functions do not check if the end of the buffer is reached within their loops. If the charset does not appear in the buffer, this could lead to reading beyond the buffer's limit, causing undefined behavior. Use of Uninitialized Variables: In the main function, the value_end pointer is set conditionally inside the if (cur == '"' || cur == '\'') block but used unconditionally outside it. This can lead to the use of an uninitialized pointer if the if block is not executed. Command Line Argument Handling: The check if (argc < 1) is unnecessary because argc will always be at least 1 in any C program, as it includes the name of the program itself. Insufficient validation of command arguments, specifically not handling scenarios where argv[1] might not be provided correctly or safely. Potential Command Injection: The execvp call uses user-supplied input (argv[1] and argv[2...]) without sanitization. If the environment or the .env file can be manipulated, it could lead to command injection vulnerabilities. Error Handling: The program's error messages (e.g., "Missing equal after key") are clear but the handling of such errors (e.g., continuing to parse or halting operations) may need to be revisited based on desired robustness. Claude: Here are a few potential security vulnerabilities and bugs I noticed in the code: The read_file function does not properly handle the case where malloc fails and returns NULL. It should check if buffer is NULL after the malloc call and handle the error appropriately, likely by returning an error code. Currently it would dereference a NULL pointer. The code trusts the contents of the .env file. If an attacker can modify the .env file, they could potentially inject malicious values into environment variables that might be used in an unsafe way by the child process. There are no checks on the size of keys or values. A malicious .env file could contain extremely long keys or values that could potentially cause issues. The code uses strchr to look for meta characters like '=', '#', etc. If the .env file contains non-ASCII characters, this could potentially lead to unexpected behavior. The code uses execvp to execute the provided command with the parsed environment. If the command is provided by an untrusted source, this is a command injection vulnerability. The code should validate the command is on an allowlist before executing. The parsing allows for escape sequences in values, but doesn't have bounds checks. A maliciously crafted escape sequence could read or write out of bounds. The code will modify the process environment even if executing the final command fails. It may be better to parse into a separate data structure and only update the environment if the final execvp call will succeed. It's a good practice to set a proper process umask before writing files or creating directories accessible to other users on the system. While the code looks generally well written, for a utility like this that parses untrusted input and executes commands, it requires very careful security review and hardening. I'd recommend a thorough security audit before using it on untrusted .env files or with untrusted commands. Adding additional input validation, using allowlists, and limiting the scope of what env vars and commands are allowed would help mitigate some risks.
whalesalad 2y ago
export $(cat .env | xargs) Agree with the premise but this can be achieved with actual Unix concepts no need for anything else. The language runtime dotenv projects are banned in my engineering org.
- netcraft 2y ago
  I tend to agree, and we do this a lot actually. But it gets a little more complicated if you have several .env files. Would love to hear more about why dotenv is banned at your org though.
  - whalesalad 2y ago
    Because I banned it haha. There should not be more than one .env file. Our projects have a .env.example that has any overrides a dev might want to override but this list is kept intentionally very short. Meanwhile .env is noted in gitignore. I absolutely hate seeing an entire application configured with environment variables. Some? Sure, where it makes sense. Most? No, those should be in version control, secrets aside. I believe in convention over configuration. Most of our apps have hard-coded config, with a concise/short and finite number of things that can be overridden (like 3-4 parameters, tops). Secrets get injected. I do subscribe to the idea of the 12 factor app, but there is a line that needs to be drawn between env config which is more dynamic and more persistent config that should be baked in to the release.
    - sureglymop 2y ago
      To add to that, SOME_SECRET env vars should be banned (or at least overridable) in favor of SOME_SECRET_FILE env vars. I usually just put an example of the env vars into the readme or link to the file in the source code handling that directly.
    - silversmith 2y ago
      But then the problem is changing configs means building a new release, and needs code push access. Pretty much every config variable has env override in my apps - allows project owner to poke about in web UI without bothering me for changes.
supriyo-biswas 2y ago
I wrote my own some time ago: https://github.com/supriyo-biswas/dotfiles/commit/39585b42c24ae2d18112d142cf6081354eb29c77 https://github.com/supriyo-biswas/dotfiles/commit/39585b42c2...
KevinMS 2y ago
I never understood why it had to be a dot file, except for naming it.
vishvananda 2y ago
Doesn’t this already exist as https://www.npmjs.com/package/dotenv-cli https://www.npmjs.com/package/dotenv-cli ?
bitwize 2y ago
C? Y u no Rust?
- masklinn 2y ago
  The same thing already exists in Rust, it’s both a library for in-process loading and a binary, I use it daily and only for the binary: https://github.com/allan2/dotenvy https://github.com/allan2/dotenvy
  - bitwize 2y ago
    Once I actually wrote a version in Emacs Lisp, for purposes of being able to run stacks that depended on .env configuration in Emacs buffers.
tuyiown 2y ago
Please just see this `env -S "$(cat .env)" <cmd>` Believe it or not that’s all you need. > S, --split-string=S process and split S into separate arguments; used to pass multiple arguments on shebang lines edit: forgot the quotes around shell substitution
- kazinator 2y ago
  This is now syntax that requires processing by the shell. The nice thing about utilities like env and dotenv is that they can be easily exec-ed: execl("/usr/bin/dotenv", "/usr/bin/dotenv", "command", "arg", (char *) 0); -S is a fairly recently added option to the GNU Coreutils env (possibly inspired by BSD?). I have a window to an Ubuntu 18 VM where it's not available. You want $(cat .env) quoted, as in "$(cat .env)" so that the content of the file is reliably passed as one argument. -S will split on whitespace; but it respects quoting, so spaces can be protected. Basically .env has to be prepared with the features of -S in mind. Of which that thing has quite a few: escape sequences like \n, commenting, environment variable substitution.
- kazinator 2y ago
  Also, if we are going to involve the shell, we could also just make .env a shell fragment and do this: sh -c '. .env; <cmd>' There is a way to pass commands to it which are reliably executed, like thisL sh -c '. .env; "$@"' -- command arg1 arg2 arg3. The non-option arguments passed to the shell are available as `"$@"`. A command consisting of nothing but `"$@"` basically executes the arguments. We can use `exec`, speaking of which: sh -c '. .env; exec "$@"' -- command arg1 arg2 arg3. What I'm getting at is that this form is fairly easily exec-able; execl("/bin/sh", "/bin/sh", ". .env; exec \"$@\"", "--", "command", "arg1", "arg2", "arg3", (char *) 0); The command and arguments can be arbitrary strings, not subject to any shell mangling.
  - cholindo 2y ago
    I wouldn't follow this approach because if you run `. .env;` you .env gets evaluated as a bash script, not as a configuration file. This means that you can get runtime errors in the .env file, and nobody wants that.
fire_lake 2y ago
Does this accept the exact same format (including quotes and whitespace) as a Docker env file? That’s a key feature for me
prmoustache 2y ago
I don't understand what more it does than sourcing a file on your shell would? Anyone can explain?
- steezeburger 2y ago
  You can't source an .env file without some munging. All the keys would need `export` in front of them I believe.
  - kazinator 2y ago
    A sourced .env would have to be correct shell syntax for a sourced environment file, yes.
    - maple3142 2y ago
      But a lot of env file doesn't do that iirc. `allexport` solves this though.
      - prmoustache 2y ago
        You can implement that in a simple shell function anyway.
  - prmoustache 2y ago
    That doesn't seem like a huge barrier compared to shipping a dotenv binary compiled specifically to all deployment arch.
    - steezeburger 2y ago
      It's not a huge barrier, but it's still a barrier. I have lots of infra using Helm/K8s, sometimes Docker. These .env files don't have `export` keywords in them. So your suggestion is to munge these for local development. And you're okay with that barrier? That's terrible dx, and it adds surface area for bugs.
wodenokoto 2y ago
Since loading dotenv files happens together with executing code I I have decided to trust my .env files just like I trust the rest of my code not to delete my entire system and therefore I source them.
iDon 2y ago
Thanks to OP and other posters - various ideas useful in different cases. The xargs idea made me think of using bash as the parser : bash -c "exec -c bash -c 'source $CONFIG/main.bash; env'" This test .bash file contains multiple source-s of other .bash files, which contain a mix of comments, functions, set and env vars - just the env vars are exported by env. This seems useful e.g. for collating & summarising an environment for docker run -e. This outputs the env vars to stdout; for the OP's purpose, the output could be sourced : envFile=$(mktemp /tmp/env.XXXXXX); bash -c "exec -c bash -c 'source $CONFIG/main.bash; env'" > $envFile; env $(cat $envFile) sh -c 'echo $API_HOST' # For Bourne shell, use env -i in place of exec -c : sh -c "env -i sh -c '. $CONFIG/main.sh; env'" > $envFile
throwaway984393 2y ago
[dead]
andy_ppp 2y ago
This looks good and neater than my solution in my .zshrc: envup() { local file=$([ -z "$1" ] && echo ".env" || echo ".env.$1") if [ -f $file ]; then set -a source $file set +a else echo "No $file file found" 1>&2 return 1 fi } You can also specify `envup development` to load .env.development files should you want. Obviously this will pollute the current shell but for me it is fine.
belthesar 2y ago
This is interestingly similar to a little tool I wrote called sops-run [0], which manages encrypted secrets for cli tools using Mozilla’s sops [1]. Biggest upshot is that you can use it more confidently for secrets with encryption at rest. Built it when I was trying out CLI tools that wanted API keys, but I didn’t want to shove them into my profile and accidentally upload them into my dotfiles repository. Do need to finally get back to making this a package, being able to install it with pip(x) would be really nice. [0] https://github.com/belthesar/sops-run https://github.com/belthesar/sops-run [1] https://github.com/getsops/sops https://github.com/getsops/sops
kazinator 2y ago
I just remembered. Adding a -f <file> option to the GNU Coreutils env utility has previously been discussed: https://lists.gnu.org/archive/html/coreutils/2021-10/msg00001.html https://lists.gnu.org/archive/html/coreutils/2021-10/msg0000... It came up in the mailing also this March. I saw the posting in my inbox and proposed that a null-terminated format be handled, which is exactly like /proc/<pid>/env: https://lists.gnu.org/archive/html/coreutils/2024-03/msg00140.html https://lists.gnu.org/archive/html/coreutils/2024-03/msg0014... If that feature were available, this becomes env -f .env command arg ...

225 points by gyf304 2y ago

kzrdude 2y ago

This idea seems to be cloned everywhere now, so something is causing the popularity

fyrn_ 2y ago
Kubernetes / containderd / docker apps are much more convenient to configure through ENV vars, as they easily pass through the sandbox layer (whatever that may be) files are not so easy to make work. Because that's how prod works devs want to be able to recreate prod to run locally, hence the cambrian explosion of tools like this.
- deleted 2y ago
  [deleted]
- forrestthewoods 2y ago
  What a tragic state of affairs. It's a shame that running modern software requires carefully packaging a virtual environment and then injecting a bunch of ugly global env vars. I still think Docker shouldn't exist. Programs should simply bundle their dependencies. Running a program should be as simple as download, unzip, run. No complex hierarchical container management needed. Alas I am not King.
  - imtringued 2y ago
    Docker is "programs bundling their dependencies".
    - forrestthewoods 2y ago
      In an extremely heavyweight and needlessly convoluted way.
  - bandie91 2y ago
    > Programs should simply bundle their dependencies. awww. i don't think it's OK in any way to download libc6/msvcrt as many times as I download __any__ software. even more, is there a strong difference between dependency and runtime environment? if sensible people does not bundle the whole python distribution to a "stuff.py" then why bundle libopenssl.so to a webserver application? IMO, a saner approach would be just not to confuse dependencies: appX depends on libY 1.9; appZ depends on libY 2.0; people are quick to declare that appX and appZ are incompatibe as they can not run on the same system due to "conflicting dependencies". but who said you have to seach libY in /usr/lib*/libY.so? if you need different versions of a lib, just install them in separate dirs and make your apps find the right one (eg. by setting RPATH or versioned .so filenames).
    - forrestthewoods 2y ago
      > is there a strong difference between dependency and runtime environment? Programs should rely on the global runtime environment as little as possible > if sensible people does not bundle the whole python distribution to a "stuff.py" Unfortunately Python deployment is such a such an unmitigated disaster that it's a leading cause of Docker images. Deploying a portable copy of Python is about 9 megabytes compressed. This is significantly preferable to multi-gigabyte Docker images. > people are quick to declare that appX and appZ are incompatibe as they can not run on the same system due to "conflicting dependencies". but who said you have to seach libY in /usr/lib*/libY.so? if you need different versions of a lib, just install them in separate dirs and make your apps find the right one (eg. by setting RPATH or versioned .so filenames). You make a strong and compelling argument as to why programs should bundle their dependencies and not rely on the system environment. Users should not have to perform any witchcraft to launch a program. Download and run. No further steps should be necessary.
  - throwaway290232 2y ago
    [dead]

petepete 2y ago

I think direnv already does a good job in this space, and it's already available in your package manager. https://direnv.net/ https://direnv.net/

kelnos 2y ago
I've used direnv, but I think a nice property of OP's dotenv is that it's explicit: if I want to pass env vars, I run my program under it. If I don't, then I don't. There's no "hidden behavior" for me to forget about and then get surprised by.
- TheRoque 2y ago
  As far as I'm aware of, Direnv's behavior is not hidden at all. Whenever you cd into the directory, you get a message listing all the new en var activated. And when you change the .envrc, you get another message saying that direnv has been deactivated. I never had happen to me "oh shoot !! I forgot this env var was activated because I'm in this dir".
  - 3836293648 2y ago
    Oh yeah, that's the default. Everyone I know im ediately disabled that and I even forgot about it till now
  - macintux 2y ago
    I don't think "hidden" and "explicit" are true antonyms. From your description, direnv is implicit and noisy, whereas dotenv seems to be (unless you embed it in a script) explicit and quiet.
    - TheRoque 2y ago
      Well basically, you cd in the shell, and then it shows: "direnv: loading ~/dev/foo/.envrc direnv: export +DATABASE_URL" Is this "implicit" to you ? Because to me it's pretty explicit. But yeah it's automatic, if you don't want this behavior, you don't install direnv. Just to be clear, implicit is "suggested but not communicated directly", and to me, this is communicated directly, so I don't see why it would be implicit...

tester457 2y ago

dotenv started as a ruby library actually. The first implementation inspired the others such as the golang version of the library.

deleted 2y ago

[deleted]

miohtama 2y ago

There is also shdotenv that allows you to load different .env file formats and convert between them, e.g. for UNIX shell. https://github.com/ko1nksm/shdotenv https://github.com/ko1nksm/shdotenv

gyf304 2y ago
This is a very neat project that seems to accomplish the same goal and have some extra features.
sirwitti 2y ago
I recently discovered shdotenv and I like it a lot!

mongol 2y ago

Would not sh -c '. .env; echo $MY_VAR' do the same thing? (I am not in front of a shell at the moment.)

mixmastamyk 2y ago
Seems to work.
kelnos 2y ago
That would, but unless each line in your .env file is prefixed with "export", those env vars won't get passed into any subprocesses you run.
- deleted 2y ago
  [deleted]
iokanuon 2y ago
You'd need to `set -a` or pass the `-a` as a flag to have them auto-exported though, so: sh -ac '. ./.env; ./prog' Also if you use the `.` builtin it's a good idea to specify the path with a slash in it, so that `.` doesn't search $PATH first.
datascienced 2y ago
Now that is unixy!
emmanueloga_ 2y ago
There are like a couple dozen different ways to do this... I have this on my .bashrc: alias loadenv='export $(xargs <.env)' source: [1] -- 1: https://stackoverflow.com/a/60406814/855105 https://stackoverflow.com/a/60406814/855105
- geysersam 2y ago
  Very nice! Thanks for the suggestion. Seems more Unix-esque. Are there any important drawbacks of this version compared to the dedicated tool? (dotenv or dotenvx)

grounder 2y ago

Compare with dotenvx - https://github.com/dotenvx/dotenvx https://github.com/dotenvx/dotenvx This is my current tool of choice.

spullara 2y ago

[flagged]

VWWHFSfQ 2y ago
what are the suggestions
- spullara 2y ago
  GPT-4: The code provided has a few potential issues, including security vulnerabilities: Buffer Overflow and Memory Allocation Errors: The malloc function in read_file does not check if the memory allocation fails (it checks if buffer is NULL instead of buffer). This can lead to a null pointer dereference if malloc fails and returns NULL. There's a possibility of buffer overflow or improper handling if the file size read by ftell is exactly MAX_FILE_SIZE, because an additional byte is added (buffer = malloc(file_size + 1);). Improper Input Validation: The skip_until_charset and skip_charset functions do not check if the end of the buffer is reached within their loops. If the charset does not appear in the buffer, this could lead to reading beyond the buffer's limit, causing undefined behavior. Use of Uninitialized Variables: In the main function, the value_end pointer is set conditionally inside the if (cur == '"' || cur == '\'') block but used unconditionally outside it. This can lead to the use of an uninitialized pointer if the if block is not executed. Command Line Argument Handling: The check if (argc < 1) is unnecessary because argc will always be at least 1 in any C program, as it includes the name of the program itself. Insufficient validation of command arguments, specifically not handling scenarios where argv[1] might not be provided correctly or safely. Potential Command Injection: The execvp call uses user-supplied input (argv[1] and argv[2...]) without sanitization. If the environment or the .env file can be manipulated, it could lead to command injection vulnerabilities. Error Handling: The program's error messages (e.g., "Missing equal after key") are clear but the handling of such errors (e.g., continuing to parse or halting operations) may need to be revisited based on desired robustness. Claude: Here are a few potential security vulnerabilities and bugs I noticed in the code: The read_file function does not properly handle the case where malloc fails and returns NULL. It should check if buffer is NULL after the malloc call and handle the error appropriately, likely by returning an error code. Currently it would dereference a NULL pointer. The code trusts the contents of the .env file. If an attacker can modify the .env file, they could potentially inject malicious values into environment variables that might be used in an unsafe way by the child process. There are no checks on the size of keys or values. A malicious .env file could contain extremely long keys or values that could potentially cause issues. The code uses strchr to look for meta characters like '=', '#', etc. If the .env file contains non-ASCII characters, this could potentially lead to unexpected behavior. The code uses execvp to execute the provided command with the parsed environment. If the command is provided by an untrusted source, this is a command injection vulnerability. The code should validate the command is on an allowlist before executing. The parsing allows for escape sequences in values, but doesn't have bounds checks. A maliciously crafted escape sequence could read or write out of bounds. The code will modify the process environment even if executing the final command fails. It may be better to parse into a separate data structure and only update the environment if the final execvp call will succeed. It's a good practice to set a proper process umask before writing files or creating directories accessible to other users on the system. While the code looks generally well written, for a utility like this that parses untrusted input and executes commands, it requires very careful security review and hardening. I'd recommend a thorough security audit before using it on untrusted .env files or with untrusted commands. Adding additional input validation, using allowlists, and limiting the scope of what env vars and commands are allowed would help mitigate some risks.

whalesalad 2y ago

export $(cat .env | xargs) Agree with the premise but this can be achieved with actual Unix concepts no need for anything else. The language runtime dotenv projects are banned in my engineering org.

netcraft 2y ago
I tend to agree, and we do this a lot actually. But it gets a little more complicated if you have several .env files. Would love to hear more about why dotenv is banned at your org though.
- whalesalad 2y ago
  Because I banned it haha. There should not be more than one .env file. Our projects have a .env.example that has any overrides a dev might want to override but this list is kept intentionally very short. Meanwhile .env is noted in gitignore. I absolutely hate seeing an entire application configured with environment variables. Some? Sure, where it makes sense. Most? No, those should be in version control, secrets aside. I believe in convention over configuration. Most of our apps have hard-coded config, with a concise/short and finite number of things that can be overridden (like 3-4 parameters, tops). Secrets get injected. I do subscribe to the idea of the 12 factor app, but there is a line that needs to be drawn between env config which is more dynamic and more persistent config that should be baked in to the release.
  - sureglymop 2y ago
    To add to that, SOME_SECRET env vars should be banned (or at least overridable) in favor of SOME_SECRET_FILE env vars. I usually just put an example of the env vars into the readme or link to the file in the source code handling that directly.
  - silversmith 2y ago
    But then the problem is changing configs means building a new release, and needs code push access. Pretty much every config variable has env override in my apps - allows project owner to poke about in web UI without bothering me for changes.

supriyo-biswas 2y ago

I wrote my own some time ago: https://github.com/supriyo-biswas/dotfiles/commit/39585b42c24ae2d18112d142cf6081354eb29c77 https://github.com/supriyo-biswas/dotfiles/commit/39585b42c2...

KevinMS 2y ago

I never understood why it had to be a dot file, except for naming it.

vishvananda 2y ago

Doesn’t this already exist as https://www.npmjs.com/package/dotenv-cli https://www.npmjs.com/package/dotenv-cli ?

bitwize 2y ago

C? Y u no Rust?

masklinn 2y ago
The same thing already exists in Rust, it’s both a library for in-process loading and a binary, I use it daily and only for the binary: https://github.com/allan2/dotenvy https://github.com/allan2/dotenvy
- bitwize 2y ago
  Once I actually wrote a version in Emacs Lisp, for purposes of being able to run stacks that depended on .env configuration in Emacs buffers.

tuyiown 2y ago

Please just see this `env -S "$(cat .env)" <cmd>` Believe it or not that’s all you need. > S, --split-string=S process and split S into separate arguments; used to pass multiple arguments on shebang lines edit: forgot the quotes around shell substitution

kazinator 2y ago
This is now syntax that requires processing by the shell. The nice thing about utilities like env and dotenv is that they can be easily exec-ed: execl("/usr/bin/dotenv", "/usr/bin/dotenv", "command", "arg", (char *) 0); -S is a fairly recently added option to the GNU Coreutils env (possibly inspired by BSD?). I have a window to an Ubuntu 18 VM where it's not available. You want $(cat .env) quoted, as in "$(cat .env)" so that the content of the file is reliably passed as one argument. -S will split on whitespace; but it respects quoting, so spaces can be protected. Basically .env has to be prepared with the features of -S in mind. Of which that thing has quite a few: escape sequences like \n, commenting, environment variable substitution.
kazinator 2y ago
Also, if we are going to involve the shell, we could also just make .env a shell fragment and do this: sh -c '. .env; <cmd>' There is a way to pass commands to it which are reliably executed, like thisL sh -c '. .env; "$@"' -- command arg1 arg2 arg3. The non-option arguments passed to the shell are available as `"$@"`. A command consisting of nothing but `"$@"` basically executes the arguments. We can use `exec`, speaking of which: sh -c '. .env; exec "$@"' -- command arg1 arg2 arg3. What I'm getting at is that this form is fairly easily exec-able; execl("/bin/sh", "/bin/sh", ". .env; exec \"$@\"", "--", "command", "arg1", "arg2", "arg3", (char *) 0); The command and arguments can be arbitrary strings, not subject to any shell mangling.
- cholindo 2y ago
  I wouldn't follow this approach because if you run `. .env;` you .env gets evaluated as a bash script, not as a configuration file. This means that you can get runtime errors in the .env file, and nobody wants that.

fire_lake 2y ago

Does this accept the exact same format (including quotes and whitespace) as a Docker env file? That’s a key feature for me

prmoustache 2y ago

I don't understand what more it does than sourcing a file on your shell would? Anyone can explain?

steezeburger 2y ago
You can't source an .env file without some munging. All the keys would need `export` in front of them I believe.
- kazinator 2y ago
  A sourced .env would have to be correct shell syntax for a sourced environment file, yes.
  - maple3142 2y ago
    But a lot of env file doesn't do that iirc. `allexport` solves this though.
    - prmoustache 2y ago
      You can implement that in a simple shell function anyway.
- prmoustache 2y ago
  That doesn't seem like a huge barrier compared to shipping a dotenv binary compiled specifically to all deployment arch.
  - steezeburger 2y ago
    It's not a huge barrier, but it's still a barrier. I have lots of infra using Helm/K8s, sometimes Docker. These .env files don't have `export` keywords in them. So your suggestion is to munge these for local development. And you're okay with that barrier? That's terrible dx, and it adds surface area for bugs.

wodenokoto 2y ago

Since loading dotenv files happens together with executing code I I have decided to trust my .env files just like I trust the rest of my code not to delete my entire system and therefore I source them.

iDon 2y ago

Thanks to OP and other posters - various ideas useful in different cases. The xargs idea made me think of using bash as the parser : bash -c "exec -c bash -c 'source $CONFIG/main.bash; env'" This test .bash file contains multiple source-s of other .bash files, which contain a mix of comments, functions, set and env vars - just the env vars are exported by env. This seems useful e.g. for collating & summarising an environment for docker run -e. This outputs the env vars to stdout; for the OP's purpose, the output could be sourced : envFile=$(mktemp /tmp/env.XXXXXX); bash -c "exec -c bash -c 'source $CONFIG/main.bash; env'" > $envFile; env $(cat $envFile) sh -c 'echo $API_HOST' # For Bourne shell, use env -i in place of exec -c : sh -c "env -i sh -c '. $CONFIG/main.sh; env'" > $envFile

throwaway984393 2y ago

[dead]

andy_ppp 2y ago

This looks good and neater than my solution in my .zshrc: envup() { local file=$([ -z "$1" ] && echo ".env" || echo ".env.$1") if [ -f $file ]; then set -a source $file set +a else echo "No $file file found" 1>&2 return 1 fi } You can also specify `envup development` to load .env.development files should you want. Obviously this will pollute the current shell but for me it is fine.

belthesar 2y ago

This is interestingly similar to a little tool I wrote called sops-run [0], which manages encrypted secrets for cli tools using Mozilla’s sops [1]. Biggest upshot is that you can use it more confidently for secrets with encryption at rest. Built it when I was trying out CLI tools that wanted API keys, but I didn’t want to shove them into my profile and accidentally upload them into my dotfiles repository. Do need to finally get back to making this a package, being able to install it with pip(x) would be really nice. [0] https://github.com/belthesar/sops-run https://github.com/belthesar/sops-run [1] https://github.com/getsops/sops https://github.com/getsops/sops

kazinator 2y ago

I just remembered. Adding a -f <file> option to the GNU Coreutils env utility has previously been discussed: https://lists.gnu.org/archive/html/coreutils/2021-10/msg00001.html https://lists.gnu.org/archive/html/coreutils/2021-10/msg0000... It came up in the mailing also this March. I saw the posting in my inbox and proposed that a null-terminated format be handled, which is exactly like /proc/<pid>/env: https://lists.gnu.org/archive/html/coreutils/2024-03/msg00140.html https://lists.gnu.org/archive/html/coreutils/2024-03/msg0014... If that feature were available, this becomes env -f .env command arg ...