9 ms·

Node.js 20.6.0 will include built-in support for .env files

196 points by mariuz 3y ago

blitz_skull 3y ago
Kinda surprising it took this long. I really hope Deno continues to grow in popularity. I greatly prefer the "batteries-included" and Typescript-first approach.
- noman-land 3y ago
  I haven't used Deno but have had an easy time using ts-node. https://typestrong.org/ts-node/docs/ https://typestrong.org/ts-node/docs/
  - hliyan 3y ago
    I've been eyeing ts-node because I'm sick of all the compilation like ceremonies i need to go through for a supposedly JIT compiled language, i.e. JS. How stable is it for production use?
    - andrew_ 3y ago
      with --swc it's damn fast and very stable. ts-node doesn't recommend you use ts-node in production, however I've seen it done without much consequence.
    - noman-land 3y ago
      I use it for a one-person personal project so hard to call that production but it's been really great. Basically seamless typescript in node as far as I've been concerned.
  - adriancooney 3y ago
    I still find `ts-node` gets in my way often with configuration related issues. `tsx` on the other hand has been a dream.
SOLAR_FIELDS 3y ago
.env desperately needs a formal specification. I wrote a bit of code using some features python_dotenv offers but realized after the fact that other software might not parse it the same way. I’ve taken to instead doing my templating in Python itself via pydantic BaseSettings
- prng2021 3y ago
  Can you provide a concrete example? What do you mean by other software not parsing it the same way?
  - moojd 3y ago
    Some execute them as a shell script and others parse them as a newline separated list of key value pairs
  - 3np 3y ago
    FOO="abc BAR=123" Is this one or two vars? What is the value of FOO?
    - prng2021 3y ago
      I've never seen something like this in an env file. No one should be putting lines like this in an env file unless it's by mistake.
      - 3np 3y ago
        Your personal opinion of how esoteric or weird this is is besides the point - there are fundamentally incompatible interpretations where I can bet that switching will be breaking and potentially dangerous for some out there.
- sergiotapia 3y ago
  I've only ever used .env as such: export TWILIO_SID="123123123123" export TWILIO_SECRET="123123123123" And then `source .env` in my terminal, then `rails server` or whatever to run my apps. The environment variables will be present to be consumed from within the app. Curious what features you use beyond this.
netcraft 3y ago
havent dug further in, wonder if it will support multiple files. we use two files, one for things that everyone should have in common and one for things that are different per individual and not committed
- c0brac0bra 3y ago
  It should. .env.local is a common pattern
  - tacker2000 3y ago
    Reminds me of vscode never being able to automatically syntax highight .env.local or similar… or is there some setting im missing?
    - rickstanley 3y ago
      `.env.*` files won't highlight in either VSCode or Neovim, but it works fine with `.*.env`. I guess it's some kind of convention.
      - qntmfred 3y ago
        it's because syntax highlighting rules are applied based on the file extension. as long as .env is the extension, it'll work. if the file ends with .local, vscode has no idea that it's supposed to be a .env file type
        tacker2000 3y ago
        Yes sure, but since .env.local and .env.prod are so widespread one could make a small filter to also include these types of files, no?
        Timon3 3y ago
        Especially considering the GitHub .gitignore template for Node only ignores .env.local, not .local.env: https://github.com/github/gitignore/blob/main/Node.gitignore#L76-L80 https://github.com/github/gitignore/blob/main/Node.gitignore...
jtwaleson 3y ago
Is it just me or does anyone else hate .env files? They accumulate unused cruft, it's not typed, and typically many systems in the same repo point at the same .env file. It all feels so ... implicit ... I guess. For example in a code base I work in, docker-compose uses it, laravel picks up the same file and Vue / webpack does too. It's a big mess. There must be better solutions out there.
- moojd 3y ago
  I like them for defining a small number of env vars for my app that only get injected at run time. I don't like them when other random tools that aren't my app use them. Docker's insistence on automatically using the .env in your home directory is absurd.
- dubcanada 3y ago
  I personally don't feel the same, I have no idea why you'd want a typed environment variable setup. And I find they work great. I don't see any problem with 4 different things using it. The point of it is basic, rather then VARIABLE=WHATEVER you just copy and paste a .env and it picks it up. But you can also just use regular environment variables if you want.
  - calibas 3y ago
    > I have no idea why you'd want a typed environment variable setup. Because you want to know what type your variables are? Of course, in this case you do know, they're strings.
    - poisonborz 3y ago
      You can trivially write a schema file and a method that checks the existence and type of the env var on startup. I actually did on some projects. It's not needed most of the time so I could only see this as an npm module.
      - maccard 3y ago
andrew_ 3y ago
(Warning: I am being Dennis Downer) Meanwhile, we're still waiting on WebSockets and it's been nearly 10 years. Everyone should read the PR https://github.com/nodejs/node/pull/48890 https://github.com/nodejs/node/pull/48890. It doesn't traverse parent directories, it doesn't have good overwrite/merge logic, multiline support, or variable expansion - things that dotenv has supported for years (sometimes with add-ons). This is another half-useful implementation that the majority will end up ignoring, and keep using better-suited third party packages for. Node is committee'd to death, and the comments in that PR are further proof. I've been in the Node ecosystem since 2010, and my money is on Bun and Deno to lead the way forward.
- madeofpalk 3y ago
  I think it is fine for node to have limited, basic, built in support for .env files, and then leave the opportunity for the ecosystem to add in more features. Easy things should be easy, and hard things should be possible through npm install.
  - c-hendricks 3y ago
    Great way of putting it. It does seem a little off compared to how `npm run` works (it traverses up to a package.json), but I agree .env files could get sketchy. Maybe it could traverse up to where it finds a package.json actually.
- cal85 3y ago
  I agree. Since you seem to know the area, do you have any comment on which contender (Bun or Deno) you see as most likely to take the crown, and also which you’d prefer to take the crown? By ‘the crown’ I mean the status of being the leading JS runtime outside of web browsers. For me, Deno seems very well thought out, I’ve used it a fair bit and found it to work reliably and fast. I like the whitelist approach to system access although I worry that it could be a hurdle for adoption. I haven’t used Bun, just read about it. My sense is that Bun is more of a 1:1 replacement for Node, still tied to npm as a first class citizen, while Deno has a long term strategy to get away from npm (while supporting it as a legacy thing). Overall I’d prefer Deno to win, but I wonder if Bun has a better chance due to its closer parity with Node.
gjvc 3y ago
"node_modules" <mutter>
laurent123456 3y ago
Why would they add built-in support when there's already a reliable well-supported package? Feels like bloatware and waste of resources. I'm sure there are lots of core functionalities that could be improved and optimised instead
- rat9988 3y ago
  https://github.com/nodejs/node/pull/48890 https://github.com/nodejs/node/pull/48890 provides the relevant context in some comment.
- dlojudice 3y ago
  There is a clear trend to absorb popular packages into the core and I believe the main reason is to reduce the number of dependencies. Deno, for example, goes on this direction. Given the recent security issues with npm packages, I think it's a good idea to reduce the number of dependencies.
  - wruza 3y ago
    According to npm, dotenv has zero dependencies. That will reduce the usual dependency count from 12345 to a mind-blowing 12344.
    - shard972 3y ago
      [dead]
  - laurent123456 3y ago
    Bloating runtimes seem like the wrong solution to this security issue though, and Node.js has security issues too. A lot of people don't ever need .env files, yet that will now be part of their deployment which will increase the attack surface of their application.
- _ZeD_ 3y ago
  because "batteries included" is a good thing.
janmarsal 3y ago
When will React env variables work on docker images without any bullshit hacks? I mean when I have my frontend and backend in separate pods, use nginx to serve the frontend in kubernetes and want use BACKEND_URL as an env. Why is it so goddamn hard. Any high IQ people have tips on how to get this to work without some weird injections before building the image? Me dumb.
- batmansmk 3y ago
  hen I see comments like you, Im so glad Kubernetes exist, because I know some of our competitors may be wasting precious cycles trying to make the tech work instead of shipping value. I pretentiously claim that 90% of people using Kubernetes should write efficient algorithms, db requests and maaaaybe look at a CDN and/or buy a bigger server. Like, are your servers really that busy? Stack overflow is 9 servers.... I can testify of some very large, compute intensive service being run on just a couple of servers with the right tech and less abstraction :)
  - janmarsal 3y ago
    Who do you think I am? I can just hardcode the values and I waste no time on this. However, it's not an elegant solution, so that's why I'm asking. Also, I wouldn't even be using kubernetes normally but this one project just happens to require it for 3rd party reasons that are out of my control. I'm sure you feel very smug but that's just delusion for the most part.
- SkyPuncher 3y ago
  There are two things that make this annoying: * most people are pre-building their packages. That means you can’t really modify anything at run time. It also means it can be extremely tedious to test changes and wait for the build process. * there’s a string of places you have to properly “expose” ENV variables to a layer deeper down. This includes some build condos to make sure you’re not exposing sensitive BE configs. It can be challenging debugging the intermediate steps.
- aastronaut 3y ago
  I'm honestly a bit confused by this statement, as "React env variables" can mean various things, depending on your setup. Backend and frontend can be separate runtimes... The backend could be an API-only-server and an OS process with all permissions that come with it (reading env vars during execution as it pleases). The frontend, eg. a React SPA, could just be a bunch of built HTML/CSS/JS-files (type-checked, bundled, minified, etc.), served through a static file server (separate backend, nginx?) and interpreted through a browser engine on the client. Are you asking for nginx to parse the served HTML/CSS/JS-files and replace unknown placeholders with env var values? If so you are prob asking for a SSR framework[0], meaning a backend that treats a React project similar to template files, and is able to inject env vars? [0]: Next.js - https://nextjs.org/docs/app/building-your-application/configuring/environment-variables https://nextjs.org/docs/app/building-your-application/config...
ilyt 3y ago
I'm sure they will find a way to do it securely and not have user uploading wrong name of file fuck up the security of the app..
rbongers 3y ago
Will it have support for "VARIABLE=${VARIABLE:-default}"?
- miohtama 3y ago
  No
  - rbongers 3y ago
    Unfortunate, thank you
miohtama 3y ago
Note that there is no .env file format standard and different tools (Node, Docker, etc) have different formats. Shdotenv helps to convert/reuse environment files across tools https://github.com/ko1nksm/shdotenv https://github.com/ko1nksm/shdotenv
aslilac 3y ago
am I still the only one who thinks storing all of your secrets in plain text while using a package manager that allows arbitrary code execution is a terrible idea?
- SahAssar 3y ago
  Nope, but if your dependencies are reasonable and don't do a bunch of stuff in pre/post-install you can disable it with `--ignore-scripts`. IMO it would be much better if that was the default and dependencies had to ask to run scripts (perhaps with a whitelist in package.json), but unfortunately node did a bunch of "helpful" mistakes and now it's hard to roll back without breaking. Maybe if they had denos permission model they could isolate so that the dependency could only read/write to it's own directory within node_modules.
INTPenis 3y ago
Why? Seems like it goes against the Unix principle. I've been using direnv to set environment for any software, containers, python virtualenvs and more, and it's working quite well. Will nodejs now conflict my direnv files?
_pdp_ 3y ago
Not a big deal! It will not change much and you may still end up using a specific, cross-platform library anyway.
hotnfresh 3y ago
What’s the motivation? I don’t want multiple things reading the file if I’m not working solely in nodejs—I want one tool reading it and setting env vars for all the rest—plus plenty of tools already exist to do this and most are trivial to install and use, and overall it seems like another way for things to go wrong and just a platform-specific Yet Another X burning person-hours to… what benefit?
AtNightWeCode 3y ago
Environment variables should never be used. This was a known security hole already in the last century. Several OS solved this by config files that comes with file permissions. The correct solution on Linux and Windows at least. And still people use envs.
- 1letterunixname 3y ago
  That's unrealistic and not scalable. Heroku and the 12factor pattern use this because it creates a boundary between values meant to be configured by the user and those internal to an application. Environment variables can be injected into a subprocess without them ever existing on disk.
  - AtNightWeCode 3y ago
    Envs are not process secure. Files are. Ram disks exists.
junon 3y ago
But why though? .env (the library) has never made sense to me. There should be a singular binary called dotenv that then takes a command+args as arguments that then runs that command with the .env file's values in the environment. Why do runtimes need first-class support for this?

196 points by mariuz 3y ago

Kinda surprising it took this long. I really hope Deno continues to grow in popularity. I greatly prefer the "batteries-included" and Typescript-first approach.

noman-land 3y ago
I haven't used Deno but have had an easy time using ts-node. https://typestrong.org/ts-node/docs/ https://typestrong.org/ts-node/docs/
- hliyan 3y ago
  I've been eyeing ts-node because I'm sick of all the compilation like ceremonies i need to go through for a supposedly JIT compiled language, i.e. JS. How stable is it for production use?
  - andrew_ 3y ago
    with --swc it's damn fast and very stable. ts-node doesn't recommend you use ts-node in production, however I've seen it done without much consequence.
  - noman-land 3y ago
    I use it for a one-person personal project so hard to call that production but it's been really great. Basically seamless typescript in node as far as I've been concerned.
- adriancooney 3y ago
  I still find `ts-node` gets in my way often with configuration related issues. `tsx` on the other hand has been a dream.

SOLAR_FIELDS 3y ago

.env desperately needs a formal specification. I wrote a bit of code using some features python_dotenv offers but realized after the fact that other software might not parse it the same way. I’ve taken to instead doing my templating in Python itself via pydantic BaseSettings

prng2021 3y ago
Can you provide a concrete example? What do you mean by other software not parsing it the same way?
- moojd 3y ago
  Some execute them as a shell script and others parse them as a newline separated list of key value pairs
- 3np 3y ago
  FOO="abc BAR=123" Is this one or two vars? What is the value of FOO?
  - prng2021 3y ago
    I've never seen something like this in an env file. No one should be putting lines like this in an env file unless it's by mistake.
    - 3np 3y ago
      Your personal opinion of how esoteric or weird this is is besides the point - there are fundamentally incompatible interpretations where I can bet that switching will be breaking and potentially dangerous for some out there.
sergiotapia 3y ago
I've only ever used .env as such: export TWILIO_SID="123123123123" export TWILIO_SECRET="123123123123" And then `source .env` in my terminal, then `rails server` or whatever to run my apps. The environment variables will be present to be consumed from within the app. Curious what features you use beyond this.

netcraft 3y ago

havent dug further in, wonder if it will support multiple files. we use two files, one for things that everyone should have in common and one for things that are different per individual and not committed

c0brac0bra 3y ago
It should. .env.local is a common pattern
- tacker2000 3y ago
  Reminds me of vscode never being able to automatically syntax highight .env.local or similar… or is there some setting im missing?
  - rickstanley 3y ago
    `.env.*` files won't highlight in either VSCode or Neovim, but it works fine with `.*.env`. I guess it's some kind of convention.
    - qntmfred 3y ago
      it's because syntax highlighting rules are applied based on the file extension. as long as .env is the extension, it'll work. if the file ends with .local, vscode has no idea that it's supposed to be a .env file type
      - tacker2000 3y ago
        Yes sure, but since .env.local and .env.prod are so widespread one could make a small filter to also include these types of files, no?
        Timon3 3y ago
        Especially considering the GitHub .gitignore template for Node only ignores .env.local, not .local.env: https://github.com/github/gitignore/blob/main/Node.gitignore#L76-L80 https://github.com/github/gitignore/blob/main/Node.gitignore...

jtwaleson 3y ago

Is it just me or does anyone else hate .env files? They accumulate unused cruft, it's not typed, and typically many systems in the same repo point at the same .env file. It all feels so ... implicit ... I guess. For example in a code base I work in, docker-compose uses it, laravel picks up the same file and Vue / webpack does too. It's a big mess. There must be better solutions out there.

moojd 3y ago
I like them for defining a small number of env vars for my app that only get injected at run time. I don't like them when other random tools that aren't my app use them. Docker's insistence on automatically using the .env in your home directory is absurd.
dubcanada 3y ago
I personally don't feel the same, I have no idea why you'd want a typed environment variable setup. And I find they work great. I don't see any problem with 4 different things using it. The point of it is basic, rather then VARIABLE=WHATEVER you just copy and paste a .env and it picks it up. But you can also just use regular environment variables if you want.
- calibas 3y ago
  > I have no idea why you'd want a typed environment variable setup. Because you want to know what type your variables are? Of course, in this case you do know, they're strings.
  - poisonborz 3y ago
    You can trivially write a schema file and a method that checks the existence and type of the env var on startup. I actually did on some projects. It's not needed most of the time so I could only see this as an npm module.
    - maccard 3y ago

andrew_ 3y ago

(Warning: I am being Dennis Downer) Meanwhile, we're still waiting on WebSockets and it's been nearly 10 years. Everyone should read the PR https://github.com/nodejs/node/pull/48890 https://github.com/nodejs/node/pull/48890. It doesn't traverse parent directories, it doesn't have good overwrite/merge logic, multiline support, or variable expansion - things that dotenv has supported for years (sometimes with add-ons). This is another half-useful implementation that the majority will end up ignoring, and keep using better-suited third party packages for. Node is committee'd to death, and the comments in that PR are further proof. I've been in the Node ecosystem since 2010, and my money is on Bun and Deno to lead the way forward.

madeofpalk 3y ago
I think it is fine for node to have limited, basic, built in support for .env files, and then leave the opportunity for the ecosystem to add in more features. Easy things should be easy, and hard things should be possible through npm install.
- c-hendricks 3y ago
  Great way of putting it. It does seem a little off compared to how `npm run` works (it traverses up to a package.json), but I agree .env files could get sketchy. Maybe it could traverse up to where it finds a package.json actually.
cal85 3y ago
I agree. Since you seem to know the area, do you have any comment on which contender (Bun or Deno) you see as most likely to take the crown, and also which you’d prefer to take the crown? By ‘the crown’ I mean the status of being the leading JS runtime outside of web browsers. For me, Deno seems very well thought out, I’ve used it a fair bit and found it to work reliably and fast. I like the whitelist approach to system access although I worry that it could be a hurdle for adoption. I haven’t used Bun, just read about it. My sense is that Bun is more of a 1:1 replacement for Node, still tied to npm as a first class citizen, while Deno has a long term strategy to get away from npm (while supporting it as a legacy thing). Overall I’d prefer Deno to win, but I wonder if Bun has a better chance due to its closer parity with Node.

gjvc 3y ago

"node_modules" <mutter>

laurent123456 3y ago

Why would they add built-in support when there's already a reliable well-supported package? Feels like bloatware and waste of resources. I'm sure there are lots of core functionalities that could be improved and optimised instead

rat9988 3y ago
https://github.com/nodejs/node/pull/48890 https://github.com/nodejs/node/pull/48890 provides the relevant context in some comment.
dlojudice 3y ago
There is a clear trend to absorb popular packages into the core and I believe the main reason is to reduce the number of dependencies. Deno, for example, goes on this direction. Given the recent security issues with npm packages, I think it's a good idea to reduce the number of dependencies.
- wruza 3y ago
  According to npm, dotenv has zero dependencies. That will reduce the usual dependency count from 12345 to a mind-blowing 12344.
  - shard972 3y ago
    [dead]
- laurent123456 3y ago
  Bloating runtimes seem like the wrong solution to this security issue though, and Node.js has security issues too. A lot of people don't ever need .env files, yet that will now be part of their deployment which will increase the attack surface of their application.
_ZeD_ 3y ago
because "batteries included" is a good thing.

janmarsal 3y ago

When will React env variables work on docker images without any bullshit hacks? I mean when I have my frontend and backend in separate pods, use nginx to serve the frontend in kubernetes and want use BACKEND_URL as an env. Why is it so goddamn hard. Any high IQ people have tips on how to get this to work without some weird injections before building the image? Me dumb.

batmansmk 3y ago
hen I see comments like you, Im so glad Kubernetes exist, because I know some of our competitors may be wasting precious cycles trying to make the tech work instead of shipping value. I pretentiously claim that 90% of people using Kubernetes should write efficient algorithms, db requests and maaaaybe look at a CDN and/or buy a bigger server. Like, are your servers really that busy? Stack overflow is 9 servers.... I can testify of some very large, compute intensive service being run on just a couple of servers with the right tech and less abstraction :)
- janmarsal 3y ago
  Who do you think I am? I can just hardcode the values and I waste no time on this. However, it's not an elegant solution, so that's why I'm asking. Also, I wouldn't even be using kubernetes normally but this one project just happens to require it for 3rd party reasons that are out of my control. I'm sure you feel very smug but that's just delusion for the most part.
SkyPuncher 3y ago
There are two things that make this annoying: * most people are pre-building their packages. That means you can’t really modify anything at run time. It also means it can be extremely tedious to test changes and wait for the build process. * there’s a string of places you have to properly “expose” ENV variables to a layer deeper down. This includes some build condos to make sure you’re not exposing sensitive BE configs. It can be challenging debugging the intermediate steps.
aastronaut 3y ago
I'm honestly a bit confused by this statement, as "React env variables" can mean various things, depending on your setup. Backend and frontend can be separate runtimes... The backend could be an API-only-server and an OS process with all permissions that come with it (reading env vars during execution as it pleases). The frontend, eg. a React SPA, could just be a bunch of built HTML/CSS/JS-files (type-checked, bundled, minified, etc.), served through a static file server (separate backend, nginx?) and interpreted through a browser engine on the client. Are you asking for nginx to parse the served HTML/CSS/JS-files and replace unknown placeholders with env var values? If so you are prob asking for a SSR framework[0], meaning a backend that treats a React project similar to template files, and is able to inject env vars? [0]: Next.js - https://nextjs.org/docs/app/building-your-application/configuring/environment-variables https://nextjs.org/docs/app/building-your-application/config...

ilyt 3y ago

I'm sure they will find a way to do it securely and not have user uploading wrong name of file fuck up the security of the app..

rbongers 3y ago

Will it have support for "VARIABLE=${VARIABLE:-default}"?

miohtama 3y ago
No
- rbongers 3y ago
  Unfortunate, thank you

miohtama 3y ago

Note that there is no .env file format standard and different tools (Node, Docker, etc) have different formats. Shdotenv helps to convert/reuse environment files across tools https://github.com/ko1nksm/shdotenv https://github.com/ko1nksm/shdotenv

aslilac 3y ago

am I still the only one who thinks storing all of your secrets in plain text while using a package manager that allows arbitrary code execution is a terrible idea?

SahAssar 3y ago
Nope, but if your dependencies are reasonable and don't do a bunch of stuff in pre/post-install you can disable it with `--ignore-scripts`. IMO it would be much better if that was the default and dependencies had to ask to run scripts (perhaps with a whitelist in package.json), but unfortunately node did a bunch of "helpful" mistakes and now it's hard to roll back without breaking. Maybe if they had denos permission model they could isolate so that the dependency could only read/write to it's own directory within node_modules.

INTPenis 3y ago

Why? Seems like it goes against the Unix principle. I've been using direnv to set environment for any software, containers, python virtualenvs and more, and it's working quite well. Will nodejs now conflict my direnv files?

_pdp_ 3y ago

Not a big deal! It will not change much and you may still end up using a specific, cross-platform library anyway.

hotnfresh 3y ago

What’s the motivation? I don’t want multiple things reading the file if I’m not working solely in nodejs—I want one tool reading it and setting env vars for all the rest—plus plenty of tools already exist to do this and most are trivial to install and use, and overall it seems like another way for things to go wrong and just a platform-specific Yet Another X burning person-hours to… what benefit?

AtNightWeCode 3y ago

Environment variables should never be used. This was a known security hole already in the last century. Several OS solved this by config files that comes with file permissions. The correct solution on Linux and Windows at least. And still people use envs.

1letterunixname 3y ago
That's unrealistic and not scalable. Heroku and the 12factor pattern use this because it creates a boundary between values meant to be configured by the user and those internal to an application. Environment variables can be injected into a subprocess without them ever existing on disk.
- AtNightWeCode 3y ago
  Envs are not process secure. Files are. Ram disks exists.

junon 3y ago

But why though? .env (the library) has never made sense to me. There should be a singular binary called dotenv that then takes a command+args as arguments that then runs that command with the .env file's values in the environment. Why do runtimes need first-class support for this?