5 ms·

I'm not sure I've seen it mentioned yet that when Vercel rolled out their environment variable UI, there was no "sensitive" option https://github.com/vercel/ver

I'm not sure I've seen it mentioned yet that when Vercel rolled out their environment variable UI, there was no "sensitive" option https://github.com/vercel/vercel/discussions/4558#discussioncomment-3665817 https://github.com/vercel/vercel/discussions/4558#discussion.... There was ~2 years or more until it was introduced https://vercel.com/changelog/sensitive-environment-variables-are-now-available https://vercel.com/changelog/sensitive-environment-variables...

deleted 6mo ago
[deleted]
_pdp_ 6mo ago
Sensitive does not mean it is not readable. It is just simply not exposed through the UI. It can be easily leaked if you return a bit too much props from the action functions or routes. The only way to defend against these types of issues is to encrypt your environment with your own keys, with secrets possibly baked into source as there are no other facilities to separate them. An attacker would need to not only read the environments but also download the compiled functions and find the decryption keys. It is not ideal but it could work as a workaround.
- harikb 6mo ago
  > with secrets possibly baked into source please don't suggest this. The right way is to have the creds fetched from a vault, which is programmed to release the creds auth-free to your VM (with machine level identify managed by the parent platform) This is how Google Secrets or AWS Vaults work.
  - SoftTalker 6mo ago
    This is just another layer of indirection (which isn't bad; it adds to the difficulty of executing a breach). The fundamental problem with encrypted secrets is that at some point you need to access and decrypt them.
    - SAI_Peregrinus 6mo ago
      HSMs & similar can at least time-limit access to secrets to the period where an attacker can make requests to the HSM.
      - recursivegirth 6mo ago
        I think the problem is the way we are using these "secrets" services traditionally. The requesting process/machine should NEVER see the Oauth client secret. The short-lived session token should be the only piece of data the server/client are ever privy too. The service that encrypts the data should be the ONLY service that holds the private key to decrypt, and therefore the only service that can process the decrypted data.
        oasisbob 6mo ago
        The service wouldn't have access to the refresh token? How does authentication with the client-secret-holding intermediary work? It's easy to see how this would work with sufficiently sophisticated clients in some use-cases, say via a vault plugin, but posing this as a universal necessity feels like a big departure from typical oauth flows, and the added complexity could be harmful depending on what home-grown solutions are used to implement it.
    - deleted 6mo ago
      [deleted]
    - zbentley 6mo ago
      I'm not sure that's necessarily a "problem", though it is fundamental to secrets. We wouldn't say that it's a fundamental problem that doors on houses need a key--that's what the key is for--the problem is if the key isn't kept secure from unauthorized actors. Like, sure, you can go HAM here and use network proxy services to do secret decryption, and only talk from the app to those proxies via short-lived tokens; that's arguably a qualitative shift from app-uses-secret-directly, and it has some real benefits (and costs, namely significant complexity/fragility). Instead, my favored option is to scope secret use to network locations. If, for example, a given NPM token can only be used for API calls issued from the public IP endpoint of the user's infrastructure, that's a significant added layer of security. People don't agree on whether or not this counts as a "token ACL", but it's certainly ACL-like in its functionality--just controlled by location, rather than identity. This approach can also be adopted gradually and with less added fragility than the proxy-all-the-things approach: token holders can initially allowlist broad or shared network location ranges, and narrow allowed access sources over time as their networks are improved. Of course, that's a fantasy. API providers would have to support network-scoped API access credentials, and almost none of them do.
  - _pdp_ 6mo ago
    I was reffering to Vercel. Other cloud environments have much better mechanisms for securing secrets.
  - jcgl 6mo ago
    > The right way is to have the creds fetched from a vault, which is programmed to release the creds auth-free to your VM Or have whatever deployment tool that currently populates the env vars instead use the same information to populate files on the filesystem (like mounting creds).
  - chatmasta 6mo ago
    Next.js renders configuration that’s shared by client and server into a JSON blob in the HTML page. These config variables often come from environment variables. It’s a very common mistake for people to not realize this, and accidentally put what should be a server-only secret into this config. I’ve seen API secrets in HTML source code because of this. The client app doesn’t even use it, but it’s part of the next config so it renders into the page.
    - whh 6mo ago
      That's essentially what NEXT_PUBLIC_ is for... but serializing process.env is a new one for me.
      - chatmasta 6mo ago
        They don’t serialize process.env, but devs will take config values from environment variables. Obviously you’re not supposed to do this but it’s a footgun.
    - socalgal2 6mo ago
      IIRC, react had this issue so they required env vars seen in react to be prefixed by REACT_ The hope being that SECRET is not prefixed and so is not available. Of course it requires you to know why they are prefixed and not make REACT_SECRET
  - elwebmaster 6mo ago
    "The parent platform" yada yada, my parent platform is bare metal, how about that?
  - luisfdias 6mo ago
    +1 on vaults. One step further: credentials that never land in the runtime environment at all. App authenticates to a gateway via workload identity, gateway proxies the call, process never sees the secret. Makes env enumeration useless even with valid admin access (I work on an open-source tool in this space, so I'm biased).
- awestroke 6mo ago
  The better way to defend against these types of issues is to avoid Vercel and similar providers
  - elwebmaster 6mo ago
    Nailed it.
- eecc 6mo ago
  Looks like how GitLab does it. As far as I’m concerned, the only sane way is to dump credentials in a well-known path and let the environment decide what to bind them with at runtime (which is how Kubernetes does it, at least the EKS version I’ve had to work with). IOW, JEE variable binding (JNDI) did it right 20 years or so ago. It might be worth for architecture designers to look back at that engineering monument (in all its possible meanings, it felt complicated at times) and study its solutions before coming up with a different solution to a problem it solved
nopointttt 6mo ago
A sensitive flag at the UI layer doesn't actually change runtime. Once it's in process.env during a build, any dep that decides to grep it can. The real problem isn't a missing checkbox, it's that we still stuff every secret into one env bag and hand the build tools the whole bag. Cloudflare scoped bindings and Fly already split it up, other platforms are just slower.
- miki123211 6mo ago
  Yes. Ideally, your application code shouldn't ever need to see your secrets, those should only be accessible to tiny components that only expose the operations that the app actually needs to do. Instead of your app having an OPENAI_API_KEY, there should be some kind of external HTTP proxy that adds this key whenever the right endpoint is called. A man can dream though.
  - fireant 6mo ago
    This could probably be a generic MITM HTTP proxy as well, keep OPENAI_API_KEY=OPENAI_API_KEY in your .env and then replace this with the real key inside the proxy. It wouldn't need to know anything about endpoints or services.

deleted 6mo ago

[deleted]

_pdp_ 6mo ago

Sensitive does not mean it is not readable. It is just simply not exposed through the UI. It can be easily leaked if you return a bit too much props from the action functions or routes. The only way to defend against these types of issues is to encrypt your environment with your own keys, with secrets possibly baked into source as there are no other facilities to separate them. An attacker would need to not only read the environments but also download the compiled functions and find the decryption keys. It is not ideal but it could work as a workaround.

harikb 6mo ago
> with secrets possibly baked into source please don't suggest this. The right way is to have the creds fetched from a vault, which is programmed to release the creds auth-free to your VM (with machine level identify managed by the parent platform) This is how Google Secrets or AWS Vaults work.
- SoftTalker 6mo ago
  This is just another layer of indirection (which isn't bad; it adds to the difficulty of executing a breach). The fundamental problem with encrypted secrets is that at some point you need to access and decrypt them.
  - SAI_Peregrinus 6mo ago
    HSMs & similar can at least time-limit access to secrets to the period where an attacker can make requests to the HSM.
    - recursivegirth 6mo ago
      I think the problem is the way we are using these "secrets" services traditionally. The requesting process/machine should NEVER see the Oauth client secret. The short-lived session token should be the only piece of data the server/client are ever privy too. The service that encrypts the data should be the ONLY service that holds the private key to decrypt, and therefore the only service that can process the decrypted data.
      - oasisbob 6mo ago
        The service wouldn't have access to the refresh token? How does authentication with the client-secret-holding intermediary work? It's easy to see how this would work with sufficiently sophisticated clients in some use-cases, say via a vault plugin, but posing this as a universal necessity feels like a big departure from typical oauth flows, and the added complexity could be harmful depending on what home-grown solutions are used to implement it.
  - deleted 6mo ago
    [deleted]
  - zbentley 6mo ago
    I'm not sure that's necessarily a "problem", though it is fundamental to secrets. We wouldn't say that it's a fundamental problem that doors on houses need a key--that's what the key is for--the problem is if the key isn't kept secure from unauthorized actors. Like, sure, you can go HAM here and use network proxy services to do secret decryption, and only talk from the app to those proxies via short-lived tokens; that's arguably a qualitative shift from app-uses-secret-directly, and it has some real benefits (and costs, namely significant complexity/fragility). Instead, my favored option is to scope secret use to network locations. If, for example, a given NPM token can only be used for API calls issued from the public IP endpoint of the user's infrastructure, that's a significant added layer of security. People don't agree on whether or not this counts as a "token ACL", but it's certainly ACL-like in its functionality--just controlled by location, rather than identity. This approach can also be adopted gradually and with less added fragility than the proxy-all-the-things approach: token holders can initially allowlist broad or shared network location ranges, and narrow allowed access sources over time as their networks are improved. Of course, that's a fantasy. API providers would have to support network-scoped API access credentials, and almost none of them do.
- _pdp_ 6mo ago
  I was reffering to Vercel. Other cloud environments have much better mechanisms for securing secrets.
- jcgl 6mo ago
  > The right way is to have the creds fetched from a vault, which is programmed to release the creds auth-free to your VM Or have whatever deployment tool that currently populates the env vars instead use the same information to populate files on the filesystem (like mounting creds).
- chatmasta 6mo ago
  Next.js renders configuration that’s shared by client and server into a JSON blob in the HTML page. These config variables often come from environment variables. It’s a very common mistake for people to not realize this, and accidentally put what should be a server-only secret into this config. I’ve seen API secrets in HTML source code because of this. The client app doesn’t even use it, but it’s part of the next config so it renders into the page.
  - whh 6mo ago
    That's essentially what NEXT_PUBLIC_ is for... but serializing process.env is a new one for me.
    - chatmasta 6mo ago
      They don’t serialize process.env, but devs will take config values from environment variables. Obviously you’re not supposed to do this but it’s a footgun.
  - socalgal2 6mo ago
    IIRC, react had this issue so they required env vars seen in react to be prefixed by REACT_ The hope being that SECRET is not prefixed and so is not available. Of course it requires you to know why they are prefixed and not make REACT_SECRET
- elwebmaster 6mo ago
  "The parent platform" yada yada, my parent platform is bare metal, how about that?
- luisfdias 6mo ago
  +1 on vaults. One step further: credentials that never land in the runtime environment at all. App authenticates to a gateway via workload identity, gateway proxies the call, process never sees the secret. Makes env enumeration useless even with valid admin access (I work on an open-source tool in this space, so I'm biased).
awestroke 6mo ago
The better way to defend against these types of issues is to avoid Vercel and similar providers
- elwebmaster 6mo ago
  Nailed it.
eecc 6mo ago
Looks like how GitLab does it. As far as I’m concerned, the only sane way is to dump credentials in a well-known path and let the environment decide what to bind them with at runtime (which is how Kubernetes does it, at least the EKS version I’ve had to work with). IOW, JEE variable binding (JNDI) did it right 20 years or so ago. It might be worth for architecture designers to look back at that engineering monument (in all its possible meanings, it felt complicated at times) and study its solutions before coming up with a different solution to a problem it solved

nopointttt 6mo ago

A sensitive flag at the UI layer doesn't actually change runtime. Once it's in process.env during a build, any dep that decides to grep it can. The real problem isn't a missing checkbox, it's that we still stuff every secret into one env bag and hand the build tools the whole bag. Cloudflare scoped bindings and Fly already split it up, other platforms are just slower.

miki123211 6mo ago
Yes. Ideally, your application code shouldn't ever need to see your secrets, those should only be accessible to tiny components that only expose the operations that the app actually needs to do. Instead of your app having an OPENAI_API_KEY, there should be some kind of external HTTP proxy that adds this key whenever the right endpoint is called. A man can dream though.
- fireant 6mo ago
  This could probably be a generic MITM HTTP proxy as well, keep OPENAI_API_KEY=OPENAI_API_KEY in your .env and then replace this with the real key inside the proxy. It wouldn't need to know anything about endpoints or services.