4 ms·

Shell-secrets – GPG-encrypted environment variables

93 points by mgarciaisaia 1y ago

woodruffw 1y ago
The more general version of this is probably sops[1]. (A general problem with these kinds of “wrap GPG” tools is that you end up with “mystery meat” encryption/signatures: your tool’s security margin is at the mercy of GPG’s opaque and historically not very good defaults.) [1]: https://github.com/getsops/sops https://github.com/getsops/sops
- aborsy 1y ago
  GPG man page is long. But to be fair, GPG, which I have used for decades, has never failed me.
- theteapot 1y ago
  This is 13 lines of Bash plus GPG which is available ~everywhere and a pretty lowish level Linux dependency. SOPS is +20KLOC of Go with support for cloud KMS etc etc. I think you got your mystery meat analogy backwards.
  - woodruffw 1y ago
    The mystery meat in question is GPG, not sops or this. (I also wouldn’t call GPG a low level dependency.)
    - theteapot 1y ago
      lowish. Meaning if you run a Linux desktop env with a mild amount of software installed it's likely pulled in already.
      - ikiris 1y ago
        So is Perl, that doesn’t make it a good argument to use it still for the same reasons.
        akoboldfrying 1y ago
        Perl is horrible, but for one-liners it's strictly less horrible than either sed or awk, which people still use because they are less horrible than pure Bourne shell for some common tasks.
      - woodruffw 1y ago
        I’ve used a Linux desktop for my entire adult life, and I’m pretty sure GPG has never been bundled directly with my environment. I used to install it directly, but I haven’t needed that in years either since everything I needed GPG for (= git) supports SSH signing instead.
- mgarciaisaia 1y ago
  I didn't know about sops, thanks for sharing! Encrypting YAML files' values may be handy for another project - will take note of it.
pluto_modadic 1y ago
for a newer password manager... https://github.com/FiloSottile/passage https://github.com/FiloSottile/passage
- qyckudnefDi5 1y ago
  Looks like FiloSottile may have switched from passage to 1Password: https://bsky.app/profile/filippo.abyssdomain.expert/post/3l5hd5x5cfy2c https://bsky.app/profile/filippo.abyssdomain.expert/post/3l5... Would be interesting to get more context why move from storing passwords locally to an online service.
  - FiloSottile 1y ago
    Team sharing with a non-technical person, mostly. I still have high-value passwords and CLI credentials in passage + age-plugin-yubikey.
bitbasher 1y ago
Couldn't you just use pass and have something like this in your bash script/env: export SOME_SECRET="$(pass show some/secret)"
- Piraty 1y ago
  this in a credentials file to source before doing some operation? sure. I usually do: ` ( . ./credentials && ./the_thing ) ` so the secrets are only in the subshell and don't linger in my shell session forever. but don't put that in <shell>rc , as it a) will be visible for all other (child) processes of your shell b) will spawn pinentry everytime the agent's cache ttl expires
- varenc 1y ago
  That hides it in the source, but doesn't hide it in the execution environment that can access the ENV. Everything you run inside your shell could still read it. (but if you're running untrusted things...you've already lost)
ognarb 1y ago
I like the idea. GPG encryption are super helful when sharing secrets. Disclaimer: I work on some UI for GPG as my day job.
hnlmorg 1y ago
Coincidentally I’ve written something similar to this too. My main takeaway was that GPG isn’t nearly as user friendly as it needs to be.
- mmh0000 1y ago
  Highly true. Yet. If you complain or even offer patches (which will, always, without fail, be rejected). You'll get told off by the GPG devs with something along the lines of "encryption is supposed to be hard".
  - 9dev 1y ago
    How hard would it be to devise an easy to use wrapper on top of GPG, kind of porcelain-like?
    - thayne 1y ago
      You may be interested in https://sequoia-pgp.org/ https://sequoia-pgp.org/ It isn't exactly a wrapper, but it has an easier to use interface (as well as a more gpg compatible interface).
    - ognarb 1y ago
      It already exists and it's called Kleopatra. It's developed by KDE with some support from the GPG developers and is part of the Gpg4Win suite. It's used by quite a few companies and public administrations.
    - akerl_ 1y ago
      The easier and more productive thing is to make an easy-to-use tool that does a specific workflow vs trying to be a swiss army knife. https://github.com/FiloSottile/age https://github.com/FiloSottile/age is this for encrypting files. https://en.wikipedia.org/wiki/Signify_(OpenBSD) https://en.wikipedia.org/wiki/Signify_(OpenBSD) and https://jedisct1.github.io/minisign/ https://jedisct1.github.io/minisign/ are this for signing files. Signal/Whatsapp/etc that use the Signal Protocal are this for messaging. It turns out solving one problem at a time and ending up with a bunch of purpose-built tools is way easier to get right than trying to jam an entire toolbox into one thing.
Valodim 1y ago
The correct way to do stuff like this these days with openpgp is to use a SOP (stateless openpgp) implementation. https://www.openpgp.org/about/sop/ https://www.openpgp.org/about/sop/
viraptor 1y ago
Unless you're good at actually maintaining your gpg keychain and need other people to access this, I really wouldn't bother with gpg. There are way better and simpler options. Age has a simpler interface and SSH key support https://github.com/FiloSottile/age https://github.com/FiloSottile/age ejson2env has the environment variable integration and ejson has multiple backends https://github.com/Shopify/ejson2env https://github.com/Shopify/ejson2env direnv can support any cli secrets manager per project directory https://direnv.net/ https://direnv.net/ I've dealt with enough "why did this break" situations with gpg secrets files used by capable teams that I'd never recommend that to anyone. And unless you really need the public key support (teams and deployment support), you're unlikely to gain anything better over a password manager.
- akoboldfrying 1y ago
  age looks really interesting, thanks. I also learned from that page that appending ".keys" to your GitHub profile URL (so https://github.com/yourusername.keys https://github.com/yourusername.keys) returns a list of your SSH public keys! (Where is this documented...?)
  - tomjakubowski 1y ago
    Another trick with github urls: you can append .patch or .diff to any PR or commit URL, and you'll get back a git-formatted patch or diff. https://github.com/rust-lang/rust/pull/139966 https://github.com/rust-lang/rust/pull/139966 https://github.com/rust-lang/rust/pull/139966.patch https://github.com/rust-lang/rust/pull/139966.patch https://github.com/rust-lang/rust/pull/139966.diff https://github.com/rust-lang/rust/pull/139966.diff
- theteapot 1y ago
  The tool is just pulling one encryption key from your local GPG keyring. What's to maintain?
  - viraptor 1y ago
    What happens when you have multiple matching keys? What happens when your key expires? What happens when the output format changes? What happens when the key expires and it's attached to a hardware device? Gpg can fail in ways which do not tell you anything about the real underlying issue. I promise this happens all the time to people for lots of stupid reasons.
asveikau 1y ago
I do something like this in my .muttrc. It was showing up in documentation iirc, as the typical way to store credentials for mutt.
dvektor 1y ago
I store my secrets in gpg encrypted files and inject them into my environment in my shell rc file. AWS_SECRET_ACCESS_KEY=$(gpg -d ~/.secrets/aws/key.asc) type of deal. its annoying to put in a password every time i open a new tmux pane but hey, better than plain text.
- viraptor 1y ago
  If you're using more complicated systems than just a single root account, have a look at https://github.com/99designs/aws-vault https://github.com/99designs/aws-vault too.
- mgarciaisaia 1y ago
  That was what I did before knowing about shell-secrets. But I also need different "contexts" on the same domains/tools (different AWS accounts and credentials for different clients), and having none "set" by default prevents me from running _whatever command_ by mistake the majority of the time.
ykonstant 1y ago
Since GPG and openssh support the TPM for some operations, I am tempted to store secrets in the TPM instead; I think a hardware safe is better than messing with persistent envars and having to pay attention to children etc. But I am very nervous about doing so, since I have heard bad things about the reliability of the TPM (limited writes or something?) and locking myself out of important places. Any people with experience using the TPM for secrets in Linux?
vcdimension 1y ago
I've forked the repo and created a zsh version: https://github.com/vapniks/shell-secrets https://github.com/vapniks/shell-secrets

93 points by mgarciaisaia 1y ago

The more general version of this is probably sops[1]. (A general problem with these kinds of “wrap GPG” tools is that you end up with “mystery meat” encryption/signatures: your tool’s security margin is at the mercy of GPG’s opaque and historically not very good defaults.) [1]: https://github.com/getsops/sops https://github.com/getsops/sops

aborsy 1y ago
GPG man page is long. But to be fair, GPG, which I have used for decades, has never failed me.
theteapot 1y ago
This is 13 lines of Bash plus GPG which is available ~everywhere and a pretty lowish level Linux dependency. SOPS is +20KLOC of Go with support for cloud KMS etc etc. I think you got your mystery meat analogy backwards.
- woodruffw 1y ago
  The mystery meat in question is GPG, not sops or this. (I also wouldn’t call GPG a low level dependency.)
  - theteapot 1y ago
    lowish. Meaning if you run a Linux desktop env with a mild amount of software installed it's likely pulled in already.
    - ikiris 1y ago
      So is Perl, that doesn’t make it a good argument to use it still for the same reasons.
      - akoboldfrying 1y ago
        Perl is horrible, but for one-liners it's strictly less horrible than either sed or awk, which people still use because they are less horrible than pure Bourne shell for some common tasks.
    - woodruffw 1y ago
      I’ve used a Linux desktop for my entire adult life, and I’m pretty sure GPG has never been bundled directly with my environment. I used to install it directly, but I haven’t needed that in years either since everything I needed GPG for (= git) supports SSH signing instead.
mgarciaisaia 1y ago
I didn't know about sops, thanks for sharing! Encrypting YAML files' values may be handy for another project - will take note of it.

pluto_modadic 1y ago

for a newer password manager... https://github.com/FiloSottile/passage https://github.com/FiloSottile/passage

qyckudnefDi5 1y ago
Looks like FiloSottile may have switched from passage to 1Password: https://bsky.app/profile/filippo.abyssdomain.expert/post/3l5hd5x5cfy2c https://bsky.app/profile/filippo.abyssdomain.expert/post/3l5... Would be interesting to get more context why move from storing passwords locally to an online service.
- FiloSottile 1y ago
  Team sharing with a non-technical person, mostly. I still have high-value passwords and CLI credentials in passage + age-plugin-yubikey.

bitbasher 1y ago

Couldn't you just use pass and have something like this in your bash script/env: export SOME_SECRET="$(pass show some/secret)"

Piraty 1y ago
this in a credentials file to source before doing some operation? sure. I usually do: ` ( . ./credentials && ./the_thing ) ` so the secrets are only in the subshell and don't linger in my shell session forever. but don't put that in <shell>rc , as it a) will be visible for all other (child) processes of your shell b) will spawn pinentry everytime the agent's cache ttl expires
varenc 1y ago
That hides it in the source, but doesn't hide it in the execution environment that can access the ENV. Everything you run inside your shell could still read it. (but if you're running untrusted things...you've already lost)

ognarb 1y ago

I like the idea. GPG encryption are super helful when sharing secrets. Disclaimer: I work on some UI for GPG as my day job.

hnlmorg 1y ago

Coincidentally I’ve written something similar to this too. My main takeaway was that GPG isn’t nearly as user friendly as it needs to be.

mmh0000 1y ago
Highly true. Yet. If you complain or even offer patches (which will, always, without fail, be rejected). You'll get told off by the GPG devs with something along the lines of "encryption is supposed to be hard".
- 9dev 1y ago
  How hard would it be to devise an easy to use wrapper on top of GPG, kind of porcelain-like?
  - thayne 1y ago
    You may be interested in https://sequoia-pgp.org/ https://sequoia-pgp.org/ It isn't exactly a wrapper, but it has an easier to use interface (as well as a more gpg compatible interface).
  - ognarb 1y ago
    It already exists and it's called Kleopatra. It's developed by KDE with some support from the GPG developers and is part of the Gpg4Win suite. It's used by quite a few companies and public administrations.
  - akerl_ 1y ago
    The easier and more productive thing is to make an easy-to-use tool that does a specific workflow vs trying to be a swiss army knife. https://github.com/FiloSottile/age https://github.com/FiloSottile/age is this for encrypting files. https://en.wikipedia.org/wiki/Signify_(OpenBSD) https://en.wikipedia.org/wiki/Signify_(OpenBSD) and https://jedisct1.github.io/minisign/ https://jedisct1.github.io/minisign/ are this for signing files. Signal/Whatsapp/etc that use the Signal Protocal are this for messaging. It turns out solving one problem at a time and ending up with a bunch of purpose-built tools is way easier to get right than trying to jam an entire toolbox into one thing.

Valodim 1y ago

The correct way to do stuff like this these days with openpgp is to use a SOP (stateless openpgp) implementation. https://www.openpgp.org/about/sop/ https://www.openpgp.org/about/sop/

viraptor 1y ago

Unless you're good at actually maintaining your gpg keychain and need other people to access this, I really wouldn't bother with gpg. There are way better and simpler options. Age has a simpler interface and SSH key support https://github.com/FiloSottile/age https://github.com/FiloSottile/age ejson2env has the environment variable integration and ejson has multiple backends https://github.com/Shopify/ejson2env https://github.com/Shopify/ejson2env direnv can support any cli secrets manager per project directory https://direnv.net/ https://direnv.net/ I've dealt with enough "why did this break" situations with gpg secrets files used by capable teams that I'd never recommend that to anyone. And unless you really need the public key support (teams and deployment support), you're unlikely to gain anything better over a password manager.

akoboldfrying 1y ago
age looks really interesting, thanks. I also learned from that page that appending ".keys" to your GitHub profile URL (so https://github.com/yourusername.keys https://github.com/yourusername.keys) returns a list of your SSH public keys! (Where is this documented...?)
- tomjakubowski 1y ago
  Another trick with github urls: you can append .patch or .diff to any PR or commit URL, and you'll get back a git-formatted patch or diff. https://github.com/rust-lang/rust/pull/139966 https://github.com/rust-lang/rust/pull/139966 https://github.com/rust-lang/rust/pull/139966.patch https://github.com/rust-lang/rust/pull/139966.patch https://github.com/rust-lang/rust/pull/139966.diff https://github.com/rust-lang/rust/pull/139966.diff
theteapot 1y ago
The tool is just pulling one encryption key from your local GPG keyring. What's to maintain?
- viraptor 1y ago
  What happens when you have multiple matching keys? What happens when your key expires? What happens when the output format changes? What happens when the key expires and it's attached to a hardware device? Gpg can fail in ways which do not tell you anything about the real underlying issue. I promise this happens all the time to people for lots of stupid reasons.

asveikau 1y ago

I do something like this in my .muttrc. It was showing up in documentation iirc, as the typical way to store credentials for mutt.

dvektor 1y ago

I store my secrets in gpg encrypted files and inject them into my environment in my shell rc file. AWS_SECRET_ACCESS_KEY=$(gpg -d ~/.secrets/aws/key.asc) type of deal. its annoying to put in a password every time i open a new tmux pane but hey, better than plain text.

viraptor 1y ago
If you're using more complicated systems than just a single root account, have a look at https://github.com/99designs/aws-vault https://github.com/99designs/aws-vault too.
mgarciaisaia 1y ago
That was what I did before knowing about shell-secrets. But I also need different "contexts" on the same domains/tools (different AWS accounts and credentials for different clients), and having none "set" by default prevents me from running _whatever command_ by mistake the majority of the time.

ykonstant 1y ago

Since GPG and openssh support the TPM for some operations, I am tempted to store secrets in the TPM instead; I think a hardware safe is better than messing with persistent envars and having to pay attention to children etc. But I am very nervous about doing so, since I have heard bad things about the reliability of the TPM (limited writes or something?) and locking myself out of important places. Any people with experience using the TPM for secrets in Linux?

vcdimension 1y ago

I've forked the repo and created a zsh version: https://github.com/vapniks/shell-secrets https://github.com/vapniks/shell-secrets