7 ms·

Show HN: Partially encrypt a file based on its HEREDOCs

38 points by higgins 5y ago

Hi HN! I wrote a tool that partially encrypts files based on the presence of a HEREDOC. Check it out: https://github.com/higgins/privatize When added to a git repo, it will automatically transparently encrypt/decrypt files you want privatized. For example if you configured your repo to privatize the file `example.txt`, you could write: ``` Today I a burrito. <<PRIVATE I was on the toilet for hours. PRIVATE I got a lot of reading done. ``` but when git-commit'ed would become: ``` Today I a burrito. <<PRIVATE xuJ0fld2vmNWaVLogTIufmWsiFso PRIVATE I got a lot of reading done. ``` Diffing works as you expect (on the unencrypted source) and only those with the `privatize` symmetric key would be able to unlock and decrypt these files. Why did I do this? I keep a public log of what I plan to accomplish and what I'm working on both personally/professionally. At the end of the day, I write a summary of everything that happened. Naturally, there are some details of my life that should be kept private (details of too-be-launched projects, sensitive family events, etc). It's helpful for me to track everything in one file so as to keep the day's context together. Would love to know what you think! Justin

higgins 5y ago
FYI: this was inspired by the great `git-crypt` (https://github.com/AGWA/git-crypt https://github.com/AGWA/git-crypt) More on my motivations here: https://encapsulate.me/writing/Privatize.html https://encapsulate.me/writing/Privatize.html
mdaniel 5y ago
Yikes, do all filters do that? https://github.com/higgins/privatize/blob/0.1.1/index.js#L98 https://github.com/higgins/privatize/blob/0.1.1/index.js#L98 Also, it's 2022 and I still have to remind people not to commit binary assets into git repos, as the repo will grow without bound: https://github.com/higgins/privatize/tree/main/release https://github.com/higgins/privatize/tree/main/release That's the very problem that GitHub Releases were designed to address, and has the extra awesome benefit of using (currently) AWS S3 for distribution, which is almost certainly going to be faster and place less load upon github.com than .../raw/main/release/whatever.exe Also, while the Brew tap indicates your code is ISC, there is no license file in your repo: https://github.com/higgins/homebrew-privatize/blob/main/Formula/privatize.rb#L8 https://github.com/higgins/homebrew-privatize/blob/main/Form...
- deleted 5y ago
  [deleted]
- higgins 5y ago
  Thanks for your thorough feedback! Much appreciated Added some more TODOs to address your comments. :) FWIW, the `git reset --hard` is only on unlocking a repo that was previously privatize'd. Still, not a great FTUX. Will update.
nnf 5y ago
This is a neat idea. I wrote something similar recently but with the aim of encrypting sensitive values (like API keys) in YAML config files, for similar reasons — so people without the key can see most of the config but not the secret parts. The script is then used by an automated deployment process to decrypt the sensitive values when the config file is moved into place.
- higgins 5y ago
  Nice! You can use it as a cli tool too: ``` # Create a stand-alone symmetric key for use outside of git privatize create-key symmetric.key # Encrypt a file and pipe to stdout cat someFileToPartiallyEncrypt | privatize encrypt symmetric.key # Decrypt a file and pipe to stdout cat someFileToPartiallyDecrypt | privatize decrypt symmetric.key ``` Open to pull requests if you want to adapt to your use cases too
klyrs 5y ago
Do you check that the sentinel string PRIVATE does not occur in the encrypted data?
- higgins 5y ago
  hey, thanks! added to my TODOs: https://github.com/higgins/privatize/commit/53f62483f943112eb7bd2e7bf7feaa1f8fc03c92 https://github.com/higgins/privatize/commit/53f62483f943112e...
  - klyrs 5y ago
    It's one of those alarm bells that got installed a very long time ago in my career... makes for some really exciting bugs.
deleted 5y ago
[deleted]
amenghra 5y ago
I might not have properly understood this tool, but is the encryption key and iv getting reused? If so, it's usually quite unsafe to reuse IVs (but consult a cryptographer, YMMV). Feels weird to stuff the IV with the key, it should live with the ciphertext. (I only glanced at the code for a few minutes, so I could be wrong: https://github.com/higgins/privatize/blob/0.1.1/index.js#L73 https://github.com/higgins/privatize/blob/0.1.1/index.js#L73)
- higgins 5y ago
  Thanks for taking a look! Yeah, I should cleanup the wording in here a bit: The IV is a sha1 hmac of the content to be encrypted. The 16B that are stored with the key is used to initialize the sha1 hmac: https://github.com/higgins/privatize/blob/0.1.1/index.js#L20-L26 https://github.com/higgins/privatize/blob/0.1.1/index.js#L20...
  - deathanatos 5y ago
    Given that you store the IV alongside the ciphertext anyways, I'm not seeing what advantage this has over just choosing the IV at random.
    - re 5y ago
      This appears to be a design decision borrowed/inherited from git-crypt: https://github.com/AGWA/git-crypt#security https://github.com/AGWA/git-crypt#security The gain is that encrypted output is deterministic (which git seems to require), with the trade-off of leaking some information (particularly that multiple encrypted ciphertexts are identical). I'm not really a crypto expert, but it seems like a better alternative might be to use AES-GCM-SIV. It is designed to allow for nonce reuse, and has the advantage of being authenticated encryption. https://en.wikipedia.org/wiki/AES-GCM-SIV https://en.wikipedia.org/wiki/AES-GCM-SIV
oh_sigh 5y ago
My fear of using something like this would be to accidentally mess up the format (<PRIVATE, <<PIRVATE, etc), and accidentally commit my private stuff in plaintext. Similarly, if I misconfigure my .gitattributes, I might try to use this on a file which the privatizer program doesn't even get run over.
- higgins 5y ago
  ooo, good call out. thanks! currently i only have one warning in place if I forget to close there heredoc: https://github.com/higgins/privatize/blob/main/index.js#L168 https://github.com/higgins/privatize/blob/main/index.js#L168 but could add some more error checking. added to TODOs: https://github.com/higgins/privatize/commit/9696f305768978988baff58b2732ba6b35404e3c https://github.com/higgins/privatize/commit/9696f30576897898...
kkfx 5y ago
Nice, but not new :-) See Emacs/org-mode org-encrypt-entry and org-decrypt-entry, it's easy to encrypt and decrypt via GNUPG text under a heading + the benefit of org-mode outlining witch dramatically improve readability.
- higgins 5y ago
  Interesting! It is a bit different feature set as the encryption is hooked onto file save and not git committing. I'm sure I could find some uses for it though. Thanks!
  - kkfx 5y ago
    You can hook as you wish, manually, automatically etc in Emacs it's just a matter of an (add-hook call, eventually at file-level with a # Local variables: call That's the big plus of classic systems compared to Unix and all others: anything is a function, anything can be called everywhere because the REPL itself is not a CLI but a GUI, a framework. That's valid for SmallTalk workstations from Xerox to Emacs :-)

38 points by higgins 5y ago

higgins 5y ago

FYI: this was inspired by the great `git-crypt` (https://github.com/AGWA/git-crypt https://github.com/AGWA/git-crypt) More on my motivations here: https://encapsulate.me/writing/Privatize.html https://encapsulate.me/writing/Privatize.html

mdaniel 5y ago

Yikes, do all filters do that? https://github.com/higgins/privatize/blob/0.1.1/index.js#L98 https://github.com/higgins/privatize/blob/0.1.1/index.js#L98 Also, it's 2022 and I still have to remind people not to commit binary assets into git repos, as the repo will grow without bound: https://github.com/higgins/privatize/tree/main/release https://github.com/higgins/privatize/tree/main/release That's the very problem that GitHub Releases were designed to address, and has the extra awesome benefit of using (currently) AWS S3 for distribution, which is almost certainly going to be faster and place less load upon github.com than .../raw/main/release/whatever.exe Also, while the Brew tap indicates your code is ISC, there is no license file in your repo: https://github.com/higgins/homebrew-privatize/blob/main/Formula/privatize.rb#L8 https://github.com/higgins/homebrew-privatize/blob/main/Form...

deleted 5y ago
[deleted]
higgins 5y ago
Thanks for your thorough feedback! Much appreciated Added some more TODOs to address your comments. :) FWIW, the `git reset --hard` is only on unlocking a repo that was previously privatize'd. Still, not a great FTUX. Will update.

nnf 5y ago

This is a neat idea. I wrote something similar recently but with the aim of encrypting sensitive values (like API keys) in YAML config files, for similar reasons — so people without the key can see most of the config but not the secret parts. The script is then used by an automated deployment process to decrypt the sensitive values when the config file is moved into place.

higgins 5y ago
Nice! You can use it as a cli tool too: ``` # Create a stand-alone symmetric key for use outside of git privatize create-key symmetric.key # Encrypt a file and pipe to stdout cat someFileToPartiallyEncrypt | privatize encrypt symmetric.key # Decrypt a file and pipe to stdout cat someFileToPartiallyDecrypt | privatize decrypt symmetric.key ``` Open to pull requests if you want to adapt to your use cases too

klyrs 5y ago

Do you check that the sentinel string PRIVATE does not occur in the encrypted data?

higgins 5y ago
hey, thanks! added to my TODOs: https://github.com/higgins/privatize/commit/53f62483f943112eb7bd2e7bf7feaa1f8fc03c92 https://github.com/higgins/privatize/commit/53f62483f943112e...
- klyrs 5y ago
  It's one of those alarm bells that got installed a very long time ago in my career... makes for some really exciting bugs.

deleted 5y ago

[deleted]

amenghra 5y ago

I might not have properly understood this tool, but is the encryption key and iv getting reused? If so, it's usually quite unsafe to reuse IVs (but consult a cryptographer, YMMV). Feels weird to stuff the IV with the key, it should live with the ciphertext. (I only glanced at the code for a few minutes, so I could be wrong: https://github.com/higgins/privatize/blob/0.1.1/index.js#L73 https://github.com/higgins/privatize/blob/0.1.1/index.js#L73)

higgins 5y ago
Thanks for taking a look! Yeah, I should cleanup the wording in here a bit: The IV is a sha1 hmac of the content to be encrypted. The 16B that are stored with the key is used to initialize the sha1 hmac: https://github.com/higgins/privatize/blob/0.1.1/index.js#L20-L26 https://github.com/higgins/privatize/blob/0.1.1/index.js#L20...
- deathanatos 5y ago
  Given that you store the IV alongside the ciphertext anyways, I'm not seeing what advantage this has over just choosing the IV at random.
  - re 5y ago
    This appears to be a design decision borrowed/inherited from git-crypt: https://github.com/AGWA/git-crypt#security https://github.com/AGWA/git-crypt#security The gain is that encrypted output is deterministic (which git seems to require), with the trade-off of leaking some information (particularly that multiple encrypted ciphertexts are identical). I'm not really a crypto expert, but it seems like a better alternative might be to use AES-GCM-SIV. It is designed to allow for nonce reuse, and has the advantage of being authenticated encryption. https://en.wikipedia.org/wiki/AES-GCM-SIV https://en.wikipedia.org/wiki/AES-GCM-SIV

oh_sigh 5y ago

My fear of using something like this would be to accidentally mess up the format (<PRIVATE, <<PIRVATE, etc), and accidentally commit my private stuff in plaintext. Similarly, if I misconfigure my .gitattributes, I might try to use this on a file which the privatizer program doesn't even get run over.

higgins 5y ago
ooo, good call out. thanks! currently i only have one warning in place if I forget to close there heredoc: https://github.com/higgins/privatize/blob/main/index.js#L168 https://github.com/higgins/privatize/blob/main/index.js#L168 but could add some more error checking. added to TODOs: https://github.com/higgins/privatize/commit/9696f305768978988baff58b2732ba6b35404e3c https://github.com/higgins/privatize/commit/9696f30576897898...

kkfx 5y ago

Nice, but not new :-) See Emacs/org-mode org-encrypt-entry and org-decrypt-entry, it's easy to encrypt and decrypt via GNUPG text under a heading + the benefit of org-mode outlining witch dramatically improve readability.

higgins 5y ago
Interesting! It is a bit different feature set as the encryption is hooked onto file save and not git committing. I'm sure I could find some uses for it though. Thanks!
- kkfx 5y ago
  You can hook as you wish, manually, automatically etc in Emacs it's just a matter of an (add-hook call, eventually at file-level with a # Local variables: call That's the big plus of classic systems compared to Unix and all others: anything is a function, anything can be called everywhere because the REPL itself is not a CLI but a GUI, a framework. That's valid for SmallTalk workstations from Xerox to Emacs :-)