5 ms·

Tumblr Opensources Kubernetes Utilities

72 points by byxorna 8y ago

AndyNemmity 8y ago
That config injector seems super useful. I immediately start to wonder what subset of config options I can inject, and actually get the application to update.
- byxorna 8y ago
  K8s will update mounted ConfigMaps "live" in your pod, by updating a symlink. If your application is using inotify or file polling to watch the ConfigMap as a volume, you can reconfigure your app when changes happen (or signal your liveness probe you should be terminated to reload latest config).
  - wetpaste 8y ago
    This partly solves a big problem for me, which was how do I keep those configmaps up to date for a bunch of customers with a legacy application that writes UI config updates to a local xml config file. I wanted that file to live inside of configmaps but I have this problem. I already found a bash script that does inotify pushes to git (called git-watch), but I didn't have the other half of this equation. The missing half was getting those new config changes applied back into the configmap.
throwaway_129 8y ago
Not a cryptographer, but a few things irked me about the secret projector encryption features: 1. Calls it AES-CBC, but it's using AES-GCM https://github.com/tumblr/k8s-secret-projector/blob/master/pkg/encryption/cbc/block.go#L88 https://github.com/tumblr/k8s-secret-projector/blob/master/p... 2. Encryption key generation (edited: from a password) uses a hash function rather than using a KDF. Short hashes are padded (takes first N bytes required to hit max length, appends to end) or trimmed to the appropriate length. https://github.com/tumblr/k8s-secret-projector/blob/master/pkg/encryption/cbc/key.go#L43 https://github.com/tumblr/k8s-secret-projector/blob/master/p...
- byxorna 8y ago
  Fair points :) Do you have any examples of using a KDF that jives with the bitlength requirements of a given cipher?
  - throwaway_129 8y ago
    https://en.wikipedia.org/wiki/PBKDF2 https://en.wikipedia.org/wiki/PBKDF2 Again, I'm not a cryptographer, and PBKDF2 is just an example. I would try to consult someone with a heavier background in the field. I've used PBKDF2 and Scrypt before, Argon2 is the newest, but also therefor least battle-tested. https://godoc.org/golang.org/x/crypto/pbkdf2 https://godoc.org/golang.org/x/crypto/pbkdf2 https://godoc.org/golang.org/x/crypto/argon2 https://godoc.org/golang.org/x/crypto/argon2 https://godoc.org/golang.org/x/crypto/scrypt https://godoc.org/golang.org/x/crypto/scrypt
  - leetbulb 8y ago
    There's an RFC for that: https://tools.ietf.org/html/rfc5869 https://tools.ietf.org/html/rfc5869 Doesn't look like the Go API[1] allows you to specify an output key material length, so you'd just truncate or iterate until you get however much material you need (The HKDF Wiki implementation does this[2]). 1: https://godoc.org/golang.org/x/crypto/hkdf https://godoc.org/golang.org/x/crypto/hkdf 2: https://en.wikipedia.org/wiki/HKDF#Example:_Python_implementation https://en.wikipedia.org/wiki/HKDF#Example:_Python_implement...
    - throwaway_129 8y ago
      You don't want to generate key material from a password with hkdf. From the RFC: On the other hand, it is anticipated that some applications will not be able to use HKDF "as-is" due to specific operational requirements, or will be able to use it but without the full benefits of the scheme. One significant example is the derivation of cryptographic keys from a source of low entropy, such as a user's password. The extract step in HKDF can concentrate existing entropy but cannot amplify entropy. In the case of password-based KDFs, a main goal is to slow down dictionary attacks using two ingredients: a salt value,and the intentional slowing of the key derivation computation. HKDF naturally accommodates the use of salt; however, a slowing down mechanism is not part of this specification. Applications interested in a password-based KDF should consider whether, for example, [PKCS5] meets their needs better than HKDF.
mikesun 8y ago
I hope Regulator has been decommissioned ;)
- byxorna 8y ago
  How did you see my Q1 TODO list?
  - heinstrom 8y ago
    ;)
p1necone 8y ago
My tinfoil hat is telling me that the engineers are pushing to open source as much as possible before the company inevitably goes under due to the recent banning of NSFW content.
- excalibur 8y ago
  That's actually not coming from the hat, you're receiving signals of the site's impending collapse through the foil. You way want to consider adding a few more layers.
- cmelbye 8y ago
  Next week: “Tumblr open sources nudity classifier”
solarkraft 8y ago
I miss Tumblr.

72 points by byxorna 8y ago

That config injector seems super useful. I immediately start to wonder what subset of config options I can inject, and actually get the application to update.

byxorna 8y ago
K8s will update mounted ConfigMaps "live" in your pod, by updating a symlink. If your application is using inotify or file polling to watch the ConfigMap as a volume, you can reconfigure your app when changes happen (or signal your liveness probe you should be terminated to reload latest config).
- wetpaste 8y ago
  This partly solves a big problem for me, which was how do I keep those configmaps up to date for a bunch of customers with a legacy application that writes UI config updates to a local xml config file. I wanted that file to live inside of configmaps but I have this problem. I already found a bash script that does inotify pushes to git (called git-watch), but I didn't have the other half of this equation. The missing half was getting those new config changes applied back into the configmap.

throwaway_129 8y ago

Not a cryptographer, but a few things irked me about the secret projector encryption features: 1. Calls it AES-CBC, but it's using AES-GCM https://github.com/tumblr/k8s-secret-projector/blob/master/pkg/encryption/cbc/block.go#L88 https://github.com/tumblr/k8s-secret-projector/blob/master/p... 2. Encryption key generation (edited: from a password) uses a hash function rather than using a KDF. Short hashes are padded (takes first N bytes required to hit max length, appends to end) or trimmed to the appropriate length. https://github.com/tumblr/k8s-secret-projector/blob/master/pkg/encryption/cbc/key.go#L43 https://github.com/tumblr/k8s-secret-projector/blob/master/p...

byxorna 8y ago
Fair points :) Do you have any examples of using a KDF that jives with the bitlength requirements of a given cipher?
- throwaway_129 8y ago
  https://en.wikipedia.org/wiki/PBKDF2 https://en.wikipedia.org/wiki/PBKDF2 Again, I'm not a cryptographer, and PBKDF2 is just an example. I would try to consult someone with a heavier background in the field. I've used PBKDF2 and Scrypt before, Argon2 is the newest, but also therefor least battle-tested. https://godoc.org/golang.org/x/crypto/pbkdf2 https://godoc.org/golang.org/x/crypto/pbkdf2 https://godoc.org/golang.org/x/crypto/argon2 https://godoc.org/golang.org/x/crypto/argon2 https://godoc.org/golang.org/x/crypto/scrypt https://godoc.org/golang.org/x/crypto/scrypt
- leetbulb 8y ago
  There's an RFC for that: https://tools.ietf.org/html/rfc5869 https://tools.ietf.org/html/rfc5869 Doesn't look like the Go API[1] allows you to specify an output key material length, so you'd just truncate or iterate until you get however much material you need (The HKDF Wiki implementation does this[2]). 1: https://godoc.org/golang.org/x/crypto/hkdf https://godoc.org/golang.org/x/crypto/hkdf 2: https://en.wikipedia.org/wiki/HKDF#Example:_Python_implementation https://en.wikipedia.org/wiki/HKDF#Example:_Python_implement...
  - throwaway_129 8y ago
    You don't want to generate key material from a password with hkdf. From the RFC: On the other hand, it is anticipated that some applications will not be able to use HKDF "as-is" due to specific operational requirements, or will be able to use it but without the full benefits of the scheme. One significant example is the derivation of cryptographic keys from a source of low entropy, such as a user's password. The extract step in HKDF can concentrate existing entropy but cannot amplify entropy. In the case of password-based KDFs, a main goal is to slow down dictionary attacks using two ingredients: a salt value,and the intentional slowing of the key derivation computation. HKDF naturally accommodates the use of salt; however, a slowing down mechanism is not part of this specification. Applications interested in a password-based KDF should consider whether, for example, [PKCS5] meets their needs better than HKDF.

mikesun 8y ago

I hope Regulator has been decommissioned ;)

byxorna 8y ago
How did you see my Q1 TODO list?
- heinstrom 8y ago
  ;)

p1necone 8y ago

My tinfoil hat is telling me that the engineers are pushing to open source as much as possible before the company inevitably goes under due to the recent banning of NSFW content.

excalibur 8y ago
That's actually not coming from the hat, you're receiving signals of the site's impending collapse through the foil. You way want to consider adding a few more layers.
cmelbye 8y ago
Next week: “Tumblr open sources nudity classifier”

solarkraft 8y ago

I miss Tumblr.