6 ms·

SaltStack has a long history of home brew protocols, internally written encryption, security issues, and bugs. I'm not surprised they would end up being used as

SaltStack has a long history of home brew protocols, internally written encryption, security issues, and bugs. I'm not surprised they would end up being used as a vector for attacks.

riffic 6y ago
Something about Salt has always rubbed me the wrong way. I'd rather stick with Puppet, Chef, or CF Engine if I was considering this tool.
- xenophonf 6y ago
  I'm all about IT automation and like the tech a lot, but after running SaltStack for 5 years, I decided to transition to plain old written documentation. When you consider everything that underlies the "infrastructure as code" tech stack, it ends up being an extremely steep learning curve. I was very productive with SaltStack but turned into the only person who could write or edit our deployment scripts. It was a bad situation. Now, nine months after dropping SaltStack, my colleagues are editing the docs I've created, and more importantly, they're also writing their own. That's a huge win. What I've lost in terms of personal productivity, I've regained in terms of wider participation in our standardization and documentation efforts. I might slowly re-introduce IT automation technologies, maybe something more popular like Ansible or Docker, but only after making sure the rest of my team has a solid understanding of general IT automation concepts. I think we're on the right track. I had a colleague today ask me about what Python programming certification they should get, so I pointed them toward Google's Python-based IT automation course on Coursera. I supported another colleague's efforts to create a library of standard server images based on our internal deployment standards for one of our private clouds, and I'm encouraging them to expand that work to our other data centers. And so on. The team as a whole works better together, so that's where I'm staying focused.
  - chucky_z 6y ago
    So you went from full automation to... Copying and pasting by hand?
  - rexarex 6y ago
    Are you seriously telling me that you went from infrastructure as code to doing things by hand!?
    - xenophonf 6y ago
      We were always doing things by hand. I was the only one doing any kind of IT automation, and I failed to account for the learning curve of my tooling when trying to expand its use. At least now, the entire team—not just me—is doing a better job of documenting what they're doing by hand. The reality is that devops toolchains are really complex. For example, to use SaltStack the way I had things set up, it meant learning: - SaltStack's domain-specific programming language, which amounts to writing Python in YAML - their macro preprocessor, Jinja, which has completely different syntax and semantics than their DSL - a programming text editor that supports YAML and Jinja - Git and GitHub - secrets management (and there were huge risks here) - the general concepts of IT automation and infrastructure-as-code Learning (never mind teaching) that tech stack is really hard. For example, the Google IT Automation with Python certification on Coursera is an 8-month-long course. That's just one bullet point on the above list, and that list doesn't include all the things I wanted to do on top of SaltStack, namely the continuous integration/continuous testing stuff, which would involve learning: - the Chef InSpec DSL, which is based on Ruby - branching and tagging - release engineering - test-driven development as a software engineering methodology - software engineering methodology in general We were struggling with even just writing good documentation, but there I was asking everyone to write good code. That's orders of magnitude harder. It was too much, like asking somebody to run a marathon without any training. I don't care how fit you are. That's just not going to happen.
dilandau 6y ago
You're getting downvoted by people who don't know shit. This is 100% true. It's a cobbled-together eldritch nightmare. A monolithic turd.
DeepYogurt 6y ago
See also the F-secure timeline where the salt team's gpg key had been expired for years, salt had a lack of clear communication, and general sloth in response https://labs.f-secure.com/advisories/saltstack-authorization-bypass/ https://labs.f-secure.com/advisories/saltstack-authorization...
shockinglytrue 6y ago
Attributing this just to Salt is deeply unfair. Ansible has had many similar issues over time, also homebrew encryption etc.
- lima 6y ago
  Ansible is relying on SSH for security and does not use any homebrew crypto or protocols by default.
  - shockinglytrue 6y ago
    ansible-vault is homebrew crypto
    - VWWHFSfQ 6y ago
      it uses the standard cryptography or pycrypto backends as everything else
    - jaden 6y ago
      I don't understand this claim. ansible-vault uses AES256 which is anything but homebrew.
      - brians 6y ago
        Salt uses AES too. The problem is it puts together standard primitives in homebrew protocols. Cryptographic protocol design is as likely to mess your system up as cryptographic primitive design.
      - shockinglytrue 6y ago
        This is why like me, you're not a cryptographer. AES256 is a cipher, its one component of a cryptosystem. Analysing cryptosystems is a complex area that does not involve vetting software for buzzwords. I can take AES256 and make it output the image here: https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_codebook_(ECB) https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation... That's not supposed to happen. The part where homebrew diverges from cryptography is that the former involves engineers like us connecting buzzwords together to produce images like the Wikipedia article, the latter involves complex math and rigorous peer review.
dpedu 6y ago
What "home brew" protocols and encryption does Salt use?
jlgaddis 6y ago
Exhibit A: CVE-2013-2228. As described inthe relevant entry [0] on Debian's Security Tracker: > SaltStack RSA Key Generation allows remote users to decrypt communications The fix [1]: - gen = RSA.gen_key(keysize, 1, callback=lambda x, y, z: None) + gen = RSA.gen_key(keysize, 65537, callback=lambda x, y, z: None) --- [0]: https://security-tracker.debian.org/tracker/CVE-2013-2228 https://security-tracker.debian.org/tracker/CVE-2013-2228 [1]: https://github.com/saltstack/salt/commit/e8ce66cf688b43aeb3e716e78b1af3a08e9940e3 https://github.com/saltstack/salt/commit/e8ce66cf688b43aeb3e...
- sushisource 6y ago
  Jesus that's some rookie stuff right there
  - lima 6y ago
    They also built their own low-level network protocol based on 0mq. Back when I tried it, it had constant keepalive issues behind NAT and would randomly lose messages (that was in 2014, probably better now).
    - xenophonf 6y ago
      It wasn't any better as of last year, and its constant connectivity issues were part of the reason I abandoned it.
  - nemothekid 6y ago
    This is rookie stuff, but it’s a recurring security issue where crypto libraries have APIs littered with footguns. The “lambda x, y, z: None” just strikes me as cargo culting copy paste code - I’m willing to bet there is other software with the same exploit as they all copied the same answer on StackOverflow
- correct_horse 6y ago
  The second parameter in the gen_key function is the RSA public exponent. I believe this means the ciphertext will be the same as the padded plaintext. Here is the documentation from the RSA.gen_key() function. def gen_key(bits, e, callback=keygen_callback): # type: (int, int, Callable) -> RSA """ Generate an RSA key pair. :param bits: Key length, in bits. :param e: The RSA public exponent. :param callback: A Python callable object that is invoked during key generation; its usual purpose is to provide visual feedback. The default callback is keygen_callback. :return: M2Crypto.RSA.RSA object. """ https://gitlab.com/m2crypto/m2crypto/-/blob/master/M2Crypto/RSA.py#L346 https://gitlab.com/m2crypto/m2crypto/-/blob/master/M2Crypto/...

SaltStack has a long history of home brew protocols, internally written encryption, security issues, and bugs. I'm not surprised they would end up being used as a vector for attacks.

riffic 6y ago

Something about Salt has always rubbed me the wrong way. I'd rather stick with Puppet, Chef, or CF Engine if I was considering this tool.

xenophonf 6y ago
I'm all about IT automation and like the tech a lot, but after running SaltStack for 5 years, I decided to transition to plain old written documentation. When you consider everything that underlies the "infrastructure as code" tech stack, it ends up being an extremely steep learning curve. I was very productive with SaltStack but turned into the only person who could write or edit our deployment scripts. It was a bad situation. Now, nine months after dropping SaltStack, my colleagues are editing the docs I've created, and more importantly, they're also writing their own. That's a huge win. What I've lost in terms of personal productivity, I've regained in terms of wider participation in our standardization and documentation efforts. I might slowly re-introduce IT automation technologies, maybe something more popular like Ansible or Docker, but only after making sure the rest of my team has a solid understanding of general IT automation concepts. I think we're on the right track. I had a colleague today ask me about what Python programming certification they should get, so I pointed them toward Google's Python-based IT automation course on Coursera. I supported another colleague's efforts to create a library of standard server images based on our internal deployment standards for one of our private clouds, and I'm encouraging them to expand that work to our other data centers. And so on. The team as a whole works better together, so that's where I'm staying focused.
- chucky_z 6y ago
  So you went from full automation to... Copying and pasting by hand?
- rexarex 6y ago
  Are you seriously telling me that you went from infrastructure as code to doing things by hand!?
  - xenophonf 6y ago
    We were always doing things by hand. I was the only one doing any kind of IT automation, and I failed to account for the learning curve of my tooling when trying to expand its use. At least now, the entire team—not just me—is doing a better job of documenting what they're doing by hand. The reality is that devops toolchains are really complex. For example, to use SaltStack the way I had things set up, it meant learning: - SaltStack's domain-specific programming language, which amounts to writing Python in YAML - their macro preprocessor, Jinja, which has completely different syntax and semantics than their DSL - a programming text editor that supports YAML and Jinja - Git and GitHub - secrets management (and there were huge risks here) - the general concepts of IT automation and infrastructure-as-code Learning (never mind teaching) that tech stack is really hard. For example, the Google IT Automation with Python certification on Coursera is an 8-month-long course. That's just one bullet point on the above list, and that list doesn't include all the things I wanted to do on top of SaltStack, namely the continuous integration/continuous testing stuff, which would involve learning: - the Chef InSpec DSL, which is based on Ruby - branching and tagging - release engineering - test-driven development as a software engineering methodology - software engineering methodology in general We were struggling with even just writing good documentation, but there I was asking everyone to write good code. That's orders of magnitude harder. It was too much, like asking somebody to run a marathon without any training. I don't care how fit you are. That's just not going to happen.

dilandau 6y ago

You're getting downvoted by people who don't know shit. This is 100% true. It's a cobbled-together eldritch nightmare. A monolithic turd.

DeepYogurt 6y ago

See also the F-secure timeline where the salt team's gpg key had been expired for years, salt had a lack of clear communication, and general sloth in response https://labs.f-secure.com/advisories/saltstack-authorization-bypass/ https://labs.f-secure.com/advisories/saltstack-authorization...

shockinglytrue 6y ago

Attributing this just to Salt is deeply unfair. Ansible has had many similar issues over time, also homebrew encryption etc.

lima 6y ago
Ansible is relying on SSH for security and does not use any homebrew crypto or protocols by default.
- shockinglytrue 6y ago
  ansible-vault is homebrew crypto
  - VWWHFSfQ 6y ago
    it uses the standard cryptography or pycrypto backends as everything else
  - jaden 6y ago
    I don't understand this claim. ansible-vault uses AES256 which is anything but homebrew.
    - brians 6y ago
      Salt uses AES too. The problem is it puts together standard primitives in homebrew protocols. Cryptographic protocol design is as likely to mess your system up as cryptographic primitive design.
    - shockinglytrue 6y ago
      This is why like me, you're not a cryptographer. AES256 is a cipher, its one component of a cryptosystem. Analysing cryptosystems is a complex area that does not involve vetting software for buzzwords. I can take AES256 and make it output the image here: https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_codebook_(ECB) https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation... That's not supposed to happen. The part where homebrew diverges from cryptography is that the former involves engineers like us connecting buzzwords together to produce images like the Wikipedia article, the latter involves complex math and rigorous peer review.

dpedu 6y ago

What "home brew" protocols and encryption does Salt use?

jlgaddis 6y ago

Exhibit A: CVE-2013-2228. As described inthe relevant entry [0] on Debian's Security Tracker: > SaltStack RSA Key Generation allows remote users to decrypt communications The fix [1]: - gen = RSA.gen_key(keysize, 1, callback=lambda x, y, z: None) + gen = RSA.gen_key(keysize, 65537, callback=lambda x, y, z: None) --- [0]: https://security-tracker.debian.org/tracker/CVE-2013-2228 https://security-tracker.debian.org/tracker/CVE-2013-2228 [1]: https://github.com/saltstack/salt/commit/e8ce66cf688b43aeb3e716e78b1af3a08e9940e3 https://github.com/saltstack/salt/commit/e8ce66cf688b43aeb3e...

sushisource 6y ago
Jesus that's some rookie stuff right there
- lima 6y ago
  They also built their own low-level network protocol based on 0mq. Back when I tried it, it had constant keepalive issues behind NAT and would randomly lose messages (that was in 2014, probably better now).
  - xenophonf 6y ago
    It wasn't any better as of last year, and its constant connectivity issues were part of the reason I abandoned it.
- nemothekid 6y ago
  This is rookie stuff, but it’s a recurring security issue where crypto libraries have APIs littered with footguns. The “lambda x, y, z: None” just strikes me as cargo culting copy paste code - I’m willing to bet there is other software with the same exploit as they all copied the same answer on StackOverflow
correct_horse 6y ago
The second parameter in the gen_key function is the RSA public exponent. I believe this means the ciphertext will be the same as the padded plaintext. Here is the documentation from the RSA.gen_key() function. def gen_key(bits, e, callback=keygen_callback): # type: (int, int, Callable) -> RSA """ Generate an RSA key pair. :param bits: Key length, in bits. :param e: The RSA public exponent. :param callback: A Python callable object that is invoked during key generation; its usual purpose is to provide visual feedback. The default callback is keygen_callback. :return: M2Crypto.RSA.RSA object. """ https://gitlab.com/m2crypto/m2crypto/-/blob/master/M2Crypto/RSA.py#L346 https://gitlab.com/m2crypto/m2crypto/-/blob/master/M2Crypto/...