4 ms·

We did take a look at D(TLS) when implementing Serf and the associated encryption mechanism. D(TLS) is much better suited when you are doing point-to-point dupl

by armon 12y ago

We did take a look at D(TLS) when implementing Serf and the associated encryption mechanism. D(TLS) is much better suited when you are doing point-to-point duplex communication. The gossip algorithm instead is doing peer-to-peer (N-to-N instead of 1-to-1) in a half-duplex model. Using DTLS would've been too heavy for our use case. We did develop the algorithm in the open (see this gist: https://gist.github.com/armon/7159161 https://gist.github.com/armon/7159161), and made a call on the community to provide feedback to help improve the design of the system. The crypto system used is actually pretty vanilla based on AES-GCM. We certainly didn't attempt to invent our crypto primitives, and instead stuck to the best practices around the most modern systems.

jimktrains2 12y ago
> The gossip algorithm instead is doing peer-to-peer (N-to-N instead of 1-to-1) in a half-duplex model. Unless you mean you're doing multicast, you're still doing 1-to-1 connections. > We did develop the algorithm in the open and made a call on the community to provide feedback to help improve the design of the system. I wouldn't trust myself, let alone many unknown people who are just volunteering their time and have no known credentials as a cryptographer. > instead stuck to the best practices around the most modern systems. Except for the whole "don't role your own crypto system" one. There are just so many things that can go wrong and when crypto fails it can do so very quietly. Additionally, if you ever want anyone to interoperate with you it'll just be a PITA for them, and depending on who it is, a PITA for you. The overhead of a proper protocol like (D)TLS isn't that much. > "On our production frontend machines, SSL/TLS accounts for less than 1% of the CPU load, less than 10 KB of memory per connection and less than 2% of network overhead. Many people believe that SSL/TLS takes a lot of CPU time and we hope the preceding numbers will help to dispel that." - Adam Langley, Google
- armon 12y ago
  You are right, the communication is still unicast in nature. I should clarify to say that there isn't a persistent 1-to-1 communication, the nodes we gossip with are randomly selected on each interval. There is no connection or session establishment between peers. I guess it depends on your definition of roll your own. We didn't invent AES-GCM or implement it. We are using the implementation shipped with the Golang stdlib.
  - bascule 12y ago
    Nobody is questioning whether AES-GCM is a good algorithm or not, however you are using AES-GCM as part of a hand-rolled transport encryption protocol, and this is what's worrisome. Designing a transport encryption protocol is a difficult endeavor, and it seems you have skipped most of the steps (e.g. replay attack prevention) but suggest that it's irrelevant because other parts of the protocol provide security (e.g. the SWIM state machine). This makes your protocol difficult to audit: someone concerned about potential attacks can't just look at your protocol in isolation, but has to factor the underlying protocol state machine into the security of your transport encryption protocol.
  - jimktrains2 12y ago
    > There is no connection or session establishment between peers. OK, so I'm not understanding why isn't a 1-to-1 message, nor why DTLS isn't an option here. > I guess it depends on your definition of roll your own. We didn't invent AES-GCM or implement it. We are using the implementation shipped with the Golang stdlib. You need to read better. I didn't say "crypto primitives", I've said "crypto _system_". That includes everything, including primitives, key management, authentication, replay (which means your application protocol is now part of the crypto system, not a good sign), field concatenation for singing/hashing, &c. The most important thing is that when your improvised system fails, you will more-than-likely never know and it'll never cause any errors.

by armon 12y ago

jimktrains2 12y ago

> The gossip algorithm instead is doing peer-to-peer (N-to-N instead of 1-to-1) in a half-duplex model. Unless you mean you're doing multicast, you're still doing 1-to-1 connections. > We did develop the algorithm in the open and made a call on the community to provide feedback to help improve the design of the system. I wouldn't trust myself, let alone many unknown people who are just volunteering their time and have no known credentials as a cryptographer. > instead stuck to the best practices around the most modern systems. Except for the whole "don't role your own crypto system" one. There are just so many things that can go wrong and when crypto fails it can do so very quietly. Additionally, if you ever want anyone to interoperate with you it'll just be a PITA for them, and depending on who it is, a PITA for you. The overhead of a proper protocol like (D)TLS isn't that much. > "On our production frontend machines, SSL/TLS accounts for less than 1% of the CPU load, less than 10 KB of memory per connection and less than 2% of network overhead. Many people believe that SSL/TLS takes a lot of CPU time and we hope the preceding numbers will help to dispel that." - Adam Langley, Google

armon 12y ago
You are right, the communication is still unicast in nature. I should clarify to say that there isn't a persistent 1-to-1 communication, the nodes we gossip with are randomly selected on each interval. There is no connection or session establishment between peers. I guess it depends on your definition of roll your own. We didn't invent AES-GCM or implement it. We are using the implementation shipped with the Golang stdlib.
- bascule 12y ago
  Nobody is questioning whether AES-GCM is a good algorithm or not, however you are using AES-GCM as part of a hand-rolled transport encryption protocol, and this is what's worrisome. Designing a transport encryption protocol is a difficult endeavor, and it seems you have skipped most of the steps (e.g. replay attack prevention) but suggest that it's irrelevant because other parts of the protocol provide security (e.g. the SWIM state machine). This makes your protocol difficult to audit: someone concerned about potential attacks can't just look at your protocol in isolation, but has to factor the underlying protocol state machine into the security of your transport encryption protocol.
- jimktrains2 12y ago
  > There is no connection or session establishment between peers. OK, so I'm not understanding why isn't a 1-to-1 message, nor why DTLS isn't an option here. > I guess it depends on your definition of roll your own. We didn't invent AES-GCM or implement it. We are using the implementation shipped with the Golang stdlib. You need to read better. I didn't say "crypto primitives", I've said "crypto _system_". That includes everything, including primitives, key management, authentication, replay (which means your application protocol is now part of the crypto system, not a good sign), field concatenation for singing/hashing, &c. The most important thing is that when your improvised system fails, you will more-than-likely never know and it'll never cause any errors.