5 ms·

The real competitor, which the article doesn't talk about is https://www.zerotier.com/ https://www.zerotier.com/ . Is is also open-source and have been availabl

The real competitor, which the article doesn't talk about is https://www.zerotier.com/ https://www.zerotier.com/ . Is is also open-source and have been available on macOS, iPhone, Android, Windows on top of Linux for many months now. It is easy to install and use. Deals with punching NAT and does smart routing when both devices are on the same network. I don't know how well it compares in terms of performance and crypto, it would be nice to know.

tptacek 8y ago
Where is ZeroTier's protocol documented? All I could find was a document that said it was custom, and "like IPSEC". What could ZeroTier do as a VPN protocol that would make it better than WireGuard, which has a pedigreed protocol that, in its WireGuard incarnation, has had multiple rounds of formal validation? I wrote the article we're commenting on. Why would I take the time to talk about something like ZeroTier? WireGuard solves the problem I'm talking about decisively. I'm prepared to hear something about ZeroTier that makes it interesting in the WireGuard setting, but I am not interested in a long list of non-VPN non-access-tunnel features that ZeroTier might have that WireGuard lacks, because lacking random features is exactly the point of WireGuard.
- comex 8y ago
  It may not be interesting to you. To me, and probably a lot of people, the ability to communicate directly between two hosts each behind a NAT (using UDP hole punching) is a big advantage for a VPN, since, compared to forwarding through a central server, it can be faster and lower latency, and it also means you don’t have to pay for that server. Edit: As to why it’s relevant to the “WireGuard setting”, that depends on your definition of the WireGuard setting, but: (1) it’s easy to find examples of people trying to use WireGuard for peer-to-peer communication, the use case ZeroTier solves better; and (2) ZeroTier shares WireGuard’s property of being stateless/connectionless, a significant convenience advantage over traditional VPN protocols.
  - tptacek 8y ago
    I think I could have worded that better. By "in the WireGuard setting" I mean "doing the job most startups will use WireGuard to do", which is, manage remote access to internal resources in the cloud. I don't think overlay networks are pointless; I am enthusiastic about them. But for the remote access problem, what I want and what I'm writing about is streamlined VPN software, not a new overlay.
    - comex 8y ago
      Fair enough, although… you know more about this use case than me, but wouldn’t you generally have a large number of servers in the cloud you want to provide remote access to? So if you’re using WireGuard, either you have to configure each client to talk to each server it needs access to – which could be good for security, but sounds like a pain to set up – or you have a central server that serves as a hub for all the VPN traffic. In your blog post you said you just had to “point the client at the server”, so I’m guessing you’re envisioning the latter. But then wouldn’t an overlay network work better in many cases? For example, if you have cloud servers in different geographical locations, and as the developer you’re near one of those locations but far from the VPN server, you’d have lower latency connecting directly…
      - tptacek 8y ago
        I'm imagining simply replacing OpenVPN servers with WireGuard servers, retaining the VPN gateway model; one WireGuard instance acts as a routing gateway for any number of servers.
- johntash 8y ago
  The only real complaint I would have over WireGuard is that with zerotier, I only need to connect to one server and then can talk to any servers on the same "network" as long as the acl permits it. The other servers are discovered automatically. With WireGuard, I'm pretty sure I would have to set up multiple peers and custom routes. Although I'm assuming you could run BGP on top of the wg interface and that would solve some of it. That said, I'd much rather something the simplicity of wireguard or tinc because it doesn't require a server who's only purpose is to manage clients. I know it's not specifically related to the protocol either uses, but I think people would consider that as a drawback when comparing the two. If you're coming from openvpn/ipsec, then setting up any sort of mesh/routed network would be a similar process. I don't think wireguard would need to implement functionality like that either, I think it would be better if something like tinc used wireguard for the plumbing as someone else mentioned in the comments.
- api 8y ago
  No RFC-type document yet but quite a bit of commenting here: https://github.com/zerotier/ZeroTierOne/blob/master/node/Packet.hpp https://github.com/zerotier/ZeroTierOne/blob/master/node/Pac... There is also a higher-level description of the entire system here: https://www.zerotier.com/manual.shtml https://www.zerotier.com/manual.shtml ZeroTier solves a broader set of problems than WireGuard and has SDN-type network flow rules, security tap capability, and is a full L2 emulation layer with support for multicast. From what I've seen the WireGuard protocols are excellent and the implementation is very nice and clean. A core ZeroTier implementation to support a simple L3 VPN use case could be as short as WireGuard but that would be without all the rules engine, multicast, etc. stuff.
  - tptacek 8y ago
    Just the .cpp code, not counting the .hpp code (which is full of code, including lots of memcpy's) in the node/ subdirectory of this repo is something like 4x the entire size of Linux WireGuard. The crypto protocol is undocumented, but the maintainer is playing games with things like the Salsa round count (which is fine, but I mean, speaks to a mindset). I might poke at it for sport at some point, but I don't see how I could ever recommend it.
    - api 8y ago
      I am the primary maintainer. :) A minimal implementation could be much shorter, but the ref implementation is not minimal and includes a lot of features that go far beyond the scope of WireGuard. Salsa20/12 is standard and according to the docs is considered secure. I recall that DJB initially specified the cipher with 12 rounds but added 8 more for margin. There are Salsa20/12 implementations in many crypto libraries. The best public cryptanalysis on Salsa breaks 8/20 rounds with a time difficulty of 2^251. Personally I think concern over algorithm strength is usually misplaced as long as you're using an algorithm that is not known to be very weak (e.g. DES, RC4). Halfway decent cryptographic algorithms are almost never broken. Implementations and protocols are broken all the time. One thing we did to guard against protocol issues was to implement what DJB calls "boring crypto." The cryptographic encapsulation is dirt simple Salsa20+Poly1305 constructed exactly as in the NaCl secret box functions, and nothing more. We've resisted adding more advanced features to the cryptographic layer because so far they haven't really been necessary and the risk is not worth the small benefit they may have. At the time this was designed (started in 2010) Salsa20/12 was the best option to achieve high performance across a variety of devices including ARM phones and slow VMs with a clean and small code base. If it were written today it would probably use ChaCha20 (faster) or AES-GCM since almost everything worth mentioning has hardware AES now. We will likely rev the crypto at some point but would also like to upgrade the asymmetric cipher as well. Waiting to see if the Goldilocks curves (curve448) get added to the NIST standard. They are proposed but not yet approved. If they are we'd probably go to curve448+AES256-GCM with a longer IV. That being said we're not solely oriented toward crypto and generally tell our users not to rely on network crypto as their only line of defense. Network crypto is sort of analogous to hard drive encryption. It protects you from people who don't have access to anything, but won't save you once someone gets any access beyond the perimeter. We advocate the use of encryption and authentication within networks as well if you care about security. This also guards against the discovery of a bug in any layer-- if someone breaks one layer they are confronted by another layer. An RFC-style write-up has been on the to-do list forever, but the to-do list is very long.

tptacek 8y ago

Where is ZeroTier's protocol documented? All I could find was a document that said it was custom, and "like IPSEC". What could ZeroTier do as a VPN protocol that would make it better than WireGuard, which has a pedigreed protocol that, in its WireGuard incarnation, has had multiple rounds of formal validation? I wrote the article we're commenting on. Why would I take the time to talk about something like ZeroTier? WireGuard solves the problem I'm talking about decisively. I'm prepared to hear something about ZeroTier that makes it interesting in the WireGuard setting, but I am not interested in a long list of non-VPN non-access-tunnel features that ZeroTier might have that WireGuard lacks, because lacking random features is exactly the point of WireGuard.

comex 8y ago
It may not be interesting to you. To me, and probably a lot of people, the ability to communicate directly between two hosts each behind a NAT (using UDP hole punching) is a big advantage for a VPN, since, compared to forwarding through a central server, it can be faster and lower latency, and it also means you don’t have to pay for that server. Edit: As to why it’s relevant to the “WireGuard setting”, that depends on your definition of the WireGuard setting, but: (1) it’s easy to find examples of people trying to use WireGuard for peer-to-peer communication, the use case ZeroTier solves better; and (2) ZeroTier shares WireGuard’s property of being stateless/connectionless, a significant convenience advantage over traditional VPN protocols.
- tptacek 8y ago
  I think I could have worded that better. By "in the WireGuard setting" I mean "doing the job most startups will use WireGuard to do", which is, manage remote access to internal resources in the cloud. I don't think overlay networks are pointless; I am enthusiastic about them. But for the remote access problem, what I want and what I'm writing about is streamlined VPN software, not a new overlay.
  - comex 8y ago
    Fair enough, although… you know more about this use case than me, but wouldn’t you generally have a large number of servers in the cloud you want to provide remote access to? So if you’re using WireGuard, either you have to configure each client to talk to each server it needs access to – which could be good for security, but sounds like a pain to set up – or you have a central server that serves as a hub for all the VPN traffic. In your blog post you said you just had to “point the client at the server”, so I’m guessing you’re envisioning the latter. But then wouldn’t an overlay network work better in many cases? For example, if you have cloud servers in different geographical locations, and as the developer you’re near one of those locations but far from the VPN server, you’d have lower latency connecting directly…
    - tptacek 8y ago
      I'm imagining simply replacing OpenVPN servers with WireGuard servers, retaining the VPN gateway model; one WireGuard instance acts as a routing gateway for any number of servers.
johntash 8y ago
The only real complaint I would have over WireGuard is that with zerotier, I only need to connect to one server and then can talk to any servers on the same "network" as long as the acl permits it. The other servers are discovered automatically. With WireGuard, I'm pretty sure I would have to set up multiple peers and custom routes. Although I'm assuming you could run BGP on top of the wg interface and that would solve some of it. That said, I'd much rather something the simplicity of wireguard or tinc because it doesn't require a server who's only purpose is to manage clients. I know it's not specifically related to the protocol either uses, but I think people would consider that as a drawback when comparing the two. If you're coming from openvpn/ipsec, then setting up any sort of mesh/routed network would be a similar process. I don't think wireguard would need to implement functionality like that either, I think it would be better if something like tinc used wireguard for the plumbing as someone else mentioned in the comments.
api 8y ago
No RFC-type document yet but quite a bit of commenting here: https://github.com/zerotier/ZeroTierOne/blob/master/node/Packet.hpp https://github.com/zerotier/ZeroTierOne/blob/master/node/Pac... There is also a higher-level description of the entire system here: https://www.zerotier.com/manual.shtml https://www.zerotier.com/manual.shtml ZeroTier solves a broader set of problems than WireGuard and has SDN-type network flow rules, security tap capability, and is a full L2 emulation layer with support for multicast. From what I've seen the WireGuard protocols are excellent and the implementation is very nice and clean. A core ZeroTier implementation to support a simple L3 VPN use case could be as short as WireGuard but that would be without all the rules engine, multicast, etc. stuff.
- tptacek 8y ago
  Just the .cpp code, not counting the .hpp code (which is full of code, including lots of memcpy's) in the node/ subdirectory of this repo is something like 4x the entire size of Linux WireGuard. The crypto protocol is undocumented, but the maintainer is playing games with things like the Salsa round count (which is fine, but I mean, speaks to a mindset). I might poke at it for sport at some point, but I don't see how I could ever recommend it.
  - api 8y ago
    I am the primary maintainer. :) A minimal implementation could be much shorter, but the ref implementation is not minimal and includes a lot of features that go far beyond the scope of WireGuard. Salsa20/12 is standard and according to the docs is considered secure. I recall that DJB initially specified the cipher with 12 rounds but added 8 more for margin. There are Salsa20/12 implementations in many crypto libraries. The best public cryptanalysis on Salsa breaks 8/20 rounds with a time difficulty of 2^251. Personally I think concern over algorithm strength is usually misplaced as long as you're using an algorithm that is not known to be very weak (e.g. DES, RC4). Halfway decent cryptographic algorithms are almost never broken. Implementations and protocols are broken all the time. One thing we did to guard against protocol issues was to implement what DJB calls "boring crypto." The cryptographic encapsulation is dirt simple Salsa20+Poly1305 constructed exactly as in the NaCl secret box functions, and nothing more. We've resisted adding more advanced features to the cryptographic layer because so far they haven't really been necessary and the risk is not worth the small benefit they may have. At the time this was designed (started in 2010) Salsa20/12 was the best option to achieve high performance across a variety of devices including ARM phones and slow VMs with a clean and small code base. If it were written today it would probably use ChaCha20 (faster) or AES-GCM since almost everything worth mentioning has hardware AES now. We will likely rev the crypto at some point but would also like to upgrade the asymmetric cipher as well. Waiting to see if the Goldilocks curves (curve448) get added to the NIST standard. They are proposed but not yet approved. If they are we'd probably go to curve448+AES256-GCM with a longer IV. That being said we're not solely oriented toward crypto and generally tell our users not to rely on network crypto as their only line of defense. Network crypto is sort of analogous to hard drive encryption. It protects you from people who don't have access to anything, but won't save you once someone gets any access beyond the perimeter. We advocate the use of encryption and authentication within networks as well if you care about security. This also guards against the discovery of a bug in any layer-- if someone breaks one layer they are confronted by another layer. An RFC-style write-up has been on the to-do list forever, but the to-do list is very long.