5 ms·

ZeroTier founder here -- We've been hard at work on this for a very long time, and we're obviously quite happy today! This is a must-have or really-want for a

by api 4y ago

ZeroTier founder here -- We've been hard at work on this for a very long time, and we're obviously quite happy today! This is a must-have or really-want for a huge number of business customers. That being said, we don't and are not going to require SSO sign-in. You'll always be able to just authorize a device. (On SSO networks that's done by setting them to SSO-exempt.) You can also self-host network controllers like always, and we're planning on making that easier in the future not harder. We're all about not forcing you into ecosystems (SSO can do that) and about decentralization. As for other stuff coming in the near-mid future: version 2.0 is not dead. We obviously got massively derailed (mostly good but very distracting things) and under-estimated the scope and time, but it's still moving forward. We have not announced this elsewhere yet, but it's a near-total port to Rust. This is to get into a more modern language but also to use a safe language for security reasons. This has added time to the job but we're of the opinion that security-related software not written in a safe language is going to be considered a bad thing in the near future (if not already). V2 also has some significant cryptography improvements that bring us more to parity with more modern constructions like the noise framework. This part has been going slow too because we've been moving carefully and soliciting a lot of peer review both informal and formally hired. Cryptography isn't something you just toss out the door and YOLO. :) Last but not least we are planning on some kind of transition from the BSL back to an OSI-compliant licensing scheme, but want to think this through rather than flail around. People at this site have really been fans and have helped us a lot over the years, and we're grateful. Thanks!

gesman 4y ago
I absolutely loved ZeroTier for last 4 yrs. Until 2 weeks ago when my Windows machines absolutely stopped seeing each other and communicating with each other. I made a post on ZeroTier discussion group 2 weeks ago with zero replies so far: https://discuss.zerotier.com/t/windows-machines-lost-access-to-each-other/7637 https://discuss.zerotier.com/t/windows-machines-lost-access-... If i can get some help - would be amazing.
- schmidp 4y ago
  not a lot of detail in your post. have you worked through: https://docs.zerotier.com/zerotier/troubleshooting/ https://docs.zerotier.com/zerotier/troubleshooting/ ?
  - grrrzant 4y ago
    Pretty much this ^^ You've given no detail that differentiates your issue from all the other general troubleshooting advice out there already, nor do you say what you've tried to debug it.
  - gesman 4y ago
    Yes. Didn't help. However suddenly one laptop was able to connect to 2 other windows boxes (even though other Windows machines cant). This tells me it's not ZT fault, but rather everlasting update and restriction nonsense from windows is a culprit
- SailingSperm 4y ago
  Had something maybe similar too - some times instant connecting, sometimes slow, sometimes 0%. Fixed and happily using ZT every day now. (TY ZT!) On both computers do: Network connections->Right click ZT network->Properties->Select (TCP/IPv4), probs 5th down. ->Properties->Advanced->Untick automatic metric-> set interface metric value to 1. Potentially reboot, though I've not needed to before. Just for some visuals if needed https://support.parsec.app/hc/en-us/articles/115002766652-Using-a-VPN-ZeroTier-to-Connect-on-Parsec https://support.parsec.app/hc/en-us/articles/115002766652-Us...
schmidp 4y ago
Always great to hear if a project adopts rust. ZeroTier, in my experience, has been one of those applications that just work. Keep up the great work!
sandstrom 4y ago
I recently tested zero tier for a company use case (mobile development, giving a dev phone access to a dev computer, or allowing sharing between dev computers; basically sharing something with a colleague). Also tested Tail Scale. Some feedback (only writing this to help you improve): 1. Your UI is horrible. Hire 1-2 front-end/designers and copy everything that tail scale does right. 2. You should add a concept of users with e.g. GitHub as SSO-provider, like tail scale does. Maybe that’s what you’re releasing now? 3. Your docs are very bad compared to tail scale, and you should have much more docs on common scenarios and use cases. For example, mine wasn’t mentioned but is fairly common. You are losing a lot of business here. 4. iOS VPN auto connect functionality is good. 5. You should add some type of global dns, such that we could map all devices/users like this: macbook1.jane.my-company.net (resolve via your network; you host the dns, we provide our domain). Basically what Consul does. 6. If user is authed (see 1 above), auth of devices should be optional. 7. Your language for network rules is too complicated. 8. My impression was that your network software is better than TailScale, but in every other way they beat you (docs, UI, usability, features). 9. iOS app is ugly and have obvious bugs, like you can’t enter text in fields in lower case without hassle. Couldn’t actually get my use-case to work on tailscale either. They stuff they’re missing is in the works though. Will revisit you both in 6 months. I’m rooting for you, but you must understand that it’s not only about software, all the packaging around it is also important (and you are severely lacking in this area).
- api 4y ago
  The SSO release today adds a concept of users. That's what it's all about. You are 1000% right about UI/UX. It was good for networking software years ago but the ecosystem has generally improved since then. If anyone who is reading wants to help: https://jobs.lever.co/zerotier/90436aee-8e55-406d-9053-a0c26f542862 https://jobs.lever.co/zerotier/90436aee-8e55-406d-9053-a0c26... Location is set to Cincinnati because we want to nucleate more engineers in this region but the position is open to anyone in the USA. We'll hire anywhere if someone is really good. We're remote-first. Edit: > 7. Your language for network rules is too complicated. It's too low-level. We are researching a higher-level way to edit rules in terms of intent rather than the current pf-esque rules language that requires you to deeply understand TCP/IP and such. That being said it is very powerful and you can do extremely neat stuff with it. It's in some ways more powerful than what you get with enterprise data center SDN products.
dvzk 4y ago
Have you described your reasons for not migrating to Wireguard in v2.0? Is it partially because Wireguard only really has a portable (and heavyweight) Go implementation that's also tightly bound to OS networking primitives? I'm mainly wondering if there are also cryptography or protocol limitations that make it unsuitable.
- api 4y ago
  Wireguard is a really nice clean transport protocol, but there are a few: (1) We use AES for performance and other reasons. ChaCha is much faster than AES in software but AES blows it away by 2-4X in hardware. We have customers using ZeroTier in an SD-WAN setting who are leveraging AES acceleration (with some of their own custom code) in network accelerators and other devices. We also think we can beat both in-kernel WG and ipsec in V2 on Linux using io_uring because the cryptographic part of ZeroTier is 2-4X faster than WG on "real hardware." My M1 Mac does 2GiB/sec/core and newer Intel/AMD chips with AVX VAES can do as much as 10GiB/sec/core. (2) We have a bunch of customers who want FIPS. Yes, we know. We're not big fans either but they're large customers and whole industries. We also have a pretty good no-compromise plan for this that treats FIPS as a floor for cryptographic quality not a ceiling and doesn't add meaningful complexity. This is a V2 thing. (You can bootstrap WG with keys from a NIST ECC key exchange to get the asymmetric part of FIPS, but ChaCha is not FIPS.) (3) We want to maintain backward compatibility with millions and millions of ZeroTier devices out there, at least as far back as we can. (4) ZeroTier uses a construction where the identity is the outer envelope and then (in V2) the ephemeral forward secrecy keys are the inner envelope. Wireguard does it the other way around. Both are equivalent in terms of cryptographic strength (and I think in noise this is a choice, but it's been a while since I read through that). The advantage of the WG construction is that session participants can't be identified from packets. That's a disadvantage in enterprise though, because they want that. It lets us implement elegant solutions to making a mesh P2P network play nice with enterprise firewall and internal/DMZ isolation practices among other things. It also lets our roots/relays basically do something analogous to MPLS, pumping vast amounts of traffic with virtually no CPU overhead because they don't need to decrypt anything. We toyed around with making an AES fork of Wireguard, basically use the exact same construction as WG with AES-GMAC-SIV instead of ChaChaPoly. Would be totally fine cryptographically but see #4. We also support virtualization at layer 2 rather than just layer 3, which is important for a bunch of our industrial use cases. It would be possible though to stuff L2 into WG pretty easily, so that's not really a reason. WG is just a transport protocol so you could stuff anything into it. Fly.io used it as a light alternative to TLS I think. It's just normally used to carry L3 IP. Edit: also in V2 we are modularizing things, separating the transport protocol from the rest of the code. It'll be in open source Rust and very portable. In our (currently private) Rust repo vl1/ is only about 5K lines of Rust including tons of comments and non-crypto stuff. We also have plans to document it a lot better.

by api 4y ago

gesman 4y ago

I absolutely loved ZeroTier for last 4 yrs. Until 2 weeks ago when my Windows machines absolutely stopped seeing each other and communicating with each other. I made a post on ZeroTier discussion group 2 weeks ago with zero replies so far: https://discuss.zerotier.com/t/windows-machines-lost-access-to-each-other/7637 https://discuss.zerotier.com/t/windows-machines-lost-access-... If i can get some help - would be amazing.

schmidp 4y ago
not a lot of detail in your post. have you worked through: https://docs.zerotier.com/zerotier/troubleshooting/ https://docs.zerotier.com/zerotier/troubleshooting/ ?
- grrrzant 4y ago
  Pretty much this ^^ You've given no detail that differentiates your issue from all the other general troubleshooting advice out there already, nor do you say what you've tried to debug it.
- gesman 4y ago
  Yes. Didn't help. However suddenly one laptop was able to connect to 2 other windows boxes (even though other Windows machines cant). This tells me it's not ZT fault, but rather everlasting update and restriction nonsense from windows is a culprit
SailingSperm 4y ago
Had something maybe similar too - some times instant connecting, sometimes slow, sometimes 0%. Fixed and happily using ZT every day now. (TY ZT!) On both computers do: Network connections->Right click ZT network->Properties->Select (TCP/IPv4), probs 5th down. ->Properties->Advanced->Untick automatic metric-> set interface metric value to 1. Potentially reboot, though I've not needed to before. Just for some visuals if needed https://support.parsec.app/hc/en-us/articles/115002766652-Using-a-VPN-ZeroTier-to-Connect-on-Parsec https://support.parsec.app/hc/en-us/articles/115002766652-Us...

schmidp 4y ago

Always great to hear if a project adopts rust. ZeroTier, in my experience, has been one of those applications that just work. Keep up the great work!

sandstrom 4y ago

I recently tested zero tier for a company use case (mobile development, giving a dev phone access to a dev computer, or allowing sharing between dev computers; basically sharing something with a colleague). Also tested Tail Scale. Some feedback (only writing this to help you improve): 1. Your UI is horrible. Hire 1-2 front-end/designers and copy everything that tail scale does right. 2. You should add a concept of users with e.g. GitHub as SSO-provider, like tail scale does. Maybe that’s what you’re releasing now? 3. Your docs are very bad compared to tail scale, and you should have much more docs on common scenarios and use cases. For example, mine wasn’t mentioned but is fairly common. You are losing a lot of business here. 4. iOS VPN auto connect functionality is good. 5. You should add some type of global dns, such that we could map all devices/users like this: macbook1.jane.my-company.net (resolve via your network; you host the dns, we provide our domain). Basically what Consul does. 6. If user is authed (see 1 above), auth of devices should be optional. 7. Your language for network rules is too complicated. 8. My impression was that your network software is better than TailScale, but in every other way they beat you (docs, UI, usability, features). 9. iOS app is ugly and have obvious bugs, like you can’t enter text in fields in lower case without hassle. Couldn’t actually get my use-case to work on tailscale either. They stuff they’re missing is in the works though. Will revisit you both in 6 months. I’m rooting for you, but you must understand that it’s not only about software, all the packaging around it is also important (and you are severely lacking in this area).

api 4y ago
The SSO release today adds a concept of users. That's what it's all about. You are 1000% right about UI/UX. It was good for networking software years ago but the ecosystem has generally improved since then. If anyone who is reading wants to help: https://jobs.lever.co/zerotier/90436aee-8e55-406d-9053-a0c26f542862 https://jobs.lever.co/zerotier/90436aee-8e55-406d-9053-a0c26... Location is set to Cincinnati because we want to nucleate more engineers in this region but the position is open to anyone in the USA. We'll hire anywhere if someone is really good. We're remote-first. Edit: > 7. Your language for network rules is too complicated. It's too low-level. We are researching a higher-level way to edit rules in terms of intent rather than the current pf-esque rules language that requires you to deeply understand TCP/IP and such. That being said it is very powerful and you can do extremely neat stuff with it. It's in some ways more powerful than what you get with enterprise data center SDN products.

dvzk 4y ago

Have you described your reasons for not migrating to Wireguard in v2.0? Is it partially because Wireguard only really has a portable (and heavyweight) Go implementation that's also tightly bound to OS networking primitives? I'm mainly wondering if there are also cryptography or protocol limitations that make it unsuitable.

api 4y ago
Wireguard is a really nice clean transport protocol, but there are a few: (1) We use AES for performance and other reasons. ChaCha is much faster than AES in software but AES blows it away by 2-4X in hardware. We have customers using ZeroTier in an SD-WAN setting who are leveraging AES acceleration (with some of their own custom code) in network accelerators and other devices. We also think we can beat both in-kernel WG and ipsec in V2 on Linux using io_uring because the cryptographic part of ZeroTier is 2-4X faster than WG on "real hardware." My M1 Mac does 2GiB/sec/core and newer Intel/AMD chips with AVX VAES can do as much as 10GiB/sec/core. (2) We have a bunch of customers who want FIPS. Yes, we know. We're not big fans either but they're large customers and whole industries. We also have a pretty good no-compromise plan for this that treats FIPS as a floor for cryptographic quality not a ceiling and doesn't add meaningful complexity. This is a V2 thing. (You can bootstrap WG with keys from a NIST ECC key exchange to get the asymmetric part of FIPS, but ChaCha is not FIPS.) (3) We want to maintain backward compatibility with millions and millions of ZeroTier devices out there, at least as far back as we can. (4) ZeroTier uses a construction where the identity is the outer envelope and then (in V2) the ephemeral forward secrecy keys are the inner envelope. Wireguard does it the other way around. Both are equivalent in terms of cryptographic strength (and I think in noise this is a choice, but it's been a while since I read through that). The advantage of the WG construction is that session participants can't be identified from packets. That's a disadvantage in enterprise though, because they want that. It lets us implement elegant solutions to making a mesh P2P network play nice with enterprise firewall and internal/DMZ isolation practices among other things. It also lets our roots/relays basically do something analogous to MPLS, pumping vast amounts of traffic with virtually no CPU overhead because they don't need to decrypt anything. We toyed around with making an AES fork of Wireguard, basically use the exact same construction as WG with AES-GMAC-SIV instead of ChaChaPoly. Would be totally fine cryptographically but see #4. We also support virtualization at layer 2 rather than just layer 3, which is important for a bunch of our industrial use cases. It would be possible though to stuff L2 into WG pretty easily, so that's not really a reason. WG is just a transport protocol so you could stuff anything into it. Fly.io used it as a light alternative to TLS I think. It's just normally used to carry L3 IP. Edit: also in V2 we are modularizing things, separating the transport protocol from the rest of the code. It'll be in open source Rust and very portable. In our (currently private) Rust repo vl1/ is only about 5K lines of Rust including tons of comments and non-crypto stuff. We also have plans to document it a lot better.