5 ms·

I was moving towards using userspace ZeroTier (via the Python bindings as a SSH proxy command) to allow Bitbucket build runners to access our private network re

I was moving towards using userspace ZeroTier (via the Python bindings as a SSH proxy command) to allow Bitbucket build runners to access our private network repo server, for a similar result. The bitbucket runners run containers in such a way that the normal Zerotier wouldn't run (something about not having a capability flag for tun/tap IIRC). In the end bitbucket released self-hosted runners and we're now running the builds in our network with normal firewall rules limiting access. I'm still looking at overlay networks (probably Nebula), and want to take another look at OpenZiti (I peeked at it a couple months ago).

qrkourier 4y ago
Author here, I want to learn more about ZeroTier too. I overlooked it for this project because it supports layer 2 and I needed tighter coupling of application+policy than layer 2 provides. Admittedly, I don't yet know enough about ZeroTier to know if I can disqualify all layer 2 points of contact with a particular policy. It sounds like the layer 2 DNA of ZeroTier tripped you up in this case too. I know more about OpenZiti than Nebula and will be listening for topics in the Discourse in case you decide to take another stab at it. Nebula is definitely interesting. I got the impression that it's all about automating spin up and tear down of point-to-point VPN tunnels on demand, which has the potential to eliminate much of the headaches around maintaining VPNs. Essentially app and policy configuration driven over hand crafting. That's definitely the approach of OpenZiti too.
- linsomniac 4y ago
  I haven't really used the layer 2 functionality of ZeroTier, I know it has it. I believe in the built-in firewalling rules in ZeroTier you can limit the protocols that it will pass. The rules for it are, in my limited use, pretty obscure, but I have had some luck at setting up rules.
PLG88 4y ago
When taking another look, you may find this brief set of 'superpowers' useful - https://www.youtube.com/playlist?list=PLMUj_5fklasKF1oisSSuLwSzLVxuL9JbC https://www.youtube.com/playlist?list=PLMUj_5fklasKF1oisSSuL.... This includes closing all inbound ports (or FW rules) so that the apps become dark. I believe ZitiTV is going to have a comparison episode too soon incl. ZeroTier and Nebula.
dovholuknf 4y ago
As someone working on OpenZiti itself, I'd be interested in why choose Nebula over OpenZiti. I assume it'd just be due to the cache that Nebula has, since it comes from Slack? I am biased, I know, but OpenZiti offers a lot of interesting stuff over Nebula. If it is a usability issue I'd really like to know since I'd want to make that experience better. Any input you wanted to share would be amazing!
- linsomniac 4y ago
  I can't recall where exactly I saw OpenZiti, it was probably from some discussion on HN. I went to look at it and thought it looked promising, but I'm already partway down the path of Nebula, and as you say it's partly because of Nebula having Slack experience behind it, partly because I have some experience with it, partly I like the architecture. I'll take a peek at OpenZiti again and refresh my understanding of it.
- linsomniac 4y ago
  I remember my first look at OpenZiti now. Here's some feedback: My first impression of OpenZiti was: Nebula but at the application level. It took a fair bit of looking around for me to see the proxy component. I'm still not exactly sure how to get a test, but it looks like the getting started steps are: - Build a network. - Figure out the identities and services. - Set up a proxy. - Test out this proxy. - Eventually migrate services from proxy to SDK. There's a lot of stuff going on here I really need to wrap my mind around before I can even see how it would fit into my network. Nebula on the other hand is an easier mapping for what I already have in my brain: It's a mesh VPN with a CA and built in firewall rules based on certificate roles. OpenZiti looks very interesting now, but it honestly took me an hour to get to that point.
  - dovholuknf 4y ago
    Yikes an hour??? I need to do a better job at explaining it then! Thanks very much for taking the time to provide feedback. Figuring out identities vs IP addresses - yeah I can see how that might be a bit of a leap. Makes me want to produce a video/doc on that topic. "Nebula at the application level" is not far off to be honest. There's probably a fair number of things OpenZiti provides that Nebula isn't trying to provide but it's a decent one-liner type of summary. OpenZiti also seems to cover many/most of the things Nebula set out to do so it's not just for applications. If you have any requests that would help you get started, I'm happy to try to satisfy them! :) cheers!

qrkourier 4y ago

Author here, I want to learn more about ZeroTier too. I overlooked it for this project because it supports layer 2 and I needed tighter coupling of application+policy than layer 2 provides. Admittedly, I don't yet know enough about ZeroTier to know if I can disqualify all layer 2 points of contact with a particular policy. It sounds like the layer 2 DNA of ZeroTier tripped you up in this case too. I know more about OpenZiti than Nebula and will be listening for topics in the Discourse in case you decide to take another stab at it. Nebula is definitely interesting. I got the impression that it's all about automating spin up and tear down of point-to-point VPN tunnels on demand, which has the potential to eliminate much of the headaches around maintaining VPNs. Essentially app and policy configuration driven over hand crafting. That's definitely the approach of OpenZiti too.

linsomniac 4y ago
I haven't really used the layer 2 functionality of ZeroTier, I know it has it. I believe in the built-in firewalling rules in ZeroTier you can limit the protocols that it will pass. The rules for it are, in my limited use, pretty obscure, but I have had some luck at setting up rules.

PLG88 4y ago

When taking another look, you may find this brief set of 'superpowers' useful - https://www.youtube.com/playlist?list=PLMUj_5fklasKF1oisSSuLwSzLVxuL9JbC https://www.youtube.com/playlist?list=PLMUj_5fklasKF1oisSSuL.... This includes closing all inbound ports (or FW rules) so that the apps become dark. I believe ZitiTV is going to have a comparison episode too soon incl. ZeroTier and Nebula.

dovholuknf 4y ago

As someone working on OpenZiti itself, I'd be interested in why choose Nebula over OpenZiti. I assume it'd just be due to the cache that Nebula has, since it comes from Slack? I am biased, I know, but OpenZiti offers a lot of interesting stuff over Nebula. If it is a usability issue I'd really like to know since I'd want to make that experience better. Any input you wanted to share would be amazing!

linsomniac 4y ago
I can't recall where exactly I saw OpenZiti, it was probably from some discussion on HN. I went to look at it and thought it looked promising, but I'm already partway down the path of Nebula, and as you say it's partly because of Nebula having Slack experience behind it, partly because I have some experience with it, partly I like the architecture. I'll take a peek at OpenZiti again and refresh my understanding of it.
linsomniac 4y ago
I remember my first look at OpenZiti now. Here's some feedback: My first impression of OpenZiti was: Nebula but at the application level. It took a fair bit of looking around for me to see the proxy component. I'm still not exactly sure how to get a test, but it looks like the getting started steps are: - Build a network. - Figure out the identities and services. - Set up a proxy. - Test out this proxy. - Eventually migrate services from proxy to SDK. There's a lot of stuff going on here I really need to wrap my mind around before I can even see how it would fit into my network. Nebula on the other hand is an easier mapping for what I already have in my brain: It's a mesh VPN with a CA and built in firewall rules based on certificate roles. OpenZiti looks very interesting now, but it honestly took me an hour to get to that point.
- dovholuknf 4y ago
  Yikes an hour??? I need to do a better job at explaining it then! Thanks very much for taking the time to provide feedback. Figuring out identities vs IP addresses - yeah I can see how that might be a bit of a leap. Makes me want to produce a video/doc on that topic. "Nebula at the application level" is not far off to be honest. There's probably a fair number of things OpenZiti provides that Nebula isn't trying to provide but it's a decent one-liner type of summary. OpenZiti also seems to cover many/most of the things Nebula set out to do so it's not just for applications. If you have any requests that would help you get started, I'm happy to try to satisfy them! :) cheers!