5 ms·

As someone working on OpenZiti itself, I'd be interested in why choose Nebula over OpenZiti. I assume it'd just be due to the cache that Nebula has, since it co

As someone working on OpenZiti itself, I'd be interested in why choose Nebula over OpenZiti. I assume it'd just be due to the cache that Nebula has, since it comes from Slack? I am biased, I know, but OpenZiti offers a lot of interesting stuff over Nebula. If it is a usability issue I'd really like to know since I'd want to make that experience better. Any input you wanted to share would be amazing!

linsomniac 4y ago
I can't recall where exactly I saw OpenZiti, it was probably from some discussion on HN. I went to look at it and thought it looked promising, but I'm already partway down the path of Nebula, and as you say it's partly because of Nebula having Slack experience behind it, partly because I have some experience with it, partly I like the architecture. I'll take a peek at OpenZiti again and refresh my understanding of it.
linsomniac 4y ago
I remember my first look at OpenZiti now. Here's some feedback: My first impression of OpenZiti was: Nebula but at the application level. It took a fair bit of looking around for me to see the proxy component. I'm still not exactly sure how to get a test, but it looks like the getting started steps are: - Build a network. - Figure out the identities and services. - Set up a proxy. - Test out this proxy. - Eventually migrate services from proxy to SDK. There's a lot of stuff going on here I really need to wrap my mind around before I can even see how it would fit into my network. Nebula on the other hand is an easier mapping for what I already have in my brain: It's a mesh VPN with a CA and built in firewall rules based on certificate roles. OpenZiti looks very interesting now, but it honestly took me an hour to get to that point.
- dovholuknf 4y ago
  Yikes an hour??? I need to do a better job at explaining it then! Thanks very much for taking the time to provide feedback. Figuring out identities vs IP addresses - yeah I can see how that might be a bit of a leap. Makes me want to produce a video/doc on that topic. "Nebula at the application level" is not far off to be honest. There's probably a fair number of things OpenZiti provides that Nebula isn't trying to provide but it's a decent one-liner type of summary. OpenZiti also seems to cover many/most of the things Nebula set out to do so it's not just for applications. If you have any requests that would help you get started, I'm happy to try to satisfy them! :) cheers!
  - linsomniac 4y ago
    I have a few questions off the top of my head if you have any thoughts on them. Thanks so much for your replies on this comment thread. - What is the performance of OpenZiti like? I'm seeing some Nebula performance numbers (from third parties, not clocked myself) that seem surprisingly bad, but I'm also not particularly performance sensitive as far as throughput for most things. - I predict problems with DNS on 127.0.0.1:53 on some hosts, thinking it might be better on 127.0.0.53:53 though I feel like that might already be commonly used for something. I know on my work laptop I have dnsmasq running there that forwards DNS around, that's the primary place I'm expecting problems, I mean with the exception of my internal DNS servers. - Automated enrollment? Most of my dev/stg machines respin every other day. I understand enrollment is done via a code that can be e-mailed, so I presume it's a one-time-use code. I need some way for the machines to re-join the OpenZiti after being deleted, a base Ubuntu installed, and then Ansible run. For Nebula this would be by delivering the certificate to the machine. Thanks!
    - dovholuknf 4y ago
      - The project is actually currently in the middle of some performance comparison testing. I myself am not conducting the tests and I'd like to review the methodology we use for the tests but initial reports I'm hearing are quite good. I do have first-hand anecdotal evidence from a heavy user of OpenZiti indicating from Australia to the US he'll usually pull 40-50mbps over open internet. OpenZiti often pulls 50 consistently while still being a zero trust overlay. One of the knocks on OpenZiti is the mTLS between links, and full end to end encryption on by default being "overkill". This anecdotal evidence however, seems to suggest that the CPU burden of "all that encryption" is not significant at all. - The private, authenticated DNS provided by the tunnellers has not been a problem whatsoever for nearly every user. Sure there are times when bugs creep in, but I'd say for the majority of times people use any of the tunneling apps the way we do private DNS has been no issue. The OP of this article might have more to offer here. He's very well versed with DNS in general. I don't recall exactly how this works in linux but we have some advanced linux users doing "interesting" things and I know of no major issues... yet... :) If you try OpenZiti, I hope you won't be the first! Each OS is a little different here too. The Windows clients work differently and the MacOS ones a bit differently. All "somewhat" the same but sometimes with subtle nuances - "For Nebula this would be by delivering the certificate to the machine" - realistically that's exactly the same process with OpenZiti's "regular/most common" type of enrollment - the "one time token" approach (using jwts). You deliver a jwt however you like with email being just one option. Then you just re-enroll that jwt which creates a primary key and cert right then/there on that machine, and off you go... You might find another feature interesting though, OpenZiti also supports 3rd party CA enrollment. This means you can register a CA with OpenZiti and if any device connects and presents a certificate signed by YOUR CA (you don't even have to trust OpenZiti, you can run your own PKI) then you can set that device up for "automatic enrollment" which is truly automatic. You don't need to deliver a new certificate (but you could). This is a somewhat detailed topic and I don't think I could fit enough words on this post and keep it legible. Would be happy to continue that conversation here or in the OpenZiti discourse forum. One other interesting thing you might like... We just released our first release of a python sdk for OpenZiti. It's able to monkey patch python to allow us to use ansible over OpenZiti. So hopefully soon we should be able to demonstrate ansible over OpenZiti. I've seen it working but it's not __quite__ ready to share to the world. Soon, I'm sure!

linsomniac 4y ago

I can't recall where exactly I saw OpenZiti, it was probably from some discussion on HN. I went to look at it and thought it looked promising, but I'm already partway down the path of Nebula, and as you say it's partly because of Nebula having Slack experience behind it, partly because I have some experience with it, partly I like the architecture. I'll take a peek at OpenZiti again and refresh my understanding of it.

linsomniac 4y ago

I remember my first look at OpenZiti now. Here's some feedback: My first impression of OpenZiti was: Nebula but at the application level. It took a fair bit of looking around for me to see the proxy component. I'm still not exactly sure how to get a test, but it looks like the getting started steps are: - Build a network. - Figure out the identities and services. - Set up a proxy. - Test out this proxy. - Eventually migrate services from proxy to SDK. There's a lot of stuff going on here I really need to wrap my mind around before I can even see how it would fit into my network. Nebula on the other hand is an easier mapping for what I already have in my brain: It's a mesh VPN with a CA and built in firewall rules based on certificate roles. OpenZiti looks very interesting now, but it honestly took me an hour to get to that point.

dovholuknf 4y ago
Yikes an hour??? I need to do a better job at explaining it then! Thanks very much for taking the time to provide feedback. Figuring out identities vs IP addresses - yeah I can see how that might be a bit of a leap. Makes me want to produce a video/doc on that topic. "Nebula at the application level" is not far off to be honest. There's probably a fair number of things OpenZiti provides that Nebula isn't trying to provide but it's a decent one-liner type of summary. OpenZiti also seems to cover many/most of the things Nebula set out to do so it's not just for applications. If you have any requests that would help you get started, I'm happy to try to satisfy them! :) cheers!
- linsomniac 4y ago
  I have a few questions off the top of my head if you have any thoughts on them. Thanks so much for your replies on this comment thread. - What is the performance of OpenZiti like? I'm seeing some Nebula performance numbers (from third parties, not clocked myself) that seem surprisingly bad, but I'm also not particularly performance sensitive as far as throughput for most things. - I predict problems with DNS on 127.0.0.1:53 on some hosts, thinking it might be better on 127.0.0.53:53 though I feel like that might already be commonly used for something. I know on my work laptop I have dnsmasq running there that forwards DNS around, that's the primary place I'm expecting problems, I mean with the exception of my internal DNS servers. - Automated enrollment? Most of my dev/stg machines respin every other day. I understand enrollment is done via a code that can be e-mailed, so I presume it's a one-time-use code. I need some way for the machines to re-join the OpenZiti after being deleted, a base Ubuntu installed, and then Ansible run. For Nebula this would be by delivering the certificate to the machine. Thanks!
  - dovholuknf 4y ago
    - The project is actually currently in the middle of some performance comparison testing. I myself am not conducting the tests and I'd like to review the methodology we use for the tests but initial reports I'm hearing are quite good. I do have first-hand anecdotal evidence from a heavy user of OpenZiti indicating from Australia to the US he'll usually pull 40-50mbps over open internet. OpenZiti often pulls 50 consistently while still being a zero trust overlay. One of the knocks on OpenZiti is the mTLS between links, and full end to end encryption on by default being "overkill". This anecdotal evidence however, seems to suggest that the CPU burden of "all that encryption" is not significant at all. - The private, authenticated DNS provided by the tunnellers has not been a problem whatsoever for nearly every user. Sure there are times when bugs creep in, but I'd say for the majority of times people use any of the tunneling apps the way we do private DNS has been no issue. The OP of this article might have more to offer here. He's very well versed with DNS in general. I don't recall exactly how this works in linux but we have some advanced linux users doing "interesting" things and I know of no major issues... yet... :) If you try OpenZiti, I hope you won't be the first! Each OS is a little different here too. The Windows clients work differently and the MacOS ones a bit differently. All "somewhat" the same but sometimes with subtle nuances - "For Nebula this would be by delivering the certificate to the machine" - realistically that's exactly the same process with OpenZiti's "regular/most common" type of enrollment - the "one time token" approach (using jwts). You deliver a jwt however you like with email being just one option. Then you just re-enroll that jwt which creates a primary key and cert right then/there on that machine, and off you go... You might find another feature interesting though, OpenZiti also supports 3rd party CA enrollment. This means you can register a CA with OpenZiti and if any device connects and presents a certificate signed by YOUR CA (you don't even have to trust OpenZiti, you can run your own PKI) then you can set that device up for "automatic enrollment" which is truly automatic. You don't need to deliver a new certificate (but you could). This is a somewhat detailed topic and I don't think I could fit enough words on this post and keep it legible. Would be happy to continue that conversation here or in the OpenZiti discourse forum. One other interesting thing you might like... We just released our first release of a python sdk for OpenZiti. It's able to monkey patch python to allow us to use ansible over OpenZiti. So hopefully soon we should be able to demonstrate ansible over OpenZiti. I've seen it working but it's not __quite__ ready to share to the world. Soon, I'm sure!