6 ms·

Tailscale Kubernetes Operator

157 points by l2dy 3y ago

thelastparadise 3y ago
It's a neat idea but I wouldn't put this in my k8s cluster. Keep it simple st*pid!
- leetrout 3y ago
  You can use, instead: Keep it simple, smarty
  - deleted 3y ago
    [deleted]
  - ses1984 3y ago
    Keep it super simple.
  - birdyrooster 3y ago
    [flagged]
  - SanderNL 3y ago
    [flagged]
  - valvar 3y ago
    Is "stupid" really one of the words that the Ministry of Truth no longer permits us to use?
    - leetrout 3y ago
      They felt the need to censor it as st*pid so I merely offered the alternative I use.
    - biohax2015 3y ago
      It's just rude
    - wenebego 3y ago
      1984 is when people dont like being called stupid
- berkle4455 3y ago
  k8s cluster and simple are rarely found together.
gmemstr 3y ago
Maybe not wise to post here yet as the docs mark this as a private alpha.
- yebyen 3y ago
  Maybe so, as what we're doing in absence of proper docs is rightly classified as something worse than alpha. The best docs they had so far up to this point: https://tailscale.com/kb/1185/kubernetes/ https://tailscale.com/kb/1185/kubernetes/ These are great, for someone who knows Kubernetes. You should understand that creating the tailscale subnet router as a pod directly means the connection is not resilient. It's also key to understand that tailscale will break if you have more than one instance of subnet router at a time, so substituting Deployment in place of where these docs use Pod is not a really good choice without some fine tuning because of the risk that a rolling update creates another copy of the pod before the old one has shut down. Maybe stateful set, if there was a way to permanently imply that statefulset can only have one replica. I appreciate the link anyway. I figured all this out on my own, and I'm using tailscale productively with Kubernetes based on the old docs, with my open source project. Tailscale has a great and generous free software tier for supporting OSS maintainers. :tada:
tecleandor 3y ago
Nice! I think I'll try to implement this in my TrueNAS this weekend, as it makes way way easier to access all the services I deploy there. TrueCharts charts have Tailscale support, but not all my charts are from there, and also I'm kind of avoiding them. Also, it's nice if you deploy something without a chart.
HappyCathode 3y ago
That's really neat. Cloudflare tunnel for external customer egress, and Tailscale for internal tool egress. No more costly cloud specific load balancers !
- BasedInfra 3y ago
  You can use Cloudflare access for internal which is tunnel + identity access management for end users.
  - HappyCathode 3y ago
    I'de rather have full network isolation for internal stuff like admin portals. Plus, I already use Tailscale to sync DBs between regions and clouds.
    - ratg13 3y ago
      My only issue with Tailscale was that it can't seem to stay logged in longer than something like 45 or 90 days.. making it a fun toy, but not for enterprise use. As someone who travels a lot with machines all over the world, if a node goes offline I can ask someone to reboot a machine .. but there is no way I am giving random people credentials to my machines and network to fix issues.
      - neurostimulant 3y ago
        There is an option to disable key expiry in the machine settings, unless you're talking about a different issue / bug. In my case, simply turning off key expiry is enough to keep the machine online for months inside tailscale network so far.
        ratg13 3y ago
        Thank you, I will take a look for this and give it a try.
      - 3y ago
sneak 3y ago
The amount of trust placed in DockerHub to serve the correct images to everyone for uncritical download and execution is insane to me. The whole industry does it, and it's the same as "curl | bash" to specify image:tag and not image@hash. If I were TAO I'd apply for a job at Docker or their hosting provider.
- Lemmi 3y ago
  That's why you only pull official images and signed ones. And that's why I have an in-between step. Harbor.io allows you to configure it as a proxy with approval mechanism and cve scanning
  - sneak 3y ago
    Docker content trust (ie signature checking) is disabled by default. We won't even do this for webpages, but we find it a fine default for code that executes inside critical infrastructure. It's utter madness. Cool to see someone is doing something about it.
- api 3y ago
  We have collectively decided as an industry to place infinite trust in a handful of vendors for the sake of convenience. Auth providers (OIDC), cloud providers, and numerous software repositories and SaaS providers effectively have root on the entire universe. A major compromise of any number of these large vendors could expose millions of systems to total compromise.
  - deleted 3y ago
    [deleted]
  - sneak 3y ago
    Resultantly, these systems are almost certainly already compromised by organizations skilled enough to not get caught.
    - api 3y ago
shawabawa3 3y ago
One thing that i think is really missing is redundancy on the proxies Currently there's no way to have two proxies that listen on the same tailscale hostname/ip Ideally in Kubernetes every pod is redundant to allow downscaling of nodes efficiently, so this means we have to eat a minute or so of downtime randomly every now and then on our tailscale endpoints
- vorpalhex 3y ago
  Can't you expose the services to two load balancers? Not ideal for anything customer facing but fine enough for staff.
  - shawabawa3 3y ago
    they would have to have two separate hostnames/ips so you wouldn't have a single redundant endpoint
- gameoverhumans 3y ago
  I agree, it's a feature that I find sorely lacking in my tailnet. These are the relevant Github issues to follow, hopefully they address these someday: https://github.com/tailscale/tailscale/issues/465 https://github.com/tailscale/tailscale/issues/465 https://github.com/tailscale/tailscale/issues/4324 https://github.com/tailscale/tailscale/issues/4324
drewda 3y ago
We've been using https://github.com/mvisonneau/docker-tailscale/ https://github.com/mvisonneau/docker-tailscale/ on k8s clusters. Good to see an official option coming.
- Axsuul 3y ago
  Same however I'm using HashiCorp Nomad so hope for an official option there.
knodi 3y ago
Love it!!! This is going to make (my) dev and testing env cluster so much easier. Now imagine running derp-server with in the DC with your k8s.
Stem0037 3y ago
Does it support headscale? https://github.com/juanfont/headscale https://github.com/juanfont/headscale
hobofan 3y ago
I think an alternative solution would be nice, where services are just registered with their service discovery, though I suppose that would require them providing an official API for that part of their product. In my current setup for Tailscale + Kubernetes, I just use their subnet router[0] and add the kubedns server for the cluster.local domains to their MagicDNS. Having proper service discovery would make this into a nice round solutionl. [0]: https://tailscale.com/kb/1185/kubernetes/#subnet-router https://tailscale.com/kb/1185/kubernetes/#subnet-router

157 points by l2dy 3y ago

thelastparadise 3y ago

It's a neat idea but I wouldn't put this in my k8s cluster. Keep it simple st*pid!

leetrout 3y ago
You can use, instead: Keep it simple, smarty
- deleted 3y ago
  [deleted]
- ses1984 3y ago
  Keep it super simple.
- birdyrooster 3y ago
  [flagged]
- SanderNL 3y ago
  [flagged]
- valvar 3y ago
  Is "stupid" really one of the words that the Ministry of Truth no longer permits us to use?
  - leetrout 3y ago
    They felt the need to censor it as st*pid so I merely offered the alternative I use.
  - biohax2015 3y ago
    It's just rude
  - wenebego 3y ago
    1984 is when people dont like being called stupid
berkle4455 3y ago
k8s cluster and simple are rarely found together.

gmemstr 3y ago

Maybe not wise to post here yet as the docs mark this as a private alpha.

yebyen 3y ago
Maybe so, as what we're doing in absence of proper docs is rightly classified as something worse than alpha. The best docs they had so far up to this point: https://tailscale.com/kb/1185/kubernetes/ https://tailscale.com/kb/1185/kubernetes/ These are great, for someone who knows Kubernetes. You should understand that creating the tailscale subnet router as a pod directly means the connection is not resilient. It's also key to understand that tailscale will break if you have more than one instance of subnet router at a time, so substituting Deployment in place of where these docs use Pod is not a really good choice without some fine tuning because of the risk that a rolling update creates another copy of the pod before the old one has shut down. Maybe stateful set, if there was a way to permanently imply that statefulset can only have one replica. I appreciate the link anyway. I figured all this out on my own, and I'm using tailscale productively with Kubernetes based on the old docs, with my open source project. Tailscale has a great and generous free software tier for supporting OSS maintainers. :tada:

tecleandor 3y ago

Nice! I think I'll try to implement this in my TrueNAS this weekend, as it makes way way easier to access all the services I deploy there. TrueCharts charts have Tailscale support, but not all my charts are from there, and also I'm kind of avoiding them. Also, it's nice if you deploy something without a chart.

HappyCathode 3y ago

That's really neat. Cloudflare tunnel for external customer egress, and Tailscale for internal tool egress. No more costly cloud specific load balancers !

BasedInfra 3y ago
You can use Cloudflare access for internal which is tunnel + identity access management for end users.
- HappyCathode 3y ago
  I'de rather have full network isolation for internal stuff like admin portals. Plus, I already use Tailscale to sync DBs between regions and clouds.
  - ratg13 3y ago
    My only issue with Tailscale was that it can't seem to stay logged in longer than something like 45 or 90 days.. making it a fun toy, but not for enterprise use. As someone who travels a lot with machines all over the world, if a node goes offline I can ask someone to reboot a machine .. but there is no way I am giving random people credentials to my machines and network to fix issues.
    - neurostimulant 3y ago
      There is an option to disable key expiry in the machine settings, unless you're talking about a different issue / bug. In my case, simply turning off key expiry is enough to keep the machine online for months inside tailscale network so far.
      - ratg13 3y ago
        Thank you, I will take a look for this and give it a try.
    - 3y ago

sneak 3y ago

The amount of trust placed in DockerHub to serve the correct images to everyone for uncritical download and execution is insane to me. The whole industry does it, and it's the same as "curl | bash" to specify image:tag and not image@hash. If I were TAO I'd apply for a job at Docker or their hosting provider.

Lemmi 3y ago
That's why you only pull official images and signed ones. And that's why I have an in-between step. Harbor.io allows you to configure it as a proxy with approval mechanism and cve scanning
- sneak 3y ago
  Docker content trust (ie signature checking) is disabled by default. We won't even do this for webpages, but we find it a fine default for code that executes inside critical infrastructure. It's utter madness. Cool to see someone is doing something about it.
api 3y ago
We have collectively decided as an industry to place infinite trust in a handful of vendors for the sake of convenience. Auth providers (OIDC), cloud providers, and numerous software repositories and SaaS providers effectively have root on the entire universe. A major compromise of any number of these large vendors could expose millions of systems to total compromise.
- deleted 3y ago
  [deleted]
- sneak 3y ago
  Resultantly, these systems are almost certainly already compromised by organizations skilled enough to not get caught.
  - api 3y ago

shawabawa3 3y ago

One thing that i think is really missing is redundancy on the proxies Currently there's no way to have two proxies that listen on the same tailscale hostname/ip Ideally in Kubernetes every pod is redundant to allow downscaling of nodes efficiently, so this means we have to eat a minute or so of downtime randomly every now and then on our tailscale endpoints

vorpalhex 3y ago
Can't you expose the services to two load balancers? Not ideal for anything customer facing but fine enough for staff.
- shawabawa3 3y ago
  they would have to have two separate hostnames/ips so you wouldn't have a single redundant endpoint
gameoverhumans 3y ago
I agree, it's a feature that I find sorely lacking in my tailnet. These are the relevant Github issues to follow, hopefully they address these someday: https://github.com/tailscale/tailscale/issues/465 https://github.com/tailscale/tailscale/issues/465 https://github.com/tailscale/tailscale/issues/4324 https://github.com/tailscale/tailscale/issues/4324

drewda 3y ago

We've been using https://github.com/mvisonneau/docker-tailscale/ https://github.com/mvisonneau/docker-tailscale/ on k8s clusters. Good to see an official option coming.

Axsuul 3y ago
Same however I'm using HashiCorp Nomad so hope for an official option there.

knodi 3y ago

Love it!!! This is going to make (my) dev and testing env cluster so much easier. Now imagine running derp-server with in the DC with your k8s.

Stem0037 3y ago

Does it support headscale? https://github.com/juanfont/headscale https://github.com/juanfont/headscale

hobofan 3y ago

I think an alternative solution would be nice, where services are just registered with their service discovery, though I suppose that would require them providing an official API for that part of their product. In my current setup for Tailscale + Kubernetes, I just use their subnet router[0] and add the kubedns server for the cluster.local domains to their MagicDNS. Having proper service discovery would make this into a nice round solutionl. [0]: https://tailscale.com/kb/1185/kubernetes/#subnet-router https://tailscale.com/kb/1185/kubernetes/#subnet-router