8 ms·
I finally understand Cloudflare Zero Trust tunnels
- plantinthebok 11mo agoWhat's the actual win here? Avoiding relay latency in the rare cases Tailscale can't punch through NAT? If that's it, a $3 VPS running Headscale seems simpler. The complexity feels like you're optimizing for the 5% case while adding permanent vendor lock in. What am I missing?
- throwaway678339 11mo agoI don't think you are missing anything. They have a bunch of half baked features like this that aren't as robust as real security vendors and lock you in just like you said.
- josteink 11mo agoMaybe I’m misunderstanding something… But are you accusing someone of promoting vendor lock-in (cloudflare) while at the same time promoting vendor lock-in (tailscale)? If you’re ok with vendor lock-in, shouldn’t you in theory be ok with any vendor?
- bingo-bongo 11mo agoHeadscale is the not-vendor-login version of Tailscale.
- uneekname 11mo agoSort of. Many tailscale clients you would use with headscale are closed source.
- fragmede 11mo agoNo. Not all vendors are equal. We can treat ProtonMail differently then Gmail, for example. Looking at what's gone down with VMware, definitely don't get in bed with Broadcom.
- killingtime74 11mo agoFor many homelabbers, just being cheap and avoiding the $3 VPS, that's it
- comrh 11mo agoI dont even pay anything, my tiny homelab is completely covered by the free tier
- kohbo 11mo agoWhat provider still has decent free tier?
- antonkochubey 11mo agoOracle. 4 vCPU, 24 GB RAM, 200 GB SSD. It’s arm64 but nowadays that doesn’t really matter.
- gear54rus 11mo agoWhich region were you able to create this in? They seem to be out of capacity all the time in EU.
- jnsaff2 11mo agoI have read that you need to write a script to constantly bombard their API in order to get one. I presume you'd be fighting other scripts.
- petronic 11mo agoWhat worked for me was handing them a credit card and transitioning myself out of the free tier. (I'd use the free credits they offer prior to doing this - they give you something like $300 immediately on signup.) The always-free infra remains free, you just have the chance of incurring a bill if you make selections that aren't free or exceed block storage/egress (200GB/10TB) limits of the always-free tier. Leaving the free/trial tier gives you access to a much larger pool of instances. I never successfully deployed an A1 instance prior to becoming a "paying" customer - now I've done it hundreds of times without ever having an issue. I've been running a small k0s cluster and a standalone webserver for months while incurring about $2.50 - $3 in spending each month, primarily from being slow to remove instance snapshots sitting in block storage. Even things that are oddly expensive on AWS - like NAT - are free on Oracle. There are zero gotchas.
- k_bx 11mo ago$3 VPS running Headscale is not simpler since you won't be able to run both headscale and tailscale on your end user machines, I don't recommend it. The solution we've found is running a white IP container (or VPS) which looks like regular Wireguard outside, while inside it "forwards" to your existing tailscale network. I don't remember if we use https://github.com/gravitl/netmaker https://github.com/gravitl/netmaker or https://github.com/juhovh/tailguard https://github.com/juhovh/tailguard Also see: https://tailscale.com/blog/peer-relays-beta https://tailscale.com/blog/peer-relays-beta
- zrail 11mo agoTailscale has what they call Peer Relays now to help solve this problem: https://tailscale.com/blog/peer-relays-beta https://tailscale.com/blog/peer-relays-beta
- codethief 11mo agoWhat does Headscale have to do with NAT hole punching? I believe what you actually mean is setting up a relay, see the link in https://news.ycombinator.com/item?id=45948806 https://news.ycombinator.com/item?id=45948806 .
- hexbin010 11mo agoThis seems like an excellent guide. I love these "how the pieces fit together" kind of guides. Perhaps CF could license it and slap it in their docs!
- jchw 11mo agoOne thing that makes Cloudflare worse for home usage is it acts as a termination point for TLS, whereas Tailscale does not. If you use a Tailscale Funnel, you get the TLS certificate on your endpoint. With Cloudflare, they get a TLS certificate for you, and then strip and optionally re-add TLS as traffic passes through them. I actually have no idea how private networks with WARP are here, but that's a pretty big privacy downgrade for tunneling from the Internet. I also consider P2P with relay fallback to be highly desirable over always relaying traffic through a third party, too. Firstly, less middlemen. Secondly, it continues working even if the coordination service is unavailable.
- keehun 11mo agoTLS termination is neither required nor enabled by default, right?
- crimsonnoodle58 11mo agoCorrect. We run it without it and just use the DNS filtering aspect.
- philipwhiuk 11mo agoHow does it do DNS filtering without TLS interception - takeover for DNS resolution?
- arcfour 11mo agoIn what way are DNS resolution and TLS related except for the little-used DoT?
- jchw 11mo agoFor tunnels many of the features basically have to work this way, so I'd be surprised if you could avoid it. It's also impossible to avoid if you use normal Cloudflare "protected" DNS entries. You can use Cloudflare as just a DNS server but it's not the default, by default it will proxy everything through Cloudflare, since that's kind of the point. You can't cache HTTP requests you can't see.
- yuvadam 11mo agoTailscale now has the awesome feature of peer relays and now there's no more excuses why you can't traverse that NAT and you can forget about all those DERP servers.
- qudat 11mo agoNice article. For easily exposing private services to the internet I’ve been using https://tuns.sh https://tuns.sh which lets you run ssh tunnels. It’s nice for a zero install solution.
- yegle 11mo agoFree Cloudflare account cannot be used to serve my Plex server. To me that's a no-go. The specific term is: https://www.cloudflare.com/service-specific-terms-application-services/#content-delivery-network-terms https://www.cloudflare.com/service-specific-terms-applicatio...
- mandelken 11mo agoMy free account runs cloudflared tunnel just fine for jellyfin. Is it just plex? I use the tunnel because my girlfriend cant install tailscale on her work laptop, so this way she can still login to jellyfin while traveling.
- 369548684892826 11mo agoIt’s not that it can’t do it, it’s that it’s against the terms of service to use it for video.
- stingraycharles 11mo agoIsn’t that for their CDN, rather than Zero Trust? I don’t think they even know what traffic is going over Zero Trust.
- mycall 11mo agoYou can't play corporate videos over Cloudflare Warp? Where is the line drawn?
- tombert 11mo agoI think they don't want you using a boatload of bandwidth, since data proxies through their servers.
- wink 11mo agoYes and that's what people are criticizing - it's just an arbitrary and thus very bad rule. Completely unrelated to CloudFlare, I streamed a single TV series from a friend's plex account 1-2y ago, that's less traffic than some of my friends use in 2 weeks. I'm not saying they can't have that rule, it's their infra - I'm just saying that "a boatload of bandwidth" can be anything, depending on who you ask.
- amluto 11mo ago> Then go into Cloudflare DNS settings and map the domain homeassistant.mydomain.com to the tunnel: > CNAME homeassistant.mydomain.com a2f17e27-cd4d-4fcd-b02a-63839f57a96f.cfargotunnel.com > Now all traffic going to this domain will go through the cloudflared tunnel, which is configured to route homeassistant.mydomain.com to 192.168.1.3. No Warp client needed, Argo tunnel does everything for us. It boggles my mind that Cloudflare ever considered this acceptable for production, let alone that this is still how tunnels work. The whole configuration scheme feels like something that someone might have kludged up as a technology demo and launched in a staging environment. But the fact that a very security sensitive production system where a “DNS” record that looks like a CNAME to a magic hostname causes traffic to get proxied and sent to a “Zero Trust” private network is just … unreal. It’s almost impossible to tell WTF is going on or what policies apply to what. Does Cloudflare’s proxy really try to fetch an upstream resource, notice that the configured domain name ends with “cfargotunnel.com” and invoke some special handling? What happens if, say, someone else adds that same CNAME to their own network? What if some route goes to foo.bar.com and foo.bar.com’s nameserver reports a CNAME to cfargotunnel.com? I’ve been using this product for several years, and the documentation and configuration pages have slowly evolved from abysmal to very slightly better. At least now it’s sort of clear how tunnels interact with strict TLS.
- pests 11mo agoThe cname is just a normal domain. That DNS entry is a real entry. The CNAME is real. You can go directly to that address too. If someone else knows the cname destination they could go to it or cname their own domain to it literally like any other domain. The only specially handling is cloud flare has a mapping from subdomain to your private network via it's agent and that's it. I don't get what's the wrong or complicated about this.
- amluto 11mo agoI gave you the benefit of the doubt for a moment, but as far as I can tell, you are incorrect for practical purposes. I went ahead and re-checked everything to make sure. Let's see: 1. I have a cloudflare domain with a working tunnel (managed through Access). In DNS Records, it shows as a CNAME to [redacted].cfargotunnel.com. But: $ dig [redacted].cfargotunnel.com ; <<>> DiG 9.10.6 <<>> [redacted].cfargotunnel.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5851 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 and no records are returned. Interestingly, it's an empty result, no NXDOMAIN. 2. I have multiple subdomains that appear to be CNAMEs to the same [redacted].cfargotunnel.net. And yet they are entirely different sites that just happen to share an instance of cloudflared at the origin. The sites aren't even served at the same origin address! They are different "Published Application Routes". They don't even have the same protocol! 2. The tunnel above is on a domain with "Full (strict)" TLS. But traffic to the origin emerges from cloudflared in cleartext. This whole configuration schema is nonsense. What should happen if a CNAME points at a tunnel that doesn't have a route for that application? What if a tunnel has a route for an application that is CNAMEd somewhere else? I imagine that what's going on is that Cloudflare internally has a rule that traffic with a cfargotunnel.com origin goes out their Tunnel infrastructure instead of out to the normal Internet. And Cloudflare applies the same JWT that it would apply if the request went out via the normal Internet, and cloudflared verifies that JWT if "Enforce Access JSON Web Token (JWT) validation" is on (maybe the request is literally TLS wrapped inside the cloudflared tunnel? I've never tried to inspect what's going on inside). And then cloudflared unwraps everything? And if you configure cloudflared wrong, then it's totally insecure?
- VladVladikoff 11mo ago[flagged]
- jorams 11mo ago> after frustration with Tailscale in environments where it couldn’t properly penetrate NAT/firewall and get a p2p connection, I decided to invest some time into learning something new: Cloudflare Zero Trust + Warp ...which doesn't even try to get a p2p connection. Instead you always get the thing you didn't want. If you're okay with that you could've just ignored how Tailscale connected those devices, that's kind of the point. You've also in the process converted your entire security model to Cloudflare's idea of "Zero Trust" which involves 100% trusting Cloudflare. The rest of the blog post is fine, but the motivation is honestly baffling.
- rainsford 11mo agoIs the connection through Cloudflare still encrypted between the two peers, as it would be going through a Tailscale relay? If not, that's definitely a downgrade using the Cloudflare approach. But if not, I'm not sure the trust model is significantly different with maybe the added benefit of the fact that Cloudflare's relay performance is likely better given that relaying traffic is kind of their main thing rather than a very secondary function like it is for Tailscale. On the other hand, my experience with Tailscale is that they're very, very good at NAT hole punching and I'd rather have a direct connection where possible from a latency standpoint.
- stingraycharles 11mo agoThe difference is that Cloudflare has their own high quality network and PoPs everywhere, so the quality is generally even better than P2P. This is my experience, we are a fully remote world-wide company and we recently migrated away from Tailscale to Cloudflare and it has been much better.
- o11c 11mo agoEh, kind of. Peering in Europe is such a mess that even Cloudflare can be pretty bad. Sometimes you have to manually calculate "okay, there's a colo in this particular city that will force the correct route if we proxy all our traffic through it ..."
- kennethallen 11mo agoI don't understand the use case here. Is this supposed to be for enterprise to control access to internal applications via network access policies?
- qball 11mo agoYes, this is exactly what this does.
- redrix 11mo agoYes. The acronym is “ZTNA” (Zero Trust Network Access). It is an alternative to a traditional corporate VPN that addresses a few architectural issues; namely: - L3 connectivity (which allows for lateral movement) to the corporate network. - Inbound exposure to the VPN gateway (scaling can become a challenge, not to mention continuous vulnerabilities from… certain vendors) - Policy management can get convoluted if you want to do micro-segmentation properly. ZTNA is essentially an “inside-out” architecture and acts (kind of) like a L4 proxy. I’m going to butcher this explanation, but: 1. Company installs apps/VMs/containers throughout their network. These must have network reachability to the internal apps/services the company wants to make available to its users. 2. These apps/VMs/containers establish TLS tunnels back to the company’s tenant in the vendor’s cloud. 3. Company rolls out the vendor’s ZTNA client to user devices. This also establishes a TLS tunnel to the vendor’s cloud. Hence the vendor’s cloud is like a MitM gatekeeper. 4. Company creates policies in the vendor’s cloud that says “User A can access App X via app/VM/container Z” 5. Even if App X is on the same LAN segment as App Y, App Y is invisible to User A because connectivity to the internal apps happens at L4. It is an interesting architecture. That being said, ZTNA solutions have their own issues as well (you can probably already spot some based on my explanation above!) (Note: I worked for a security vendor that sold a ZTNA solution as part of their ~4-5 years ago. Things could be different now.)
- siwatanejo 11mo agoBut is this vendor-lockin, as in CloudFlare being the vendor here? Because at least with Tailscale there's no vendor lock-in.
- siwatanejo 11mo agoActually, seems TailScale is also a vendor? huh, and I thought it was an opensource project...
- wg0 11mo agoTailscale is almost open source with Wireguard itself being open source. - Most of the clients are open source probably. - Tailscale allows you to run custom control server of your own. - One open source control server "headscale" is sponsored by Tailscale themselves.
- jijji 11mo agowhat's the difference between this and a reverse SSH tunnel, for example making a local port on your laptop accessible to a public-facing internet server or even running on localhost on that same server... or using sshuttle to access your local network from a remote server .... it doesn't sound like "zero trust" if you're proxying everything through some third-party company that you know nothing about what they're doing with your actual data that you're sending across the wire...
- ghoshbishakh 11mo agoZero trust is a marketing term used by them - surprisingly it has nothing to do with end-to-end encryption also.
- iku 11mo agoThanks a lot. Both the post itself and the comments are very useful. I'd like to comment on the content of this post, but it'll take much time for me to wrap my head around. In the mean time: the images in the article seem to be broken — produce 404 errors. Like this one: https://david.coffee/targets-config-screen.png https://david.coffee/targets-config-screen.png
- suckow 11mo agoOh man, someone has to talk about this!! Cory told me about CF's gold issues and it really does seem problematic to me, I'm glad ZT is finally being criticised.
- jumski 11mo agoI'm using Netbird [0] for my home / private needs: - Synology NAS - All the laptops and desktops my family uses - All family mobile phones Given i work in Tmux, its super convenient to take a laptop with me and just use it as a thin client to my Desktop wherever I am. [0] https://netbird.io/ https://netbird.io/
- mrbluecoat 11mo agoI stopped reading at "everything goes through the Cloudflare network, no direct p2p" https://github.com/alecbcs/hyprspace https://github.com/alecbcs/hyprspace has penetrated every NAT I've ever encountered. No megacorporation required.
- 8organicbits 11mo agoThat project appears abandoned and unmaintained.
- sylens 11mo agoI've experimented with Cloudflare tunnels before to sit in front of my Immich instance in my homelab. Only issue is the 100MB upload size for videos. But Immich added upload chunking support to their roadmap so its possible this will work very well in the future.
- ranguna 11mo agoImmich also has the ability to use different domains for different networks. Meaning that I connect directly to my server when I'm connected to my local home network and connect through cloudflare when I'm out of my house. This way I can upload big videos when I get home.
- youngbum 11mo agoBig fan of Cloudflare Tunnel here, too. We use our Windows workstations as WSL SSH tunnels, protected with email verification (only for our domain), and it’s been working perfectly. I’m curious, though, about how we can expose Docker services. It would be fantastic to have a remote build server set up with Cloudflare Tunnel.
- HenriTEL 11mo agoWith that it becomes clear that some service is self hosted (the DNS record points to a private IP). It can be a security issue when the Whois record or the domain name allows the identification of the hosting entity. Finding its physical address can be an easy task depending on its social presence. Then probably the hosting place is an easier target than a data center.
- favflam 11mo agoDon't ISPs now provide ipv6 addresses? Why not just connect directly home via ipv6 address. I think many ISPs in Asia where ipv4 addresses are scarce have been moving to MAP-e, which is ipv6 centric. I don't see why I want to loop in a 3rd party to connect back to my house.
- spiffytech 11mo agoMy AT&T U-Verse router does IPv6 but won't forward unsolicited IPv6 traffic to local devices :(