8 ms·
NAT between identical networks using VRF
- d-z-m 4y agoRelated: https://www.netfilter.org/documentation/HOWTO/netfilter-double-nat-HOWTO-1.html https://www.netfilter.org/documentation/HOWTO/netfilter-doub...
- hummus_bae 4y ago[dead]
- tarasglek 4y agoThis is a very real problem when using a VPN into another network with same ip-space. Though I'm not sure the complexity is worth it. Suppose it would be nice if tailscale implemented this type of substitution-routing-NAT this for users.
- dilyevsky 4y agoI mean they kinda do by assigning each node a cgnat space IP?
- tarasglek 4y agoThey have NAT routing support for exposing nodes that don't/can't run tailscale. Meant that ip space
- zamadatix 4y agoAnother possible option for the VPN use case, depending on what you're trying to do and how you're doing it, is to just put the VPN into the VRF (or a netns) and only bind connections you intend to run through the VPN to that context. It's not particularly fantastic either but it's a little more straightforward.
- bob1029 4y ago> This is a very real problem when using a VPN I've been really careful about selecting virtual network IP spaces throughout due to this concern. The hardest part is accounting for the point-to-site VPNs on top of everything else (i.e. employees connecting from home/Starbucks/airport). For internal corporate networks, I typically will provision smaller targeted networks in order to get around this. Lopping-off all of 10.0/16 or 192.168/16 for your internal LAN will put you at a fairly high statistical likelihood of overlapping domestic ISP address spaces. Smaller, odd networks like 192.168.111/24 are much easier to plan around.
- m463 4y agoOne network I used allocated from 172.16.0.0 – 172.31.255.255 but then docker wouldn't work if you were in 172.17
- rudasn 4y agoAh what a fun issue to figure out and resolve 10 minutes before endofday,especially if you are the one handling docker and some dude in another country is handling the vpn that wants access to said docker.
- deleted 4y ago[deleted]
- rzzzt 4y agoFor me, this was WSL2's virtual machine and (IIRC) not the default bridge network, but the next one created by docker-compose.
- jrockway 4y agoI ran into this problem a long time ago with EKS. I picked a network for EKS, but it also needed a network for docker running on each node, and this happened to conflict with a network that we had peered to AWS (we were an ISP, this was our internal management network). I learned on that day to put everything in our IP address management system (netbox). Allocate private networks like we allocated public networks to our customers. (It would have been nice if I allocated Docker's defaults before they had been allocated to an internal network, but ... hindsight 20/20. At least with that network in the IPAM system, people could figure out why it broke though.)
- linsomniac 4y agoTailscale does do something along these lines, but it's kind of surprising... If there is a tailscale route announced, it will take precedence over other routes, even if they are more specific. The primary use case for this seems to be if you are at a coffee shop, you can access a tailscale routed network that has the same IP range as the coffee shop. The down side of this is that if you have, say, a wider announced network routed over your tailscale (say 10.0.0.0/16), and then you have machines within that network also connect to the tailscale (say a machine with 10.0.1.1/24), it now will try to reach machines on its local network interfaces via routing over the tailnet. Pretty unexpected. https://github.com/tailscale/tailscale/issues/6231 https://github.com/tailscale/tailscale/issues/6231
- WirelessGigabit 4y agoWe had this issue with Docker, which uses 172.19.0.0/16 for the default bridge. You can change it but it was weird figuring that out the first 2 months.
- tecleandor 4y agoTailscale does a fun thing that's mapping overlapping ipv4 networks into different ipv6 networks: https://tailscale.com/kb/1201/4via6-subnets/ https://tailscale.com/kb/1201/4via6-subnets/
- staringback 4y agoCan we let IPv4 and NAT die already?
- greyface- 4y agoNo. Even in IPv6-only-land, multihomed networks without PI space will have good reason to NAT.
- ArchOversight 4y agoIf you generate a ULA address using the algorithm that is recommended the likelihood of a collision in IPv6 addressing space in networks is absolutely miniscule.
- greyface- 4y agoIf you're using ULAs, you don't NAT to avoid addressing collisions; you NAT so that your traffic is routable on the Internet. If you try to pass traffic sourced from a ULA to your upstream without NAT, it or its response is going to get dropped on the floor.
- ArchOversight 4y agoYou wouldn't route traffic over the NAT using ULA to the outside world. You'd use GUA space for that. Collisions between two private networks is very low was my primary point, and thus NAT is not a thing that needs to exist.
- greyface- 4y agoYes, exactly. And if you have two upstreams, there's no single GUA prefix that makes sense to use in all situations. You make your routing decision, then you NAT (er, sorry, NPTv6, which is Totally Not The Same Thing As NAT) to the GUA prefix corresponding with the network that you're egressing from. If you don't need Internet connectivity, yes, NAT-free ULAs work fine.
- dilyevsky 4y agoTl;dr: remap each side to a different /24 using iptables and vrf device. If you dont want to mess around with iptables this can also be trivially achieved using OVS
- ryanschneider 4y agoIf each network also had unique dns services, then I think in theory you could run a split horizon DNS server on the middleman that rewrites results from the other network as the “alt” address. E.g. in the example middleman could rewrite a query for `outernode0` that comes from the inner node network to resolve to 192.169.3.10 (instead of .2.10). I’m not sure which dns servers allow rewriting logic like that though.
- zamadatix 4y agoThe usual suspect, BIND, has a feature called Response Policy Zones that can serve different responses to different ACL matches (such as source IP). It can even autogenerate the modified records if you ask it politely. Of course a screwdriver also has a feature called Quick Eyeball Removal which may be preferable. That or I'm just bitter about having done this :).
- 3np 4y agoYou can also achieve similar goals in dnsmasq via --localise-queries and --dynamic-host. It's a bit clunky but not complicated.
- Already__Taken 4y agoI run coredns as internal dns and the advice for this solution from them is to run another coredns and solve it in config management. While not a pleasing answer its probably the easiest thing to do. Someone did write a plugin but it requires a fork to get some hooks and that was declined its PR.
- supriyo-biswas 4y agoI am piggybacking my question on this discussion: as a beginner to Linux networking, are there good resources one can learn from (such as the use of ip link for route management as discussed in the article?)
- faragon 4y agoCheck the FRR project. https://frrouting.org/ https://frrouting.org/
- andix 4y agoWho are this powerful sorcerers that implement those network features? It’s one thing to figure out how to do that, but there must’ve been somebody who crafted all those components first. Truly impressed.
- KaiserPro 4y agoSo, whilst possible to do, and this blog is a brilliant guide to doing it, but there are better ways. if this is a network you own either end, then if you really do have the same subnets issued to different sites, then you would be better served having virtual interfaces/VLANs to have a second IP address. Or, just use ipv6, and you have enough IPs to do what you want. You can use 6to4 or nat64 to get IPv4 connectivity. most modern networks terminated services on the edge with loadbalancers, so you can use them as 6->4 bridges.
- yrro 4y agoVery cool. Surprised nftables doesn't support NETMAP yet though!
- MrStonedOne 4y ago[dead]
- thedougd 4y agoIf you want to do VRF on Linux and don’t have a philosophical objection to systemd, I strongly recommend using systemd-networkd. VRF, wireguard, and many other complicated things are easy to configure out of the box and don’t require extra scripts or custom units. My home router is a Raspberry Pi running Ubuntu server. VRF, Wireguard, VLANs, bonding, llrp, route advertisement, prefix delegation, dhcp server and leases, all configured only with systemd-networkd.
- pm2222 4y agoNetwork overlap alright. If the hosts do not overlap I bet you can enable end to end comm with just /32 host routes
- solotronics 4y agoAll of this just because people didn't want to figure out IPv6!
- bogantech 4y agoAny system connecting to the internet will need IPv4, if you go down the IPv6 route you're going to have to maintain a dual-stack infrastructure (with host, firewall, rotuer configs etc) and get at least 2x the amount of headaches whenever anything goes wrong. What is the business case for all that extra pain?
- yrro 4y agoI was looking for some information on how VRFs compare to Linux network namespaces & found https://www.toddpigram.com/2017/03/vrf-for-linux-contribution-to-linux.html https://www.toddpigram.com/2017/03/vrf-for-linux-contributio... which does a good job of explaining why it makes sense to use VRFs even though there is some functionality overlap with network namespaces. I also found some information about using VRFs in the documentation for RHEL 8: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/reusing-the-same-ip-address-on-different-interfaces_configuring-and-managing-networking https://access.redhat.com/documentation/en-us/red_hat_enterp...