4 ms·

You can additionally set up ULA: https://en.wikipedia.org/wiki/Unique_local_address https://en.wikipedia.org/wiki/Unique_local_address The way I do this, my in

You can additionally set up ULA: https://en.wikipedia.org/wiki/Unique_local_address https://en.wikipedia.org/wiki/Unique_local_address The way I do this, my internal DNS resolves hosts to their fixed ULA addresses. For the handful that are accessible externally, public DNS resolves to their address on the current public prefix.

herczegzsolt 1y ago
I did try that, but it ended in an infinite fight with the source address selection algorithm and DNS caches. Also, unique-local addresses are deprecated as far as I know.
- simoncion 1y ago
  > ...it ended in an infinite fight with the source address selection algorithm and DNS caches... What did this fight look like? For the past fifteen, twenty years, I have NATted IPv4, globally-routable IPv6, and ULA IPv6 addresses on all of my machines attached to the internet-accessible VLANs on my LAN. The only trouble I've noticed was when ISP maintenance caused me to lose the globally-routable prefix for a little while and my franken-router started passing ULA traffic out the WAN interface. [0] I'd love to hear what you've been seeing, so I can see if there's trouble that I've been overlooking. > ...unique-local addresses are deprecated as far as I know. ULAs are not deprecated. You may be thinking of site-local addresses. See the first paragraph of section 1 here: [1] [0] The obvious firewall rule fixed that. [1] <https://www.ietf.org/archive/id/draft-ietf-v6ops-ula-usage-considerations-05.html https://www.ietf.org/archive/id/draft-ietf-v6ops-ula-usage-c...>
  - herczegzsolt 1y ago
    1) DNS caching: as devices roam in and out of my network, they don't always evict the cache and remember the wrong address. When trying to connect to ULA from outside, that's not going to work. When trying to connect to the public address from the inside, I face a firewal/config problem. Either I treat them as "outsider" and block/limit them incorrectly, or I need to dynamically update the setup to know what is internal. 2) DNS over HTTPS: which is forced by more and more things by default. They will always resolve to the outside address, even when on the local network. Causing the same problem as caching. 3) Source addr selection "stuck" on ULAs: When I loose the public prefix for a brief period, things tend to start using the ULA as a source for public destinations. This is not going to work as expected. However, when I get a new prefix, some things tend to be stuck for a long time on trying to use the ULA instead of using the new prefix. 4) I've seen devices incorrectly keep a ULA address from a different network, and attempting to use that on mine. This is definitely a bug in the device, but now it is my problem. 5) A variant of issue 4. is that some apple devices (TVs and similar) will just make up and start advertising their own ULA prefix. This is related to Matter support. They should detect that there is already a ULA prefix and use that, but this detection is far from perfect, so sometimes they just do it anyway. Now my whole network has two ULAs. All devices should still use their "nearest" address to communicate, but - bugs again - sometimes they don't do that. But hey, apple provides me with free pentesting to see if I have the RA filtering correctly setup in switches :-) --- The list goes on, this is just what came to my mind immediately. There are really 3 conclusions I came to: - Split-brain DNS is an outdated concept, being broken every day by new tech. You can't rely on the devices talking to the specific NS you want. - There are a lot of buggy IPv6 implementations out there. Unless I control all devices on the network, I need to keep the advertised config very basic to keep everything working. - Dual-stack and happy-eyeballs hide these issues too well. It is hard to detect and report them. Even if you do, vendors often just don't care or bother fixing because the issues do not have an immediate impact.
    - simoncion 1y ago
      > 3) Source addr selection "stuck" on ULAs So, I think #3 will be solved by rejecting traffic from fc00::/7 that's going to be routed out the router's WAN interface. To do this on my network, I have this on the 'filter' table of my router firewall: -A FORWARD -s fc00::/7 -o wan-interface -j REJECT This should cause machines that attempt to use the only valid prefix during the outage to have their attempts to establish Internet connections during the outage actively rejected, rather than quietly dropped by your ISP. That will almost certainly cause them to use the globally-accessible prefix when it comes back up. You probably also need to do the same for RFC1918 and RFC3927 addresses on your IPv4 firewall... just in case. > 1) ... When trying to connect to the public address from the inside, I face a firewal/config problem. Either I treat them as "outsider" and block/limit them incorrectly... I'm fairly confused about why you need to treat them as an "outsider"? Same-subnet communications do not go through the router; hosts contact each other directly... so you're talking about firewalls running on each host, right? What are you doing to traffic destined to the globally-reachable address to treat it as an "outsider", and why? [0] > 4) ... This is definitely a bug in the device, but now it is my problem. How is this your problem? Your router won't have a route for traffic from that source address and will either drop or reject it... which is exactly the same as if a device erroneously retained an RFC1918 address in a subnet that isn't used on your LAN. [1] Are you seeing different behavior? > 5) A variant of issue 4. is that some apple devices (TVs and similar) will just make up and start advertising their own ULA prefix. That's pretty stupid behavior from the TV, but it's exactly the same sort of problem as the TV setting up a DHCP server on the LAN. It should not be doing that, and should be doing communications over the addresses already assigned to the interface (even the link-local address would work!). Do the RAs sent by the TV at least set the preference of the router to "low"? [2] > Unless I control all devices on the network, I need to keep the advertised config very basic to keep everything working. I'm curious about what things you put in the advertisement (other than DNS server) that didn't work. > Split-brain DNS is an outdated concept, being broken every day by new tech. Meh. Keep the TTL for your internal hostnames very low (60 seconds or less) and any problems you might have go away quickly. You're not serving a lot of DNS traffic, and it is all local, so the low TTLs won't be a problem. [0] For services that I don't want to be Internet-reachable I either block the ports at the router (for things like NFS which cannot be bound to certain IPs), or I configure the daemon to listen on my ULA address. If you use "privacy" addresses, and the daemon doesn't permit bind addresses expressed as a subnet (e.g. fd06:0:0:1::/64), then this will probably be difficult to do. I recommend NOT using "privacy" addresses for your daemon-running machines. [1] It's actually better than the typical "Apple devices fail to renew their DHCP lease on wake" situation with IPv4, as the chances of duplicate IP addresses (and resulting ARP/ND flapping) are effectively zero. [2] Though, it's an Apple device, so I bet they set the preference to "high" and have an infinite lifetime for the prefix. XD
throw0101d 1y ago
Note that currently with ULA if you have dual-stack IPv4 will be given priority over ULA. There is a late-stage—Submitted to IESG for Publication—draft that will change this: * https://datatracker.ietf.org/doc/html/draft-ietf-6man-rfc6724-update https://datatracker.ietf.org/doc/html/draft-ietf-6man-rfc672...
- CaliforniaKarl 1y ago
  Ah, thanks for posting that, I was wondering how things were going there.
- clysm 1y ago
  More than just IPv4 priorities, almost all other IPv6 addresses are given higher priority which makes routing between ULAs on an internal network problematic. That draft doc seems to fix multiple problems at once.
dogcow 1y ago
Doesn't using ULAs kind of defeat the purpose (or one of the main intents) of IPv6, which is every device having a globally rotatable IP address? It kind of puts us right back in the IPv4 with NAT situation, only with longer, uglier addresses. I personally think it is absurd that the ISPs that do actually support IPv6 are being so difficult and stingy about assigning static v6 prefixes.
- RiverCrochet 1y ago
  IPv4/NAT is not the only "to get to system X you must pass through system Y" scenario. Example: You have a bastion host that is Internet-accessible, and it has one or more server behind it you only want accessible "through" the bastion host. The bastion host might be running nginx and reverse proxying multiple servers behind it, and this host is doing caching in addition to WAF and some other stuff. So this bastion host would have at least 2 NICs, one for the Internet-facing connection and one or more where servers exist on a non-public LAN. The small network(s) connecting these servers to the bastion host can use a ULA and thus be guaranteed to not be globally routable. Link-locals are suboptimal because since they are link local, they only have to be unique per link. This means some commands insist you specify interface name with the LLA, e.g. fe80::aaaa%eth1.

herczegzsolt 1y ago

I did try that, but it ended in an infinite fight with the source address selection algorithm and DNS caches. Also, unique-local addresses are deprecated as far as I know.

simoncion 1y ago
> ...it ended in an infinite fight with the source address selection algorithm and DNS caches... What did this fight look like? For the past fifteen, twenty years, I have NATted IPv4, globally-routable IPv6, and ULA IPv6 addresses on all of my machines attached to the internet-accessible VLANs on my LAN. The only trouble I've noticed was when ISP maintenance caused me to lose the globally-routable prefix for a little while and my franken-router started passing ULA traffic out the WAN interface. [0] I'd love to hear what you've been seeing, so I can see if there's trouble that I've been overlooking. > ...unique-local addresses are deprecated as far as I know. ULAs are not deprecated. You may be thinking of site-local addresses. See the first paragraph of section 1 here: [1] [0] The obvious firewall rule fixed that. [1] <https://www.ietf.org/archive/id/draft-ietf-v6ops-ula-usage-considerations-05.html https://www.ietf.org/archive/id/draft-ietf-v6ops-ula-usage-c...>
- herczegzsolt 1y ago
  1) DNS caching: as devices roam in and out of my network, they don't always evict the cache and remember the wrong address. When trying to connect to ULA from outside, that's not going to work. When trying to connect to the public address from the inside, I face a firewal/config problem. Either I treat them as "outsider" and block/limit them incorrectly, or I need to dynamically update the setup to know what is internal. 2) DNS over HTTPS: which is forced by more and more things by default. They will always resolve to the outside address, even when on the local network. Causing the same problem as caching. 3) Source addr selection "stuck" on ULAs: When I loose the public prefix for a brief period, things tend to start using the ULA as a source for public destinations. This is not going to work as expected. However, when I get a new prefix, some things tend to be stuck for a long time on trying to use the ULA instead of using the new prefix. 4) I've seen devices incorrectly keep a ULA address from a different network, and attempting to use that on mine. This is definitely a bug in the device, but now it is my problem. 5) A variant of issue 4. is that some apple devices (TVs and similar) will just make up and start advertising their own ULA prefix. This is related to Matter support. They should detect that there is already a ULA prefix and use that, but this detection is far from perfect, so sometimes they just do it anyway. Now my whole network has two ULAs. All devices should still use their "nearest" address to communicate, but - bugs again - sometimes they don't do that. But hey, apple provides me with free pentesting to see if I have the RA filtering correctly setup in switches :-) --- The list goes on, this is just what came to my mind immediately. There are really 3 conclusions I came to: - Split-brain DNS is an outdated concept, being broken every day by new tech. You can't rely on the devices talking to the specific NS you want. - There are a lot of buggy IPv6 implementations out there. Unless I control all devices on the network, I need to keep the advertised config very basic to keep everything working. - Dual-stack and happy-eyeballs hide these issues too well. It is hard to detect and report them. Even if you do, vendors often just don't care or bother fixing because the issues do not have an immediate impact.
  - simoncion 1y ago
    > 3) Source addr selection "stuck" on ULAs So, I think #3 will be solved by rejecting traffic from fc00::/7 that's going to be routed out the router's WAN interface. To do this on my network, I have this on the 'filter' table of my router firewall: -A FORWARD -s fc00::/7 -o wan-interface -j REJECT This should cause machines that attempt to use the only valid prefix during the outage to have their attempts to establish Internet connections during the outage actively rejected, rather than quietly dropped by your ISP. That will almost certainly cause them to use the globally-accessible prefix when it comes back up. You probably also need to do the same for RFC1918 and RFC3927 addresses on your IPv4 firewall... just in case. > 1) ... When trying to connect to the public address from the inside, I face a firewal/config problem. Either I treat them as "outsider" and block/limit them incorrectly... I'm fairly confused about why you need to treat them as an "outsider"? Same-subnet communications do not go through the router; hosts contact each other directly... so you're talking about firewalls running on each host, right? What are you doing to traffic destined to the globally-reachable address to treat it as an "outsider", and why? [0] > 4) ... This is definitely a bug in the device, but now it is my problem. How is this your problem? Your router won't have a route for traffic from that source address and will either drop or reject it... which is exactly the same as if a device erroneously retained an RFC1918 address in a subnet that isn't used on your LAN. [1] Are you seeing different behavior? > 5) A variant of issue 4. is that some apple devices (TVs and similar) will just make up and start advertising their own ULA prefix. That's pretty stupid behavior from the TV, but it's exactly the same sort of problem as the TV setting up a DHCP server on the LAN. It should not be doing that, and should be doing communications over the addresses already assigned to the interface (even the link-local address would work!). Do the RAs sent by the TV at least set the preference of the router to "low"? [2] > Unless I control all devices on the network, I need to keep the advertised config very basic to keep everything working. I'm curious about what things you put in the advertisement (other than DNS server) that didn't work. > Split-brain DNS is an outdated concept, being broken every day by new tech. Meh. Keep the TTL for your internal hostnames very low (60 seconds or less) and any problems you might have go away quickly. You're not serving a lot of DNS traffic, and it is all local, so the low TTLs won't be a problem. [0] For services that I don't want to be Internet-reachable I either block the ports at the router (for things like NFS which cannot be bound to certain IPs), or I configure the daemon to listen on my ULA address. If you use "privacy" addresses, and the daemon doesn't permit bind addresses expressed as a subnet (e.g. fd06:0:0:1::/64), then this will probably be difficult to do. I recommend NOT using "privacy" addresses for your daemon-running machines. [1] It's actually better than the typical "Apple devices fail to renew their DHCP lease on wake" situation with IPv4, as the chances of duplicate IP addresses (and resulting ARP/ND flapping) are effectively zero. [2] Though, it's an Apple device, so I bet they set the preference to "high" and have an infinite lifetime for the prefix. XD

throw0101d 1y ago

Note that currently with ULA if you have dual-stack IPv4 will be given priority over ULA. There is a late-stage—Submitted to IESG for Publication—draft that will change this: * https://datatracker.ietf.org/doc/html/draft-ietf-6man-rfc6724-update https://datatracker.ietf.org/doc/html/draft-ietf-6man-rfc672...

CaliforniaKarl 1y ago
Ah, thanks for posting that, I was wondering how things were going there.
clysm 1y ago
More than just IPv4 priorities, almost all other IPv6 addresses are given higher priority which makes routing between ULAs on an internal network problematic. That draft doc seems to fix multiple problems at once.

dogcow 1y ago

Doesn't using ULAs kind of defeat the purpose (or one of the main intents) of IPv6, which is every device having a globally rotatable IP address? It kind of puts us right back in the IPv4 with NAT situation, only with longer, uglier addresses. I personally think it is absurd that the ISPs that do actually support IPv6 are being so difficult and stingy about assigning static v6 prefixes.

RiverCrochet 1y ago
IPv4/NAT is not the only "to get to system X you must pass through system Y" scenario. Example: You have a bastion host that is Internet-accessible, and it has one or more server behind it you only want accessible "through" the bastion host. The bastion host might be running nginx and reverse proxying multiple servers behind it, and this host is doing caching in addition to WAF and some other stuff. So this bastion host would have at least 2 NICs, one for the Internet-facing connection and one or more where servers exist on a non-public LAN. The small network(s) connecting these servers to the bastion host can use a ULA and thus be guaranteed to not be globally routable. Link-locals are suboptimal because since they are link local, they only have to be unique per link. This means some commands insist you specify interface name with the LLA, e.g. fe80::aaaa%eth1.