6 ms·

Don't pick weird subnets for embedded networks, use VRFs

122 points by LorenDB 1y ago

mindslight 1y ago
The real official answer is to register/allocate a new subnet, with no intention of putting it into the global routing tables. IPv6 only comes into play because doing that with IPv4 is mostly impractical these days. The author lost me when they got into raw iproute commands. Not because I'm not acquainted (I run my own custom complex router using a standard Linux distro). But rather if someone knows enough to configure things at this level, then they would just come to this solution on their own. Most people trying to solve this problem will not - eg think that mobile video rack belonging to a touring musician. Readily-accessible solutions I can come up with off the top of my head: 1. Two off the shelf routers and double NAT. The middle network can be changed if it conflicts with the outer network 2. One router/NAT, but two IP networks on the inner network - one statically assigned for devices to communicate with each other, and one assigned via DHCP for accessing the horizon through NAT. That second network can then easily be changed. 3. Play battleship more strategically using class E address space, DOD/BigCo address space, and/or smaller subnets in the middle of the customary size for a range (eg 192.168.1.160/27).
- MartijnBraam 1y ago
  Allocating a subnet is way further away from reality for most people than configuring one router feature on the router they're using. There's also a lot of people that configure these devices (or linux routers) themselves but have never heard of VRFs, you got to learn about them somewhere so I just hope this helps some people :)
  - mindslight 1y ago
    I see your point if someone is at the level of tinkering enough to learn Mikrotik gear. I was loosely equating VRF with general Linux policy routing, where you end up owning a bit more of an overarching config with fwmarks etc. And then I reasoned that Mikrotik was more complex than that, because I personally avoid doing config on my Mikrotik devices in favor of the Linux router (which is the opposite if you're coming at it unopinionated). But if someone wants to understand just enough networking to copy and paste Mikrotik examples, I do think your post is good general suggestion for that.
- stirfish 1y ago
  I've been using a router as 4.20.69.1. It's good to hear other solutions, as I've just been figuring it all out as I go along
  - JSR_FDED 1y ago
    I’ve never had any issues using a .666 subnet
- master_crab 1y ago
  If I can avoid Double NATing, I do because it tends to degrade network performance and can have interesting characteristics on some inbound traffic (although that depends on the use case). But the third option honestly isn’t recommended enough. DoD space is rarely routable, and if you are on a private network already, even moreso. It’s also less common than RFC1918. However, there is one caveat. Some large corporates do use it for just the same reason. Even though it is rarer than 10 or 172 space, you’d be surprised how many large orgs do run DoD internally. (Disclaimer: I use DoD space for my travel router at hotels)
- 1oooqooq 1y ago
  can't they just put each port on it's own vlan and call it a day?
McNulty2 1y ago
As a network engineer it was interesting to read about VRFs and routing from the perspective of an A/V engineer. Thanks
dlcarrier 1y ago
It's still difficult to find equipment and ISPs that reliably work with IPv6. I'm sure it will be ready for prime time in the next decade or two, right when we get fusion power plants, self driving cars that don't require supervision, and the AI singularity.
- whatevaa 1y ago
  We will never get rid of ipv4. Apparently my country ISPs have more ipv4 than they need (they actually rent them to others), so they just don't bother with ipv6.
  - cassianoleal 1y ago
    I guess when the world moves away from IPv4, your country will have the entire IPv4 space for itself, and massive headaches with NAT to route out, and the relative impossibility to route in.
    - sieabahlpark 1y ago
      [dead]
- john01dav 1y ago
  I think that government intervention here is needed in order to coordinate people. We'll all be better off once ivp4 is dead and buried, but the incentive for each entity to do so is minimal. It could be some other powerful entity instead of government, for example tier 1 ISPs degrading ipv4 performance substantially or Google delisting websites that don't support ipv6 (and eventually delisting websites that do support ipv4).
  - dlcarrier 1y ago
    Really? The network protocol someone uses is so critical that it should be illegal to use the wrong one? What happens when IPv9 becomes the hot new thing, but everyone is stuck on IPv6, because of some outdated government regulation? Do Americans have the right to use whatever internet protocol they want, under the first amendment? IPv6 has made enough progress that it's totally possible to run your network off of it, regardless of what everyone else is doing, and if all of your neghbors are using IPv4, it won't harm your IPv6 network. Also, part of the delay in the switch to IPv6 is that some work is needed to ensure that home routers and IoT devices default to reasonable security settings, and the absolute worst thing to do is force them to switch first, and figure out security later. The answer isn't to force everyone to use something before its ready; the answer is to address every impediment, so it's worth it for everyone to switch. Sure it's slower, but it's much better than making users worse off by switching, converting them to detractors instead of supporters.
1718627440 1y ago
Maybe I'm dumb, but how do hosts from different subnets address each other, when they get the same IP addresses?
- majormunky 1y ago
  In the VRF case they can't: "This comes with a tradeoff of course and in this case is that you no longer can reach devices on the venue network, which shouldn't be a problem if you're only connected there for internet connectivity."
  - 1718627440 1y ago
    Ok, but why want you to connect to local networks, when they can't reach each other, wouldn't you just then connect to the upstream router instead?
    - SoftTalker 1y ago
      That's typically all you care about, the upstream router/gateway so you can get to the internet. But it's on the local network, so you need to connect to it.
deleted 1y ago
[deleted]
treesknees 1y ago
I’ve encountered this issue with Docker. They randomly assign an RFC1918 subnet for internal services, and occasionally, this subnet clashes with a genuine corporate network or IP address. This can be quite frustrating to troubleshoot, especially if you’re not involved in Docker networking on a daily basis.
- theideaofcoffee 1y ago
  I've also run into this issue a lot, especially in AWS when developers with less networking experience start randomly picking VPC subnets (why they were doing that is a whole 'nuther issue). They wonder why connectivity in docker on their ec2 hosts doesn't work, by default docker chooses 172.17/16 which seems to be a common choice for VPC subnets. Annoying, but a teachable moment nonetheless.
jcalvinowens 1y ago
I've always been able to solve this problem more simply using IPv4 link-local addressing (https://www.rfc-editor.org/rfc/rfc3927 https://www.rfc-editor.org/rfc/rfc3927), it doesn't matter if somebody abuses the same range because the outbound interface is explicitly specified in sin6_scope_id passed to connect() and bind(). But if you're writing code at a higher level than the bsd sockets api, it's a whole can of worms.
champtar 1y ago
2 big address block that have few chances of conflict: - CGNAT 100.64.0.0/10 - "Benchmark" 198.18.0.0/15
- telotortium 1y ago
  CGNAT is used by Tailscale and presumably in the wild for its intended purpose.
  - Lammy 1y ago
    And `100.115.92.0/23` is used by ChromeOS for PatchPanel: https://chromium.googlesource.com/chromiumos/platform2/+/main/patchpanel/address_manager.cc?autodive=0%2F#23 https://chromium.googlesource.com/chromiumos/platform2/+/mai...
londons_explore 1y ago
I think most regular off the shelf consumer WiFi routers can already do this. The "192.168.0.1" ip address being the internal network and the external network because you're chaining routers together is awfully common and consumers expect it to just work without needing to understand IP addresses like a nerd.

122 points by LorenDB 1y ago

The real official answer is to register/allocate a new subnet, with no intention of putting it into the global routing tables. IPv6 only comes into play because doing that with IPv4 is mostly impractical these days. The author lost me when they got into raw iproute commands. Not because I'm not acquainted (I run my own custom complex router using a standard Linux distro). But rather if someone knows enough to configure things at this level, then they would just come to this solution on their own. Most people trying to solve this problem will not - eg think that mobile video rack belonging to a touring musician. Readily-accessible solutions I can come up with off the top of my head: 1. Two off the shelf routers and double NAT. The middle network can be changed if it conflicts with the outer network 2. One router/NAT, but two IP networks on the inner network - one statically assigned for devices to communicate with each other, and one assigned via DHCP for accessing the horizon through NAT. That second network can then easily be changed. 3. Play battleship more strategically using class E address space, DOD/BigCo address space, and/or smaller subnets in the middle of the customary size for a range (eg 192.168.1.160/27).

MartijnBraam 1y ago
Allocating a subnet is way further away from reality for most people than configuring one router feature on the router they're using. There's also a lot of people that configure these devices (or linux routers) themselves but have never heard of VRFs, you got to learn about them somewhere so I just hope this helps some people :)
- mindslight 1y ago
  I see your point if someone is at the level of tinkering enough to learn Mikrotik gear. I was loosely equating VRF with general Linux policy routing, where you end up owning a bit more of an overarching config with fwmarks etc. And then I reasoned that Mikrotik was more complex than that, because I personally avoid doing config on my Mikrotik devices in favor of the Linux router (which is the opposite if you're coming at it unopinionated). But if someone wants to understand just enough networking to copy and paste Mikrotik examples, I do think your post is good general suggestion for that.
stirfish 1y ago
I've been using a router as 4.20.69.1. It's good to hear other solutions, as I've just been figuring it all out as I go along
- JSR_FDED 1y ago
  I’ve never had any issues using a .666 subnet
master_crab 1y ago
If I can avoid Double NATing, I do because it tends to degrade network performance and can have interesting characteristics on some inbound traffic (although that depends on the use case). But the third option honestly isn’t recommended enough. DoD space is rarely routable, and if you are on a private network already, even moreso. It’s also less common than RFC1918. However, there is one caveat. Some large corporates do use it for just the same reason. Even though it is rarer than 10 or 172 space, you’d be surprised how many large orgs do run DoD internally. (Disclaimer: I use DoD space for my travel router at hotels)
1oooqooq 1y ago
can't they just put each port on it's own vlan and call it a day?

McNulty2 1y ago

As a network engineer it was interesting to read about VRFs and routing from the perspective of an A/V engineer. Thanks

dlcarrier 1y ago

It's still difficult to find equipment and ISPs that reliably work with IPv6. I'm sure it will be ready for prime time in the next decade or two, right when we get fusion power plants, self driving cars that don't require supervision, and the AI singularity.

whatevaa 1y ago
We will never get rid of ipv4. Apparently my country ISPs have more ipv4 than they need (they actually rent them to others), so they just don't bother with ipv6.
- cassianoleal 1y ago
  I guess when the world moves away from IPv4, your country will have the entire IPv4 space for itself, and massive headaches with NAT to route out, and the relative impossibility to route in.
  - sieabahlpark 1y ago
    [dead]
john01dav 1y ago
I think that government intervention here is needed in order to coordinate people. We'll all be better off once ivp4 is dead and buried, but the incentive for each entity to do so is minimal. It could be some other powerful entity instead of government, for example tier 1 ISPs degrading ipv4 performance substantially or Google delisting websites that don't support ipv6 (and eventually delisting websites that do support ipv4).
- dlcarrier 1y ago
  Really? The network protocol someone uses is so critical that it should be illegal to use the wrong one? What happens when IPv9 becomes the hot new thing, but everyone is stuck on IPv6, because of some outdated government regulation? Do Americans have the right to use whatever internet protocol they want, under the first amendment? IPv6 has made enough progress that it's totally possible to run your network off of it, regardless of what everyone else is doing, and if all of your neghbors are using IPv4, it won't harm your IPv6 network. Also, part of the delay in the switch to IPv6 is that some work is needed to ensure that home routers and IoT devices default to reasonable security settings, and the absolute worst thing to do is force them to switch first, and figure out security later. The answer isn't to force everyone to use something before its ready; the answer is to address every impediment, so it's worth it for everyone to switch. Sure it's slower, but it's much better than making users worse off by switching, converting them to detractors instead of supporters.

1718627440 1y ago

Maybe I'm dumb, but how do hosts from different subnets address each other, when they get the same IP addresses?

majormunky 1y ago
In the VRF case they can't: "This comes with a tradeoff of course and in this case is that you no longer can reach devices on the venue network, which shouldn't be a problem if you're only connected there for internet connectivity."
- 1718627440 1y ago
  Ok, but why want you to connect to local networks, when they can't reach each other, wouldn't you just then connect to the upstream router instead?
  - SoftTalker 1y ago
    That's typically all you care about, the upstream router/gateway so you can get to the internet. But it's on the local network, so you need to connect to it.

deleted 1y ago

[deleted]

treesknees 1y ago

I’ve encountered this issue with Docker. They randomly assign an RFC1918 subnet for internal services, and occasionally, this subnet clashes with a genuine corporate network or IP address. This can be quite frustrating to troubleshoot, especially if you’re not involved in Docker networking on a daily basis.

theideaofcoffee 1y ago
I've also run into this issue a lot, especially in AWS when developers with less networking experience start randomly picking VPC subnets (why they were doing that is a whole 'nuther issue). They wonder why connectivity in docker on their ec2 hosts doesn't work, by default docker chooses 172.17/16 which seems to be a common choice for VPC subnets. Annoying, but a teachable moment nonetheless.

jcalvinowens 1y ago

I've always been able to solve this problem more simply using IPv4 link-local addressing (https://www.rfc-editor.org/rfc/rfc3927 https://www.rfc-editor.org/rfc/rfc3927), it doesn't matter if somebody abuses the same range because the outbound interface is explicitly specified in sin6_scope_id passed to connect() and bind(). But if you're writing code at a higher level than the bsd sockets api, it's a whole can of worms.

champtar 1y ago

2 big address block that have few chances of conflict: - CGNAT 100.64.0.0/10 - "Benchmark" 198.18.0.0/15

telotortium 1y ago
CGNAT is used by Tailscale and presumably in the wild for its intended purpose.
- Lammy 1y ago
  And `100.115.92.0/23` is used by ChromeOS for PatchPanel: https://chromium.googlesource.com/chromiumos/platform2/+/main/patchpanel/address_manager.cc?autodive=0%2F#23 https://chromium.googlesource.com/chromiumos/platform2/+/mai...

londons_explore 1y ago

I think most regular off the shelf consumer WiFi routers can already do this. The "192.168.0.1" ip address being the internal network and the external network because you're chaining routers together is awfully common and consumers expect it to just work without needing to understand IP addresses like a nerd.