3 ms·

Really love when I get a /30 with one IP from hosting providers. (Oops, /30 not 22. Subtracted two from the wrong number.)

candiddevmike 3y ago
Not sure what you mean, what's wrong with being issued a single IP on a /22?
- ilyt 3y ago
  think he just mistyped /32
ilyt 3y ago
we assign /32 to our VMs then add a /32 device route to the router. Not an ISP tho, just inside datacenter.
- jve 3y ago
  Can you elaborate? Say you give 1.1.1.1/32 to a VM (Sorry CF). What route do you add to reach internet? Do you somehow configure the router too? What do you put in as a gateway? And what would be router IP in this example, if you had, say 1.1.0.0/16 ? Wouldn't you need a /0 route for gateway?
  - jleahy 3y ago
    It’s very nasty, but there are a lot of options. For example you can give the gateway an address in the link local range and add a route pointing at that. I think that’s the best way. Another option would be to set the subnet mask to /0 and enable ARP proxy on the gateway (that is truly diabolical). Another way is to have a private /30 or /31 as the linknet and then add the /32 public ip as an additional one with a /0 route to the routers ip in the private /30 (and the router can have a /32 route to your ip in the private /30). 1:1 NAT is another option (but that’s not quite the question).
    - ilyt 3y ago
      see my other answer for a way. We do actually use proxy-arp for install only as it's hard to inject a route into installer
      - jleahy 3y ago
        ah, onlink, indeed that is another way.
  - ilyt 3y ago
    inet 10.252.1.20/32 brd 10.252.1.20 scope global eth0 valid_lft forever preferred_lft forever # ip r sh default via 10.252.0.1 dev eth0 onlink yes you do need a route for gateway, we just add it via CM > Do you somehow configure the router too? interface on the hypervisor (each vm gets their own) inet 10.252.0.1/32 brd 10.252.0.1 scope global v.vmname.0 and a route 10.252.1.20 dev vvmname.0 src 10.252.0.1 uid 0 added in a hook, and then distributed via ospf to rest of devices
    - jve 3y ago
      So each VM needs 2 IP addresse - 1 GW, another one VM? What is the advantage here...? Or is this done to prevent VM using another address from 10.252.0.0/x "subnet" ?
  - ogurechny 3y ago
    What does a subnet mask do? Almost nothing. In case of Ethernet: — If some data is to be delivered to an IP address that belongs to a subnet, send a packet with that destination IP in a frame with corresponding destination MAC. If you don't know that MAC, use ARP to learn it. — If some data is to be delivered to an IP address that does not belong to a subnet, send a packet with that destination IP in a frame with gateway destination MAC. What happens next is not our problem. If you don't know that MAC, use ARP to learn it. Therefore, /32 simply makes host relay packets to all other addresses to gateway. Note that gateway IP address is not included in any packet. We only use it to learn gateway MAC address we actually need. Note that gateway IP address can be anything, inside or outside of local subnet. The only requirement is that host gets an ARP response when asking about it (from actual interface or some other device acting on gateway's behalf). This gets trickier when host have multiple interfaces to multiple networks, and may have to prioritize, or some interfaces depend on others (as with VPN connections), but it's certainly not a problem in case of a single network connection. Note that physical network connection does not imply that only packets from a single subnet get transferred. Anything can be sent or received, the difference is in what gets accepted and what gets rejected.
schoen 3y ago
So, if the hosting provider's router supports lowest-address, you'll be able to host two usable addresses on that subnet, at least using Linux, FreeBSD, or OpenBSD (hopefully more OSes in the future). Maybe we should (be able to) get rid of the broadcast address in this situation too. Cf. RFC 3021 (adding a special case for /31). (If the hosting provider literally only intends to give you a single address, and insists on giving you a subnet, it should probably give you a /31 instead of a /30, because of the RFC 3021 behavior. Then it's not throwing away addresses for no reason.)
waiwai933 3y ago
Doesn't a /30, like a /31 have two routeable IPs, not one? What am I missing?
- tedunangst 3y ago
  Well, network number, gateway, broadcast, one left for host.

candiddevmike 3y ago

Not sure what you mean, what's wrong with being issued a single IP on a /22?

ilyt 3y ago
think he just mistyped /32

ilyt 3y ago

we assign /32 to our VMs then add a /32 device route to the router. Not an ISP tho, just inside datacenter.

jve 3y ago
Can you elaborate? Say you give 1.1.1.1/32 to a VM (Sorry CF). What route do you add to reach internet? Do you somehow configure the router too? What do you put in as a gateway? And what would be router IP in this example, if you had, say 1.1.0.0/16 ? Wouldn't you need a /0 route for gateway?
- jleahy 3y ago
  It’s very nasty, but there are a lot of options. For example you can give the gateway an address in the link local range and add a route pointing at that. I think that’s the best way. Another option would be to set the subnet mask to /0 and enable ARP proxy on the gateway (that is truly diabolical). Another way is to have a private /30 or /31 as the linknet and then add the /32 public ip as an additional one with a /0 route to the routers ip in the private /30 (and the router can have a /32 route to your ip in the private /30). 1:1 NAT is another option (but that’s not quite the question).
  - ilyt 3y ago
    see my other answer for a way. We do actually use proxy-arp for install only as it's hard to inject a route into installer
    - jleahy 3y ago
      ah, onlink, indeed that is another way.
- ilyt 3y ago
  inet 10.252.1.20/32 brd 10.252.1.20 scope global eth0 valid_lft forever preferred_lft forever # ip r sh default via 10.252.0.1 dev eth0 onlink yes you do need a route for gateway, we just add it via CM > Do you somehow configure the router too? interface on the hypervisor (each vm gets their own) inet 10.252.0.1/32 brd 10.252.0.1 scope global v.vmname.0 and a route 10.252.1.20 dev vvmname.0 src 10.252.0.1 uid 0 added in a hook, and then distributed via ospf to rest of devices
  - jve 3y ago
    So each VM needs 2 IP addresse - 1 GW, another one VM? What is the advantage here...? Or is this done to prevent VM using another address from 10.252.0.0/x "subnet" ?
- ogurechny 3y ago
  What does a subnet mask do? Almost nothing. In case of Ethernet: — If some data is to be delivered to an IP address that belongs to a subnet, send a packet with that destination IP in a frame with corresponding destination MAC. If you don't know that MAC, use ARP to learn it. — If some data is to be delivered to an IP address that does not belong to a subnet, send a packet with that destination IP in a frame with gateway destination MAC. What happens next is not our problem. If you don't know that MAC, use ARP to learn it. Therefore, /32 simply makes host relay packets to all other addresses to gateway. Note that gateway IP address is not included in any packet. We only use it to learn gateway MAC address we actually need. Note that gateway IP address can be anything, inside or outside of local subnet. The only requirement is that host gets an ARP response when asking about it (from actual interface or some other device acting on gateway's behalf). This gets trickier when host have multiple interfaces to multiple networks, and may have to prioritize, or some interfaces depend on others (as with VPN connections), but it's certainly not a problem in case of a single network connection. Note that physical network connection does not imply that only packets from a single subnet get transferred. Anything can be sent or received, the difference is in what gets accepted and what gets rejected.

schoen 3y ago

So, if the hosting provider's router supports lowest-address, you'll be able to host two usable addresses on that subnet, at least using Linux, FreeBSD, or OpenBSD (hopefully more OSes in the future). Maybe we should (be able to) get rid of the broadcast address in this situation too. Cf. RFC 3021 (adding a special case for /31). (If the hosting provider literally only intends to give you a single address, and insists on giving you a subnet, it should probably give you a /31 instead of a /30, because of the RFC 3021 behavior. Then it's not throwing away addresses for no reason.)

waiwai933 3y ago

Doesn't a /30, like a /31 have two routeable IPs, not one? What am I missing?

tedunangst 3y ago
Well, network number, gateway, broadcast, one left for host.