5 ms·

Thank you for this explanation, I understand now. Originally I thought you would want all routers in the path to hash the same flow the same way, but didn't thi

Thank you for this explanation, I understand now. Originally I thought you would want all routers in the path to hash the same flow the same way, but didn't think about how it interacts with layers of routers. On first guess, I would think a per-device salt is another way to address polarization. What are other ways?

bogomipz 8y ago
>"Originally I thought you would want all routers in the path to hash the same flow the same way," This is actually what you do want.
- toast0 8y ago
  The problem is if all of the equipment hashes the same way, many of your links will be underutilized. If A has a link to B1 and B2, and each of B1 and B2 have two links to C, and everything hashes the same, there will only be two possible paths to C, instead of the four you should have. Ex: if packet hashes to 0, it goes to B1, and then over the 0 link to C. If packet hashes to 1, it goes to B2 and over the 1 link to C. If the Bs hash on different values than A (including if they have a different salt), you'll have better distribution on the B to C links, and actually be able to hit all 4 paths.
- sleepydog 8y ago
  The property you want is that all packets for a given flow take the same path through the network (absent topology changes), to minimize out-of-order packets and ease troubleshooting. A1 +- B1 +- C1 | `- C2 | `- B2 +- C3 `- C4 In this example, if all routers compute the same hash value for a given packet, links B1 -> C2 and B2 -> C4 never get used, for any flows. So all you really need is for each router to make a consistent decision about packets in the same flow. They don't have to use the exact same hashing function. In addition to the cumulus networks link provided in the GP, it looks like Cisco gear also has a per-device salt: https://www.cisco.com/c/en/us/support/docs/ip/express-forwarding-cef/116376-technote-cef-00.html#anc5 https://www.cisco.com/c/en/us/support/docs/ip/express-forwar... Note that this has nothing to do with endpoint selection, just the intermediate hops. Edit: I'm also leaving out how the final modulo can produce different selections between layers. So in practice, if you have layers that are different sizes you'd get better utilization. That seems like a fragile thing to depend upon, though.
  - bogomipz 8y ago
    >"So all you really need is for each router to make a consistent decision about packets in the same flow." Indeed this is what I was trying to articulate but maybe I didn't do a good job of that. I mentioned somewhere else you can also just change the hash at each tier in your network - add IP protocol in one, don't use IP protocol in another. This should achieve the same as adding a "seed." to a router's hash.
mcpherrinm 8y ago
Per device salt is the usual way. See https://docs.cumulusnetworks.com/plugins/servlet/mobile#content/view/8358175 https://docs.cumulusnetworks.com/plugins/servlet/mobile#cont... for some example documentation (ctrlf hash seed). Or you can change the inputs to the hash function (eg, one tier hashes on source is and the other doesn't) but that's more troublesome.

bogomipz 8y ago

>"Originally I thought you would want all routers in the path to hash the same flow the same way," This is actually what you do want.

toast0 8y ago
The problem is if all of the equipment hashes the same way, many of your links will be underutilized. If A has a link to B1 and B2, and each of B1 and B2 have two links to C, and everything hashes the same, there will only be two possible paths to C, instead of the four you should have. Ex: if packet hashes to 0, it goes to B1, and then over the 0 link to C. If packet hashes to 1, it goes to B2 and over the 1 link to C. If the Bs hash on different values than A (including if they have a different salt), you'll have better distribution on the B to C links, and actually be able to hit all 4 paths.
sleepydog 8y ago
The property you want is that all packets for a given flow take the same path through the network (absent topology changes), to minimize out-of-order packets and ease troubleshooting. A1 +- B1 +- C1 | `- C2 | `- B2 +- C3 `- C4 In this example, if all routers compute the same hash value for a given packet, links B1 -> C2 and B2 -> C4 never get used, for any flows. So all you really need is for each router to make a consistent decision about packets in the same flow. They don't have to use the exact same hashing function. In addition to the cumulus networks link provided in the GP, it looks like Cisco gear also has a per-device salt: https://www.cisco.com/c/en/us/support/docs/ip/express-forwarding-cef/116376-technote-cef-00.html#anc5 https://www.cisco.com/c/en/us/support/docs/ip/express-forwar... Note that this has nothing to do with endpoint selection, just the intermediate hops. Edit: I'm also leaving out how the final modulo can produce different selections between layers. So in practice, if you have layers that are different sizes you'd get better utilization. That seems like a fragile thing to depend upon, though.
- bogomipz 8y ago
  >"So all you really need is for each router to make a consistent decision about packets in the same flow." Indeed this is what I was trying to articulate but maybe I didn't do a good job of that. I mentioned somewhere else you can also just change the hash at each tier in your network - add IP protocol in one, don't use IP protocol in another. This should achieve the same as adding a "seed." to a router's hash.

mcpherrinm 8y ago

Per device salt is the usual way. See https://docs.cumulusnetworks.com/plugins/servlet/mobile#content/view/8358175 https://docs.cumulusnetworks.com/plugins/servlet/mobile#cont... for some example documentation (ctrlf hash seed). Or you can change the inputs to the hash function (eg, one tier hashes on source is and the other doesn't) but that's more troublesome.