5 ms·

Can you summarize why this is better than just using UDP?

wmf 5y ago
UDP doesn't have reliability, flow control, congestion control, etc. Blasting RPCs over UDP can cause poor performance due to congestion.
- bullen 5y ago
  On a switch you could probably trust UDP no?
  - magicalhippo 5y ago
    Well, if by "probably trust" you mean you are happy if that prediction fails every now and then, then sure. If a link is saturated, say via TCP, then UDP data can still get dropped AFAIK. Searching I found this blog post[1] which takes a small stab at it. Would be interesting to try other setups with more data. Regardless, using UDP one should always be prepared to handle dropped and out-of-order packets. https://openmymind.net/How-Unreliable-Is-UDP/ https://openmymind.net/How-Unreliable-Is-UDP/
    - bullen 5y ago
      That link actually proves my point, thx! Nothing is perfect, so I'll keep using UDP on a switch and TCP on the internet.
      - Hikikomori 5y ago
        Has nothing to do with using a switch or not. If send a 1Gbit udp stream over a switch to another machine there will be no drops if both are connected with 1Gbit, the same is true if you use tcp, and over a router assuming its capable of forwarding that amount of traffic. If you have a third machine sending udp or tcp traffic towards the one receiving the 1Gbit udp stream you'll have drops on both streams. Doesn't matter what protocol you use, if you have congestion you'll have drops. You typically use udp if your application is real time, or if you want to create your own reliability mechanism and/or avoid issues with devices in the middle that does things with tcp.
        magicalhippo 5y ago
        That was my point. If you design your system so that the links stay well below saturation, then you would not expect to see drops. However they can still occur, maybe due to some unexpected congestion or other issues. So long as your application can deal with that you should be good.
        nitrogen 5y ago
        other issues Like intermittent EMI causing bit flips and checksum failures, which happened to me once in an IoT application where the Ethernet cable to an outbuilding was buried next to the power line, and the network would die whenever the furnace kicked on.
        bullen 5y ago
        So where can one buy consumer 10Gb/s switches?
        magicalhippo 5y ago
        MikroTik makes some, like the CRS305-1G-4S+IN[1] which is a 5-port variant at a very reasonable price. Nice review by ServeTheHome here[2]. [1]: https://mikrotik.com/product/crs305_1g_4s_in https://mikrotik.com/product/crs305_1g_4s_in [2]: https://www.servethehome.com/mikrotik-crs305-1g-4sin-review-4-port-must-have-10gbe-switch/ https://www.servethehome.com/mikrotik-crs305-1g-4sin-review-...
        bullen 5y ago
        Thx, say I have 4x 1Gb/s old-school eth-port machines that want to saturate their capacity at the same time on one of these, should I just buy 4x 10GBase-T SFP+ transceivers for it? Edit: Apparently LGS105/LGS108 has 10Gb/s switching capacity allready so I'm good!
- bcrl 5y ago
  The data center bridging extensions ensure that packets won't get lost due to congestion or flow control. They were created in part because fibre channel over ethernet couldn't handle any packet loss in the fabric.
  - wmf 5y ago
    Running lossless with no congestion control leads to high tail latency though.
    - bcrl 5y ago
      Quite true. However, in the case of FCoE, given the performance issues with the Fibre Channel SANs I worked on a few years ago, tail latency in the fabric was the least of the concerns we ran into. Customers would wonder why their persistent messaging rates tanked when the system started pushing messages to disk. Meanwhile their SAN can't even sustain 100MB/s of sequential writes.
deleted 5y ago
[deleted]

wmf 5y ago

UDP doesn't have reliability, flow control, congestion control, etc. Blasting RPCs over UDP can cause poor performance due to congestion.

bullen 5y ago
On a switch you could probably trust UDP no?
- magicalhippo 5y ago
  Well, if by "probably trust" you mean you are happy if that prediction fails every now and then, then sure. If a link is saturated, say via TCP, then UDP data can still get dropped AFAIK. Searching I found this blog post[1] which takes a small stab at it. Would be interesting to try other setups with more data. Regardless, using UDP one should always be prepared to handle dropped and out-of-order packets. https://openmymind.net/How-Unreliable-Is-UDP/ https://openmymind.net/How-Unreliable-Is-UDP/
  - bullen 5y ago
    That link actually proves my point, thx! Nothing is perfect, so I'll keep using UDP on a switch and TCP on the internet.
    - Hikikomori 5y ago
      Has nothing to do with using a switch or not. If send a 1Gbit udp stream over a switch to another machine there will be no drops if both are connected with 1Gbit, the same is true if you use tcp, and over a router assuming its capable of forwarding that amount of traffic. If you have a third machine sending udp or tcp traffic towards the one receiving the 1Gbit udp stream you'll have drops on both streams. Doesn't matter what protocol you use, if you have congestion you'll have drops. You typically use udp if your application is real time, or if you want to create your own reliability mechanism and/or avoid issues with devices in the middle that does things with tcp.
      - magicalhippo 5y ago
        That was my point. If you design your system so that the links stay well below saturation, then you would not expect to see drops. However they can still occur, maybe due to some unexpected congestion or other issues. So long as your application can deal with that you should be good.
        nitrogen 5y ago
        other issues Like intermittent EMI causing bit flips and checksum failures, which happened to me once in an IoT application where the Ethernet cable to an outbuilding was buried next to the power line, and the network would die whenever the furnace kicked on.
        bullen 5y ago
        So where can one buy consumer 10Gb/s switches?
        magicalhippo 5y ago
        MikroTik makes some, like the CRS305-1G-4S+IN[1] which is a 5-port variant at a very reasonable price. Nice review by ServeTheHome here[2]. [1]: https://mikrotik.com/product/crs305_1g_4s_in https://mikrotik.com/product/crs305_1g_4s_in [2]: https://www.servethehome.com/mikrotik-crs305-1g-4sin-review-4-port-must-have-10gbe-switch/ https://www.servethehome.com/mikrotik-crs305-1g-4sin-review-...
        bullen 5y ago
        Thx, say I have 4x 1Gb/s old-school eth-port machines that want to saturate their capacity at the same time on one of these, should I just buy 4x 10GBase-T SFP+ transceivers for it? Edit: Apparently LGS105/LGS108 has 10Gb/s switching capacity allready so I'm good!
bcrl 5y ago
The data center bridging extensions ensure that packets won't get lost due to congestion or flow control. They were created in part because fibre channel over ethernet couldn't handle any packet loss in the fabric.
- wmf 5y ago
  Running lossless with no congestion control leads to high tail latency though.
  - bcrl 5y ago
    Quite true. However, in the case of FCoE, given the performance issues with the Fibre Channel SANs I worked on a few years ago, tail latency in the fabric was the least of the concerns we ran into. Customers would wonder why their persistent messaging rates tanked when the system started pushing messages to disk. Meanwhile their SAN can't even sustain 100MB/s of sequential writes.

deleted 5y ago

[deleted]