5 ms·

I wonder why SCTP isn't more widely-supported. It seems like a lot of Internet traffic is message-based rather than stream-based and would benefit from a more

by panic 10y ago

I wonder why SCTP isn't more widely-supported. It seems like a lot of Internet traffic is message-based rather than stream-based and would benefit from a more appropriate protocol.

abstractbeliefs 10y ago
I've actually been looking into (and started using) SCTP. It's pretty good, except some places seem to have it turned off. In particular, Windows has little support.
mikeash 10y ago
Seems to me that the basic Internet protocols got more or less frozen in time in the 90s, not because they had reached perfection, but just because it became too difficult to support new ones. With ubiquitous firewalls and NAT devices, anything that didn't fit into the existing TCP/UDP system would fail to work with so much of the Internet that people just wouldn't bother with it. It really is unfortunate that we ended up with UDP and TCP as the only choices. There are so many more things you might want outside of "unreliable packets" and "automatically recovered streams."
- nocarrier 10y ago
  The list of firewall-punchable level 4 protocols is frozen in time to just TCP and UDP, and it's sad at first glance, but it's not as bad as we think: * Most ISPs don't mangle UDP, and if you care enough about perf to be using UDP in the first place, you can afford to figure out who mangles UDP and fall back to TCP for those networks. It's a very small number of networks. * Any TCP/SCTP/etc inspired protocol you can think of can be implemented on top of UDP. Its header is only eight bytes long and it adds minimal overhead. UDP socket semantics are simple enough that you can treat them as a slightly smarter IP socket and build your protocol as if you were running directly on top of IPv6. * There's a huge advantage to being able to implement your protocol in userland since you get fine grained control over congestion and connection logic. There's a huge disadvantage too, since it's actually hard to implement that stuff yourself. But for the people who are large enough to build their own protocols (FB, Google, etc), it's worth it. * You can even encapsulate SCTP inside UDP to punch through middleware firewalls if you want to. I don't know of anyone doing that for real in the wild, but there's a RFC for it (RFC 6951).
  - DanielDent 10y ago
    +1 for your userland comment. I think that's a wildly under-appreciated element of UDP encapsulation. The fine grained control over congestion and congestion logic is cool, sure. But more important in my mind is the fact that there's a cross-platform API available and you can rapidly make changes to how the protocol works without needing to require a specific kernel version or a specific operating system. There's a whole lot of really cool frameworks for SDN to allow people to work at lower levels, but that's the problem - a standard API that everyone can expect to be widely available hasn't really emerged yet.
  - 15155 10y ago
    > I don't know of anyone doing that for real in the wild, but there's a RFC for it Aren't WebRTC data streams implemented with SCTP over UDP?
- DanielDent 10y ago
  At long last, many of the 'hacks' for dealing with the ugly world of NAT etc have become standardized. E.g. "ICE" provides a negotiation protocol to make a couple arbitrary machines able to talk to each other directly, including the NAT hole punching part of things. In the limited % of cases where there's no known way to punch a hole through NATs, an automatic fallback (using a relay server) is provided as a part of the protocol. It's not elegant, at least we don't have everybody re-inventing the wheel. And, good results when ICE is used can become part of the criteria now used to judge the quality of NATs and other evil middlebox devices.
- vvanders 10y ago
  Genuine question, other than NAT punchthrough what does UDP not include that you would want? It seems like unreliable packets are the foundation for anything else you'd want to build into a higher level protocol.
  - mikeash 10y ago
    Just to clarify, UDP is a great primitive, it's about as close to how the underlying hardware works as you can reasonably get for a userland-accesible protocol, and anything that could work over the Internet in general can be built on top of UDP. I'd just like things to be a bit more "batteries included" as the Python folks would say. Consider, for example, that even TCP is unnecessary, as you could implement it on top of UDP, but we all benefit from having it built in. Anyway, I'd love to be able to pick and chose among: 1. Packets versus streams. 2. Reliable versus unreliable. 3. Immediate delivery, or delivery in the same order as data was sent. Right now, we have UDP, which is packets, unreliable, immediate, and TCP, which is streams, reliable, delivery in order of transmission. But all eight possible combinations make sense (albeit some more than others), and it's unfortunate that we have to reimplement stuff that already exists if we don't fit within the two choices we have. And if we're going beyond what TCP provides, I'd really like to see: 4. Choice of ACK versus NACK based loss notification. 5. Forward error correction. 6. Multihoming.
    - vvanders 10y ago
      #1,2,3,4 are all pretty straightforward to build. I've done that more than a few times. There's a bunch of parameters that tend to increase complexity(how many ordered streams do you need, additional data associated with an endpoint, etc) that makes me think a one-size fits-all would be hard to do. #5 should really be done at the framing layer, otherwise your frame delimiters can also be corrupted. It's also very tied to physical layer and probably better off not being configured in platform agnostic software. No opinion on #6, I've not deal with it much AFAIK.
      - mikeash 10y ago
        I agree they're largely straightforward, but there would be a huge advantage in having one standardized version of each one rather than a million different buggy implementations. I also missed one, which is hard to get right: rate limiting. Especially if you want to coexist with TCP rather than drown it out (or be drowned out by it).

by panic 10y ago

I wonder why SCTP isn't more widely-supported. It seems like a lot of Internet traffic is message-based rather than stream-based and would benefit from a more appropriate protocol.

abstractbeliefs 10y ago

I've actually been looking into (and started using) SCTP. It's pretty good, except some places seem to have it turned off. In particular, Windows has little support.

mikeash 10y ago

Seems to me that the basic Internet protocols got more or less frozen in time in the 90s, not because they had reached perfection, but just because it became too difficult to support new ones. With ubiquitous firewalls and NAT devices, anything that didn't fit into the existing TCP/UDP system would fail to work with so much of the Internet that people just wouldn't bother with it. It really is unfortunate that we ended up with UDP and TCP as the only choices. There are so many more things you might want outside of "unreliable packets" and "automatically recovered streams."

nocarrier 10y ago
The list of firewall-punchable level 4 protocols is frozen in time to just TCP and UDP, and it's sad at first glance, but it's not as bad as we think: * Most ISPs don't mangle UDP, and if you care enough about perf to be using UDP in the first place, you can afford to figure out who mangles UDP and fall back to TCP for those networks. It's a very small number of networks. * Any TCP/SCTP/etc inspired protocol you can think of can be implemented on top of UDP. Its header is only eight bytes long and it adds minimal overhead. UDP socket semantics are simple enough that you can treat them as a slightly smarter IP socket and build your protocol as if you were running directly on top of IPv6. * There's a huge advantage to being able to implement your protocol in userland since you get fine grained control over congestion and connection logic. There's a huge disadvantage too, since it's actually hard to implement that stuff yourself. But for the people who are large enough to build their own protocols (FB, Google, etc), it's worth it. * You can even encapsulate SCTP inside UDP to punch through middleware firewalls if you want to. I don't know of anyone doing that for real in the wild, but there's a RFC for it (RFC 6951).
- DanielDent 10y ago
  +1 for your userland comment. I think that's a wildly under-appreciated element of UDP encapsulation. The fine grained control over congestion and congestion logic is cool, sure. But more important in my mind is the fact that there's a cross-platform API available and you can rapidly make changes to how the protocol works without needing to require a specific kernel version or a specific operating system. There's a whole lot of really cool frameworks for SDN to allow people to work at lower levels, but that's the problem - a standard API that everyone can expect to be widely available hasn't really emerged yet.
- 15155 10y ago
  > I don't know of anyone doing that for real in the wild, but there's a RFC for it Aren't WebRTC data streams implemented with SCTP over UDP?
DanielDent 10y ago
At long last, many of the 'hacks' for dealing with the ugly world of NAT etc have become standardized. E.g. "ICE" provides a negotiation protocol to make a couple arbitrary machines able to talk to each other directly, including the NAT hole punching part of things. In the limited % of cases where there's no known way to punch a hole through NATs, an automatic fallback (using a relay server) is provided as a part of the protocol. It's not elegant, at least we don't have everybody re-inventing the wheel. And, good results when ICE is used can become part of the criteria now used to judge the quality of NATs and other evil middlebox devices.
vvanders 10y ago
Genuine question, other than NAT punchthrough what does UDP not include that you would want? It seems like unreliable packets are the foundation for anything else you'd want to build into a higher level protocol.
- mikeash 10y ago
  Just to clarify, UDP is a great primitive, it's about as close to how the underlying hardware works as you can reasonably get for a userland-accesible protocol, and anything that could work over the Internet in general can be built on top of UDP. I'd just like things to be a bit more "batteries included" as the Python folks would say. Consider, for example, that even TCP is unnecessary, as you could implement it on top of UDP, but we all benefit from having it built in. Anyway, I'd love to be able to pick and chose among: 1. Packets versus streams. 2. Reliable versus unreliable. 3. Immediate delivery, or delivery in the same order as data was sent. Right now, we have UDP, which is packets, unreliable, immediate, and TCP, which is streams, reliable, delivery in order of transmission. But all eight possible combinations make sense (albeit some more than others), and it's unfortunate that we have to reimplement stuff that already exists if we don't fit within the two choices we have. And if we're going beyond what TCP provides, I'd really like to see: 4. Choice of ACK versus NACK based loss notification. 5. Forward error correction. 6. Multihoming.
  - vvanders 10y ago
    #1,2,3,4 are all pretty straightforward to build. I've done that more than a few times. There's a bunch of parameters that tend to increase complexity(how many ordered streams do you need, additional data associated with an endpoint, etc) that makes me think a one-size fits-all would be hard to do. #5 should really be done at the framing layer, otherwise your frame delimiters can also be corrupted. It's also very tied to physical layer and probably better off not being configured in platform agnostic software. No opinion on #6, I've not deal with it much AFAIK.
    - mikeash 10y ago
      I agree they're largely straightforward, but there would be a huge advantage in having one standardized version of each one rather than a million different buggy implementations. I also missed one, which is hard to get right: rate limiting. Especially if you want to coexist with TCP rather than drown it out (or be drowned out by it).