9 ms·

Multipath TCP for Linux (2022)

• vlovich123 3y ago
  Why does this require explicit opt in by applications if there’s transparent fallback? Wouldn’t it make most sense for the kernel to do it transparently for every TCP connection so that it can make more global decisions about path aggregation / link preference?
  • chmod775 3y ago
    Using this implies that there are multiple IPs per endpoint associated with a single TCP connection. That is going to need explicit support/awarness by the application in many cases.
  • brenns10 3y ago
    My understanding is that it was basically a condition enforced by the maintainers of the Linux TCP / networking subsystems. If you look at the initial upstreaming discussions[1], this was setup as a ground rule. If you look at the older multipath TCP implementation, prior to the upstreaming, it was intended to be fully transparent to the application, which I think makes more sense for the intent of the protocol. Sure, in many cases MPTCP may be better with application-guided logic, but having a standard system approach (e.g. establish sub-flows on an LTE connection for automatic failover, but don't send any data along those sub-flows) would have worked for 95% of cases. [1] https://lore.kernel.org/all/alpine.OSX.2.21.1707181728570.11990@dhruvmeh-mobl1.amr.corp.intel.com/ https://lore.kernel.org/all/alpine.OSX.2.21.1707181728570.11...
    • matttbe 3y ago
      Yes, that's correct. In the Linux kernel, it would not be possible to switch to MPTCP by default. But apps can use it by default. For the server case, it really makes sense: https://www.mptcp.dev/faq.html#why--when-should-mptcp-be-enabled-by-default https://www.mptcp.dev/faq.html#why--when-should-mptcp-be-ena... GNU/Linux distributions could even switch MPTCP on by default (via eBPF).
  • londons_explore 3y ago
    I can imagine new security holes being opened up by allowing multiple IP's to talk over the same TCP connection... Imagine you have an application which checks the clients IP (eg. against a whitelist) at the time of connection and then assumes it doesn't change...
• apitman 3y ago
  Is this something that middle boxes tend to support in practice?
  • flounder3 3y ago
    We found that most proxies/firewalls (90%+ ? I forget) didn't tamper with it. The largest hurdle was working with load balancer vendors to implement it.
  • deleted 3y ago
    [deleted]
• iknowstuff 3y ago
  Apple supports it too and uses it for Siri https://developer.apple.com/documentation/foundation/urlsessionconfiguration/improving_network_reliability_using_multipath_tcp https://developer.apple.com/documentation/foundation/urlsess...
  • Arcanum-XIII 3y ago
    It’s quite easy to use in other app - it’s there in the basics. We were quite amazed to find our voip app to be quite robust in 2011 :D
• ohnoesjmr 3y ago
  I've heard about MPTCP back in 2013. It made so much sense back then, when mobile apps were not that robust to networks changing, I assumed it's going to get adopted in no time due to how much of a ux improvement it would have been back in the day. It's incredibly depressing that this gained barely any traction in the last 10 years, and kernel options are appearing just recently, after everyone has wrapped they http calls in multiple retry handlers, and mobile operating systems have abstracted network connectivity to the point where it feels more like you are using zeromq rather than tcp.
  • zoobab 3y ago
    ZeroMQ yeah!
  • quectophoton 3y ago
    > I've heard about MPTCP back in 2013. > I assumed it's going to get adopted in no time due to how much of a ux improvement it would have been back in the day. You might also be interested in SCTP[1] from the year 2000, which also hasn't gotten any traction so far. [1]: https://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol https://en.wikipedia.org/wiki/Stream_Control_Transmission_Pr...
    • gamegod 3y ago
      WebRTC data channels use SCTP, which ain't nothing! https://datatracker.ietf.org/doc/html/rfc8831 https://datatracker.ietf.org/doc/html/rfc8831 (SCTP over DTLS, that is...)
    • fanf2 3y ago
      SCTP is used a lot inside telco networks for carrying switching control metadata for voice connections. https://en.wikipedia.org/wiki/SIGTRAN https://en.wikipedia.org/wiki/SIGTRAN
    • throw0101b 3y ago
      > You might also be interested in SCTP[1] from the year 2000, which also hasn't gotten any traction so far. Probably partly because middleware boxes (e.g., firewalls) either didn't/don't support it and/or rules were written to only support "TCP" (as opposed to 'stream') or "UDP" (as opposed to 'dgram'; see also "DCCP").
• zoobab 3y ago
  Too bad there is no link to projects using mptcp, such as Openwrt derivatives. PS: I mentored a student for 2 years at GSOC to patch Openwrt with mptcp: https://blog.freifunk.net/2017/05/29/gsoc-2017-add-mptcp-support-ledeopenwrt-trunk/ https://blog.freifunk.net/2017/05/29/gsoc-2017-add-mptcp-sup...
  • atarvaneitor 3y ago
    one example on openwrt http://www.openmptcprouter.com/ http://www.openmptcprouter.com/
    • matttbe 3y ago
      Good point, the last version (v0.60) is using the upstream kernel by default. I just added it in the list: https://www.mptcp.dev/apps.html#misc https://www.mptcp.dev/apps.html#misc
  • londons_explore 3y ago
    Whats the benefit of an openwrt router supporting mptcp? I'd think support on web servers and mobile devices would be most important...
    • ranger_danger 3y ago
      Link aggregation and redundancy
    • DarrisMackelroy 3y ago
      I got fiber run to my neighborhood, and for a while, had a 1gb coax connection and a 1gb fiber connection. I used openmptcprouter to aggregate my connections through a droplet and I effectively had a 2 gigabit internet connection. I would have stuck with it, but having a datacenter IP for your home network really doesn’t work.
      • londons_explore 3y ago
        Except TCP is just a bad protocol to start with for tunnelling, because packetized data has to be delivered in-order, and head of line blocking messes up congestion control algorithms in the tunnelled data.
• vesinisa 3y ago
  > If any middlebox in between does not support it, the returned SYN+ACK packet will not contain MPTCP options in the TCP option field. That sounds .. quite restrictive. Is the only requirement on a middlebox to just forward the MPTCP options as-is?
  • mhandley 3y ago
    Before we finalized the specs we did a lot of testing to make sure that enabling MPTCP wouldn't break connectivity. Either it would be passed correctly, or it would fall back to single-path TCP safely. Generally, so long as a middlebox passes through unknown options unmodified, and does not try to enforce that the TCP sequence space it sees is contiguous, MPTCP should work through that middlebox. If you're interested, we wrote a couple of papers on this: [1] https://www.usenix.org/conference/nsdi12/technical-sessions/presentation/raiciu https://www.usenix.org/conference/nsdi12/technical-sessions/... [2] https://www.researchgate.net/publication/229002024_Is_it_still_possible_to_extend_TCP https://www.researchgate.net/publication/229002024_Is_it_sti...
• extraduder_ire 3y ago
  The examples given on the page seem to focus on multipath to get to a device over the internet, but I can see this being more likely to work properly without needing to fallback on home networks.
  • txutxu 3y ago
    At home/lan we use LACP, VRRP... I mean link aggregation and HA needs are solved time ago. With multiple ISPs, or on a complex enough LAN, we can use multiple routing tables + weights too. Also, if the ISP at home can do 10Gbps, 1Gbps, 300 Mbps whatever... I want to be able to use them with a single path, so there is no gain using multiple paths. Eventually, when I have cable+wifi connected at the same time, I use to force one of both, cannot see a reason to prefer using both at the same time. Maybe the latency thing? Never had that issue at home, but could understand that usage case "just use the network segment with less latency to reach $thing".
    • matttbe 3y ago
      Some ISPs in Europe are using MPTCP for people being too far from the street cabinets. Typically, for people in the countryside, with < 50 Mbps. Thanks to a transparent proxy installed in the home gateway, and servers in the ISP's network, they can combine both the fixed and cellular networks, and use the fixed one in priority. MPTCP can also be very interesting for mobility use-cases, even when one network is used at a time, e.g. switching from WiFi to cellular, or different cellular networks in the train, etc.
    • lo0dot0 3y ago
      For a long time enterprise firewalls (and more recently SD-WAN) allowed load balancing between different links, but unlike MPTCP the traffic of a single TCP connection is not split up. This is in line with the established network admin wisdom saying that reordering packets of a TCP connection hurts performance. https://community.fortinet.com/t5/FortiGate/Technical-Tip-How-to-load-balance-traffic-between-two-interfaces/ta-p/210812 https://community.fortinet.com/t5/FortiGate/Technical-Tip-Ho...
    • semi 3y ago
• slt2021 3y ago
  This can help in security/privacy setting. for example Great Chinese firewall: if you can split your traffic across multiple uplink channels, the firewall will have a hard time to put them together for enforcement?
  • Hikikomori 3y ago
    They just block or rate limit it severely when they dont know what it is.
• EvanAnderson 3y ago
  I don't know which makes me sadder-- IPv4 only having a 32-bit address space or TCP using the source and destination IP addresses in the connection tuple. That's one of those "if I had a time machine" of things-- I'd go back and have Cert and Kahn change both of those items.
  • bobmcnamara 3y ago
    How would you change TCP? Do you mean how we have to use 4 fields to track a connection - IP address and port for both ends?
    • EvanAnderson 3y ago
      If TCP had a protocol specific identifier for connections (a couple of 32-bit values, for example-- a client nonce and server nonce) rather than using the source/destination IP addresses multi-homed hosts and seamless transition between different networks would become native features of the protocol. A client could roam between two different IP networks and TCP connections would "survive", for example. (I'm oversimplifying nearly to the point of hyperbole, to be sure...) (Another fun future would have been one where SCTP got widespread adoption.)
      • mauriciob 3y ago
        How would routing be done without source/destination? When the device changes networks, how does the origin and all routers along the way know that this device is on a new network?
        • zare_st 3y ago
          Yeah it's a shortsighted plan. How would i set up rules on firewall if I don't know how to distingush the connections?
          • mlyle 3y ago
            There's two separate ideas here: * Where to send a frame to get to the other side of the connection * Whose connection this is. TCP combined the two, because we didn't have mobile clients or a lot of multihomed systems that would benefit from distinguishing them. Also, every octet in the header counted. In practice, this means we have to keep building a lot of infrastructure on top of TCP (or parallel to it, in datagram protocols) to handle retries and splitting flows well. In turn, these things are completely opaque to the network and it's difficult to write rules about them. Whereas if we had different packet fields for "where am I sending this packet right now" and "whose flow does this belong to"? we could write better firewall rules, have less infrastructure built on top of TCP, and have better typical application performance.
• infogulch 3y ago
  I found [1] which describes the architectural difference between MPTCP and QUIC, and also introduces the authors' proposed MPQUIC protocol: > QUIC multiplexes application streams on a single UDP flow, whereas MPTCP splits a single stream on multiple TCP subflows. MPQUIC combines both features by multiplex- ing application streams on multiple UDP subflows. [1]: "Multipath QUIC: A Deployable Multipath Transport Protocol" https://www.researchgate.net/publication/327122884_Multipath_QUIC_A_Deployable_Multipath_Transport_Protocol https://www.researchgate.net/publication/327122884_Multipath... Now I'm curious about how these protocols compare in production operation. Anybody have experience with both?
  • matttbe 3y ago
    Note that MPQUIC is still being discussed at the IETF. At the last IETF meeting, more changes have been discussed. Unfortunately, that slows down its adoption. https://lwn.net/Articles/964377/ https://lwn.net/Articles/964377/ But both tries to achieve the same goal. Technically, you can have a very similar behaviour. MPTCP is implemented in the Linux kernel, while QUIC is on the userspace side.
• bdd8f1df777b 3y ago
  The only practical use of MPTCP for me is to use mobile and Wi-Fi network together to boost the speed. iOS and WeChat both support this. However, I always turn them off because my mobile network is metered. So in the end, MPTCP is useless for me *personally*.
  • psaux 3y ago
    I worked on this. We called it the parking lot bug. WiFi still shows signal but no proper connection. With MPTCP, it will failover to cell.
    • taken_usernam 3y ago
      You can even learn to predict these handovers to make it really smooth [0]. [0] https://arxiv.org/pdf/1907.10493.pdf https://arxiv.org/pdf/1907.10493.pdf
• suprjami 3y ago
  I work supporting, debugging, fixing the Linux network stack and drivers. I am amazed how little adoption this has seen. Like everything which came along and tried to supplant regular TCP, such as SCTP, it seems MPTCP has also been confined to a niche of application developers who will use it forever while the rest of the world forgets about it.
  • dilyevsky 3y ago
    Other folks have commented - Apple Siri uses mptcp so not exactly niche considering how many devices that is
    • suprjami 3y ago
      Similarly, I promise you everything you've ever done over a 4G or 5G phone has used SCTP inside the phone provider. That doesn't mean the general developer population know about it. I bet most developers have never even heard of it.
      • dilyevsky 3y ago
        I worked on IMS before it was widely rolled out and at least back then its was just SIP over TCP and UDP (RTP/RTCP) for media. SCTP was widely used for eNodeB - MME comms tho iirc