7 ms·

Mptcp: Moving laptop from GBit to wireless without applications noticing?

64 points by globalc 4y ago

globalc 4y ago
I played through more scenarios with the "I have a laptop, and want seamless switching between ethernet and wireless". shadowsocks proxy seems like the best choice, allowing transparent proxy for TCP applications, but also needs 2 helper systems for the setup. Next best choice seems bonding, just needs helpers on the LAN. Details: https://fluxcoil.net/wiki/software/mptcp https://fluxcoil.net/wiki/software/mptcp
linsomniac 4y ago
MPTCP has been on my radar for a long while and seems very cool. I will say that for seamless wireless/wired switching, Linux bonding works very well. ~15+ years ago I had my laptop set up with eth and wifi bonded, with eth as the preferred, I believe using link-beat as the method. I could be running a big copy, plug in the ethernet and the transfer rate would almost instantly shoot up, and then when done I could unplug and all of my connections would stay up. As WiFi got faster and faster, I used this less and less, so I haven't had it set up for for quite a few years now. But it does show the promise of MPTCP.
- globalc 4y ago
  Nice, was not sure how stable such a setup would be. The applications will not need further modification and just need to be sent over the bond. For MPTCP, as of today the applications over the mptcp-stream (which has subflows over the 2 media) need to be modified to use MPTCP, or an additional layer like with shadowsocks needs to be used to transparently encapsulate traffic over the link.
  - jeroenhd 4y ago
    The article mentions the tool `mptcpize` which converts TCP into MPTCP by hooking calls to the networking API. With the tooling already there, I think this makes for an excellent use case for MPTCP.
    - iforgotpassword 4y ago
      As you need to do it on both ends it is limited to your own stuff. The bonding trick works for everything automagically.
- thrtythreeforty 4y ago
  Re: bonding, what a good idea! I may have to set this up on my laptop. Are there any downsides? (Besides being unable to connect to different networks of course)
  - justsomehnguy 4y ago
    You need to be in the same L2/L3 domain. Not always the case for the corporate networks.
    - ale42 4y ago
      Definitely not the case in our network... IP ranges are totally different on wired and wireless here.
  - globalc 4y ago
    You should pay attention to make the ethernet the preferred interface. If aggregation of bandwidth over the various underlying media is important for you: depending on the exact mode, aggregation might not work. Even if working, via bonding you might only get aggregation if the traffic goes over multiple TCP channels, while MPTCP also aggregates for a single TCP channel.
- cmsj 4y ago
  Yep, I used to do that when I had a thinkpad and a docking station and didn't want all my ssh sessions to drop. Before that I was even doing awful things like forcing the same IP onto both eth and wifi :D
  - zootboy 4y ago
    I've had a concept in mind for this exact scenario that I'm reasonably certain will work, I've just never taken the time to fully implement it. No need to bond interfaces or apply the same IP to both interfaces. Let them both acquire different IP addresses in the same subnet. Linux will start announcing its ARP replies on the ethernet path since it has a better path metric. Thus the return path for both the wifi IP and the ethernet IP will become the ethernet link. Any existing connections that were traveling over the wifi link will just move onto the ethernet link automatically. The only difficulty is new connections. Since the ethernet path has the better metric, new connections will establish with the ethernet link's IP address. But the routing table can be adjusted to instead make the wifi link's IP address the default for both the wifi and the ethernet links. Then new connections will always use the wifi IP, and will migrate over to the wifi link when the ethernet link disappears. This should work as long as your wifi link always remains up whether or not you're in the dock.
    - linsomniac 4y ago
      I guess you could do this by just sending gratuitous ARPs for the wifi IP with the eth MAC addr, over the eth interface. Probably have to turn off rp_filter as well?
      - zootboy 4y ago
        You don't even need to do that. The ARP packets seem to be routed to the network through the normal route table, so replies for the wifi IP get sent via the ethernet link and the network switches seem to learn that as the return path. As for rp_filter, it's set to "2" on my machines, which seems to allow return packets on the "wrong" port as long as it's an address that any other port owns.
deleted 4y ago
[deleted]
fulafel 4y ago
The net community gave up on the TCP->SCTP upgrade because of NAT (and other middle-) boxes breaking IP. It would have fixed many other things too (eg head-of-line blocking). The good side was that we kept the simplicity of TCP. I wonder what's the middlebox story of MPTCP.
- mister_goo 4y ago
  Newer protocols like QUIC are encapsulated by UDP instead of adding new protocol numbers. NAT box breaks because NAT must be implemented for each protocol (port number not at IP level).
  - fulafel 4y ago
    SCTP is over 20 years old. I think the explanation for today's situation is slightly more complex than NAT boxes not having enough advance notice. NAT isn't allowed by the TCP/IP specs, for the exact reason that it breaks IP and new internet applications. Everyone was supposed to move to IPv6 with enough addresses so the temptation to use NAT would go away. Instead we ignored the stewardship of IETF and ISOC, and just adapted to the laissez faire world of middleboxes and broken IPv4.
    - mister_goo 4y ago
      So there is a difference between what people are supposed to do and what people actually do. What I am wondering now is why IPv6 didn't adapt NAT as a first class feature, if IPv6 added something like port number in IPv6 header, NAT could not break protocols. It seems IPv6 strongly resists NAT, but in reality, people still use NAT on IPv6.
      - fulafel 4y ago
        Using NAT with IPv6 sounds tragic, hopefully this isn't common.
        jeroenhd 4y ago
rubatuga 4y ago
You can use WireGuard and our service hoppy.network to achieve the same thing! By having a static IP address exposed, you can switch from WiFi to ethernet or cellular without any network connections breaking. Probably not the best solution from an architectural standpoint, but certainly practical.
- nickmooney 4y ago
  This is a neat project. You're up against people setting up their own WireGuard exit nodes -- are these folks just not in your target audience?
  - Couponplay 4y ago
    [dead]
  - rubatuga 4y ago
    It’s mostly people who need good IP reputation and a user friendly managed service. It’s hard to get clean IPs from massive cloud services.
- goodpoint 4y ago
  A VPS is much cheaper and the user is not reqired to trust a VPN-like service not to snoop traffic. But it still does not solve the problem of adding a lot of latency.
  - rubatuga 4y ago
    If your threat model is based on your ISP not snooping on you then you need to improve your security…
    - goodpoint 4y ago
      Improve it by not having Internet connectivity I guess? Even Tor is vulnerable to timing correlation attacks, imagine everything else, given how leaky most protocols are...
Couponplay 4y ago
easytiger 4y ago
Surely if your application doesn't have some TCP reconnect handling its gonna have a problem eventually? Some switch fail-over scenarios i've tested will blackhole a connection which is why nearly anything serious will have some application level heartbeating to determine connection health. Can't depend on the OS for this stuff
thedougd 4y ago
MPTCP might also be interesting combined with an overlay/tunneling protocol like GENEVE. Although I believe software defined networks do a pretty good job already load balancing / failing over between VTEPs.
jiveturkey 4y ago
relevant: https://apenwarr.ca/log/20170810 https://apenwarr.ca/log/20170810 wow, did he write that 5 years ago? I guess I could just google it, but does anyone know if the OSI stack is portable to the application layer, the way that apenwarr was promoting, and the way that mptcp simulates? Since the time of that article, I've always wondered if that was the case. Or is OSI "broken" in its layering, like TCP/IP.
luckman212 4y ago
Anyone know the actual status of this landing in FreeBSD? All I could find was this[0] and the link to an old kernel patch that references a now-defunct git repo. [0] https://freebsdfoundation.org/project/multipath-tcp-for-freebsd/ https://freebsdfoundation.org/project/multipath-tcp-for-free...

64 points by globalc 4y ago

I played through more scenarios with the "I have a laptop, and want seamless switching between ethernet and wireless". shadowsocks proxy seems like the best choice, allowing transparent proxy for TCP applications, but also needs 2 helper systems for the setup. Next best choice seems bonding, just needs helpers on the LAN. Details: https://fluxcoil.net/wiki/software/mptcp https://fluxcoil.net/wiki/software/mptcp

linsomniac 4y ago

MPTCP has been on my radar for a long while and seems very cool. I will say that for seamless wireless/wired switching, Linux bonding works very well. ~15+ years ago I had my laptop set up with eth and wifi bonded, with eth as the preferred, I believe using link-beat as the method. I could be running a big copy, plug in the ethernet and the transfer rate would almost instantly shoot up, and then when done I could unplug and all of my connections would stay up. As WiFi got faster and faster, I used this less and less, so I haven't had it set up for for quite a few years now. But it does show the promise of MPTCP.

globalc 4y ago
Nice, was not sure how stable such a setup would be. The applications will not need further modification and just need to be sent over the bond. For MPTCP, as of today the applications over the mptcp-stream (which has subflows over the 2 media) need to be modified to use MPTCP, or an additional layer like with shadowsocks needs to be used to transparently encapsulate traffic over the link.
- jeroenhd 4y ago
  The article mentions the tool `mptcpize` which converts TCP into MPTCP by hooking calls to the networking API. With the tooling already there, I think this makes for an excellent use case for MPTCP.
  - iforgotpassword 4y ago
    As you need to do it on both ends it is limited to your own stuff. The bonding trick works for everything automagically.
thrtythreeforty 4y ago
Re: bonding, what a good idea! I may have to set this up on my laptop. Are there any downsides? (Besides being unable to connect to different networks of course)
- justsomehnguy 4y ago
  You need to be in the same L2/L3 domain. Not always the case for the corporate networks.
  - ale42 4y ago
    Definitely not the case in our network... IP ranges are totally different on wired and wireless here.
- globalc 4y ago
  You should pay attention to make the ethernet the preferred interface. If aggregation of bandwidth over the various underlying media is important for you: depending on the exact mode, aggregation might not work. Even if working, via bonding you might only get aggregation if the traffic goes over multiple TCP channels, while MPTCP also aggregates for a single TCP channel.
cmsj 4y ago
Yep, I used to do that when I had a thinkpad and a docking station and didn't want all my ssh sessions to drop. Before that I was even doing awful things like forcing the same IP onto both eth and wifi :D
- zootboy 4y ago
  I've had a concept in mind for this exact scenario that I'm reasonably certain will work, I've just never taken the time to fully implement it. No need to bond interfaces or apply the same IP to both interfaces. Let them both acquire different IP addresses in the same subnet. Linux will start announcing its ARP replies on the ethernet path since it has a better path metric. Thus the return path for both the wifi IP and the ethernet IP will become the ethernet link. Any existing connections that were traveling over the wifi link will just move onto the ethernet link automatically. The only difficulty is new connections. Since the ethernet path has the better metric, new connections will establish with the ethernet link's IP address. But the routing table can be adjusted to instead make the wifi link's IP address the default for both the wifi and the ethernet links. Then new connections will always use the wifi IP, and will migrate over to the wifi link when the ethernet link disappears. This should work as long as your wifi link always remains up whether or not you're in the dock.
  - linsomniac 4y ago
    I guess you could do this by just sending gratuitous ARPs for the wifi IP with the eth MAC addr, over the eth interface. Probably have to turn off rp_filter as well?
    - zootboy 4y ago
      You don't even need to do that. The ARP packets seem to be routed to the network through the normal route table, so replies for the wifi IP get sent via the ethernet link and the network switches seem to learn that as the return path. As for rp_filter, it's set to "2" on my machines, which seems to allow return packets on the "wrong" port as long as it's an address that any other port owns.

deleted 4y ago

[deleted]

fulafel 4y ago

The net community gave up on the TCP->SCTP upgrade because of NAT (and other middle-) boxes breaking IP. It would have fixed many other things too (eg head-of-line blocking). The good side was that we kept the simplicity of TCP. I wonder what's the middlebox story of MPTCP.

mister_goo 4y ago
Newer protocols like QUIC are encapsulated by UDP instead of adding new protocol numbers. NAT box breaks because NAT must be implemented for each protocol (port number not at IP level).
- fulafel 4y ago
  SCTP is over 20 years old. I think the explanation for today's situation is slightly more complex than NAT boxes not having enough advance notice. NAT isn't allowed by the TCP/IP specs, for the exact reason that it breaks IP and new internet applications. Everyone was supposed to move to IPv6 with enough addresses so the temptation to use NAT would go away. Instead we ignored the stewardship of IETF and ISOC, and just adapted to the laissez faire world of middleboxes and broken IPv4.
  - mister_goo 4y ago
    So there is a difference between what people are supposed to do and what people actually do. What I am wondering now is why IPv6 didn't adapt NAT as a first class feature, if IPv6 added something like port number in IPv6 header, NAT could not break protocols. It seems IPv6 strongly resists NAT, but in reality, people still use NAT on IPv6.
    - fulafel 4y ago
      Using NAT with IPv6 sounds tragic, hopefully this isn't common.
      - jeroenhd 4y ago

rubatuga 4y ago

You can use WireGuard and our service hoppy.network to achieve the same thing! By having a static IP address exposed, you can switch from WiFi to ethernet or cellular without any network connections breaking. Probably not the best solution from an architectural standpoint, but certainly practical.

nickmooney 4y ago
This is a neat project. You're up against people setting up their own WireGuard exit nodes -- are these folks just not in your target audience?
- Couponplay 4y ago
  [dead]
- rubatuga 4y ago
  It’s mostly people who need good IP reputation and a user friendly managed service. It’s hard to get clean IPs from massive cloud services.
goodpoint 4y ago
A VPS is much cheaper and the user is not reqired to trust a VPN-like service not to snoop traffic. But it still does not solve the problem of adding a lot of latency.
- rubatuga 4y ago
  If your threat model is based on your ISP not snooping on you then you need to improve your security…
  - goodpoint 4y ago
    Improve it by not having Internet connectivity I guess? Even Tor is vulnerable to timing correlation attacks, imagine everything else, given how leaky most protocols are...

Couponplay 4y ago

easytiger 4y ago

Surely if your application doesn't have some TCP reconnect handling its gonna have a problem eventually? Some switch fail-over scenarios i've tested will blackhole a connection which is why nearly anything serious will have some application level heartbeating to determine connection health. Can't depend on the OS for this stuff

thedougd 4y ago

MPTCP might also be interesting combined with an overlay/tunneling protocol like GENEVE. Although I believe software defined networks do a pretty good job already load balancing / failing over between VTEPs.

jiveturkey 4y ago

relevant: https://apenwarr.ca/log/20170810 https://apenwarr.ca/log/20170810 wow, did he write that 5 years ago? I guess I could just google it, but does anyone know if the OSI stack is portable to the application layer, the way that apenwarr was promoting, and the way that mptcp simulates? Since the time of that article, I've always wondered if that was the case. Or is OSI "broken" in its layering, like TCP/IP.

luckman212 4y ago

Anyone know the actual status of this landing in FreeBSD? All I could find was this[0] and the link to an old kernel patch that references a now-defunct git repo. [0] https://freebsdfoundation.org/project/multipath-tcp-for-freebsd/ https://freebsdfoundation.org/project/multipath-tcp-for-free...