6 ms·

I've used this for years when passing large files between systems in weird network environments, it's almost always flawless. For some more exotic testing, I w

I've used this for years when passing large files between systems in weird network environments, it's almost always flawless. For some more exotic testing, I was able to run my own magic wormhole relay[1], which let me tweak some things for faster/more reliable huge file copies. I still hate how often Google Drive will fall over when you throw a 10s-of-GB file at it. [1] https://www.jeffgeerling.com/blog/2023/my-own-magic-wormhole-relay-zippier-transfers https://www.jeffgeerling.com/blog/2023/my-own-magic-wormhole...

cl3misch 2y ago
> you need a machine that can handle whatever link speeds you need I would have expected the relay server only being used for initial handshake to punch through NAT, after which the transfer is P2P. Only in the case of some network restrictions the data really flows through the relay. How could they afford running the free relay otherwise?
- lotharrr 2y ago
  There are two servers. The "mailbox server" helps with handshakes and metadata transfers, and is super-low bandwidth, a few hundred bytes per connection. The "transit relay helper" is the one that handles the bulk data transfer iff the two sides were unable to establish a direct connection. I've been meaning to find the time to add NAT-hole-punching for years, but haven't managed it yet. We'd use the mailbox server messages to help the two sides learn about the IP addresses to use. That would increase the percentage of transfers that avoid the relay, but the last I read, something like 20% of peer-pairs would still need the relay, because their NATs are too restrictive. The relay usage hasn't been expensive enough to worry about, but if it gets more popular, that might change.
  - bscphil 2y ago
    The folks on the wormhole-rs fork (who appear to share your Github organization? [1]) already have NAT punching working 95+% of the time in my testing, so maybe what they're doing could be ported over to the Python implementation. [1] https://github.com/magic-wormhole https://github.com/magic-wormhole
    - Fnoord 2y ago
      The Rust implementation on Tailscale worked well for me. Except on a layer 7 firewall have to be quick to permit the connection or else it tries fallback.
    - meejah 2y ago
      That is just using normal STUN/TURN via another server that one of those developers is running
- from-nibly 2y ago
  You cant make a p2p connection over a NAT without exposing a port on the public side of the NAT.
  - t0mas88 2y ago
    You can: https://en.wikipedia.org/wiki/Hole_punching_(networking) https://en.wikipedia.org/wiki/Hole_punching_(networking)
  - voxic11 2y ago
    You aren't guaranteed to be able to do that but in practice most times you can.
  - kccqzy 2y ago
    Go check out STUN and ICE. The best article I've found about NAT traversal is this article from Tailscale: https://tailscale.com/blog/how-nat-traversal-works https://tailscale.com/blog/how-nat-traversal-works
    - jvansc 2y ago
      https://sendfiles.dev/ https://sendfiles.dev/
      - yownie 2y ago
        is there a filesize limit for this?
      - kccqzy 2y ago
        That uses WebRTC, which uses the same NAT traversal tricks.
    - Fnoord 2y ago
      Not bad, though you don't even need STUN or ICE; https://github.com/samyk/pwnat https://github.com/samyk/pwnat https://github.com/samyk/slipstream https://github.com/samyk/slipstream
dangoodmanUT 2y ago
I just read this out in your voice
- geerlingguy 2y ago
  Heh and I was able to do some of that work in service of the dumb but fun test of Internet vs Pigeon data transfer speeds.
bscphil 2y ago
> For some more exotic testing, I was able to run my own magic wormhole relay[1], which let me tweak some things for faster/more reliable huge file copies. The lack of improvement in these tools is pretty devastating. There was a flurry of activity around PAKEs like 6 years ago now, but we're still missing: * reliable hole punching so you don't need a slow relay server * multiple simultaneous TCP streams (or a carefully designed UDP protocol) to get large amounts of data through long fat pipes quickly Last time I tried using a Wormhole to transmit a large amount of data, I was limited to 20 MB/sec thanks to the bandwidth-delay product. I ended up using plain old http, with aria2c and multiple streams I maxed out a 1 Gbps line. IMO there's no reason why PAKE tools shouldn't have completely displaced over-complicated stuff like Globus (proprietary) for long distance transfer of huge data, but here we are stuck in the past.
- themoonisachees 2y ago
  I overall agree, but "reliable holpunching" is an oxymoron. Hole punching is by definition an exploit of undefined behavior, and I don't see the specs getting updated to support it. UPnP IGD was supposed to be that, but well...
  - namibj 2y ago
    Well, with v6 you're down from NAT-hole-punching to Firewall-hole-punching, which in principle should be as simple as arranging the IP:Port pairs of both ends via the setup channel, and then sending a "SYN" packet in both directions at once. Then, trying to use e.g. TCP Prague (or, I guess, it's congestion control with UDP-native QUIC) as a scalable congestion controller, to take care of the throughout restrictions caused by high bandwidth delay product.
    - a_subsystem 2y ago
      >> sending a "SYN" packet in both directions at once. Might be a totally dumb question but how does this work? Wouldn’t you already have to have communication to set a time?
bsharper 2y ago
I end up using a combination of scp, LocalSend, magic wormhole and sharedrop.io. Occasionally `python -m http.server` in a pinch for local downloads. It's unfortunate that this xkcd comic is still as relevant as it was in 2011: https://xkcd.com/949/ https://xkcd.com/949/

cl3misch 2y ago

> you need a machine that can handle whatever link speeds you need I would have expected the relay server only being used for initial handshake to punch through NAT, after which the transfer is P2P. Only in the case of some network restrictions the data really flows through the relay. How could they afford running the free relay otherwise?

lotharrr 2y ago
There are two servers. The "mailbox server" helps with handshakes and metadata transfers, and is super-low bandwidth, a few hundred bytes per connection. The "transit relay helper" is the one that handles the bulk data transfer iff the two sides were unable to establish a direct connection. I've been meaning to find the time to add NAT-hole-punching for years, but haven't managed it yet. We'd use the mailbox server messages to help the two sides learn about the IP addresses to use. That would increase the percentage of transfers that avoid the relay, but the last I read, something like 20% of peer-pairs would still need the relay, because their NATs are too restrictive. The relay usage hasn't been expensive enough to worry about, but if it gets more popular, that might change.
- bscphil 2y ago
  The folks on the wormhole-rs fork (who appear to share your Github organization? [1]) already have NAT punching working 95+% of the time in my testing, so maybe what they're doing could be ported over to the Python implementation. [1] https://github.com/magic-wormhole https://github.com/magic-wormhole
  - Fnoord 2y ago
    The Rust implementation on Tailscale worked well for me. Except on a layer 7 firewall have to be quick to permit the connection or else it tries fallback.
  - meejah 2y ago
    That is just using normal STUN/TURN via another server that one of those developers is running
from-nibly 2y ago
You cant make a p2p connection over a NAT without exposing a port on the public side of the NAT.
- t0mas88 2y ago
  You can: https://en.wikipedia.org/wiki/Hole_punching_(networking) https://en.wikipedia.org/wiki/Hole_punching_(networking)
- voxic11 2y ago
  You aren't guaranteed to be able to do that but in practice most times you can.
- kccqzy 2y ago
  Go check out STUN and ICE. The best article I've found about NAT traversal is this article from Tailscale: https://tailscale.com/blog/how-nat-traversal-works https://tailscale.com/blog/how-nat-traversal-works
  - jvansc 2y ago
    https://sendfiles.dev/ https://sendfiles.dev/
    - yownie 2y ago
      is there a filesize limit for this?
    - kccqzy 2y ago
      That uses WebRTC, which uses the same NAT traversal tricks.
  - Fnoord 2y ago
    Not bad, though you don't even need STUN or ICE; https://github.com/samyk/pwnat https://github.com/samyk/pwnat https://github.com/samyk/slipstream https://github.com/samyk/slipstream

dangoodmanUT 2y ago

I just read this out in your voice

geerlingguy 2y ago
Heh and I was able to do some of that work in service of the dumb but fun test of Internet vs Pigeon data transfer speeds.

bscphil 2y ago

> For some more exotic testing, I was able to run my own magic wormhole relay[1], which let me tweak some things for faster/more reliable huge file copies. The lack of improvement in these tools is pretty devastating. There was a flurry of activity around PAKEs like 6 years ago now, but we're still missing: * reliable hole punching so you don't need a slow relay server * multiple simultaneous TCP streams (or a carefully designed UDP protocol) to get large amounts of data through long fat pipes quickly Last time I tried using a Wormhole to transmit a large amount of data, I was limited to 20 MB/sec thanks to the bandwidth-delay product. I ended up using plain old http, with aria2c and multiple streams I maxed out a 1 Gbps line. IMO there's no reason why PAKE tools shouldn't have completely displaced over-complicated stuff like Globus (proprietary) for long distance transfer of huge data, but here we are stuck in the past.

themoonisachees 2y ago
I overall agree, but "reliable holpunching" is an oxymoron. Hole punching is by definition an exploit of undefined behavior, and I don't see the specs getting updated to support it. UPnP IGD was supposed to be that, but well...
- namibj 2y ago
  Well, with v6 you're down from NAT-hole-punching to Firewall-hole-punching, which in principle should be as simple as arranging the IP:Port pairs of both ends via the setup channel, and then sending a "SYN" packet in both directions at once. Then, trying to use e.g. TCP Prague (or, I guess, it's congestion control with UDP-native QUIC) as a scalable congestion controller, to take care of the throughout restrictions caused by high bandwidth delay product.
  - a_subsystem 2y ago
    >> sending a "SYN" packet in both directions at once. Might be a totally dumb question but how does this work? Wouldn’t you already have to have communication to set a time?

bsharper 2y ago

I end up using a combination of scp, LocalSend, magic wormhole and sharedrop.io. Occasionally `python -m http.server` in a pinch for local downloads. It's unfortunate that this xkcd comic is still as relevant as it was in 2011: https://xkcd.com/949/ https://xkcd.com/949/