4 ms·

The problem is that SSH uses a terrible buffering algorithm. This fixes it: http://www.psc.edu/index.php/hpn-ssh http://www.psc.edu/index.php/hpn-ssh It's ab

• pmoriarty 12y ago
  HPN looks interesting, but the objections to it here[1] give me pause. bbcp,[2] which was mentioned in the original article, also looks promising. Particularly because it actually uses ssh, unlike pigz and nc. [1] - https://news.ycombinator.com/item?id=4710061 https://news.ycombinator.com/item?id=4710061 [2] - http://www.slac.stanford.edu/~abh/bbcp/ http://www.slac.stanford.edu/~abh/bbcp/
• dmm 12y ago
  > a terrible buffering algorithm The main difference now it that hpn tweaks the tcp window to function better on high-latency links. > It's able to consistently fill 1G ethernet links. Between two stock Debian sid laptops: dd if=/dev/zero bs=1048576 count=4096 | ssh <otherhost> dd of=/dev/null 8388608+0 records in 8388608+0 records out 4294967296 bytes (4.3 GB) copied, 37.0848 s, 116 MB/s That's 90% of the theoretical bandwith, right? If you're really transferring huge file files across fat WAN links, why not use GridFTP like the supercomputing centers use? They transfer TB daily. I wouldn't mess with patching openssh, especially with patches for old versions of openssh.