4 ms·

I'm intrigued by p2p distribution like this but it makes me wonder if it's really more cost effective to switch/route all these small torrent packets instead of

I'm intrigued by p2p distribution like this but it makes me wonder if it's really more cost effective to switch/route all these small torrent packets instead of just using fileservers with 10G interfaces and maybe tiered caching.

justinsaccount 8y ago
fileservers don't send packets? "... a test where a 3G Docker image with 2 layers is downloaded by 2600 hosts concurrently (5200 blob downloads)" they show this finishing in 20s, so that's 7,800 GB transferred in 20 seconds. That works out to 390 GB/sec. A 10Gbps interface can transfer about 1GB/sec, so you'd need 195 dual attached fileservers, or about 100 if they were running at 40gbps.
- jsight 8y ago
  > fileservers don't send packets? This line is far funnier than it should be. It reminds me of the time that I was told about how awful some architecture was because it would overload the network. Their improved architecture sent lots of smaller files over NFS instead. That was novel.
  - thekhatribharat 8y ago
    Well, probably they meant network congestion. Network congestion depends on both time distribution and link distribution of data, which differs widely with the choice of network protocols and network topologies (even if total data flowing through network remains the same)
    - discordianfish 8y ago
      Yep. I'll admit, I didn't crunch the numbers but routing torrent traffic isn't cheap. I haven't found a definitive resources but AFAIK/IIRC torrent uses at least an order of magnitude more packets per second for the same throughput as HTTP.
      - justinsaccount 8y ago
        > torrent uses at least an order of magnitude more packets per second for the same throughput as HTTP. This is total nonsense. This would mean that instead of using 1200+ byte packets, bittorrent uses 120 byte packets to transfer data. This is easily disproven by looking at the implementation of any client or just by looking at the traffic and seeing that it uses 1200+ byte packets for sending chunks around.
    - jsight 8y ago
      That is definitely possible, but in my scenario the discussion wasn't that deep. It was more like the difference between NFS sending lots of small files and HTTP sending a tarball. The real differences were always going to be in the details of the implementation, not the choice of protocol.

justinsaccount 8y ago

fileservers don't send packets? "... a test where a 3G Docker image with 2 layers is downloaded by 2600 hosts concurrently (5200 blob downloads)" they show this finishing in 20s, so that's 7,800 GB transferred in 20 seconds. That works out to 390 GB/sec. A 10Gbps interface can transfer about 1GB/sec, so you'd need 195 dual attached fileservers, or about 100 if they were running at 40gbps.

jsight 8y ago
> fileservers don't send packets? This line is far funnier than it should be. It reminds me of the time that I was told about how awful some architecture was because it would overload the network. Their improved architecture sent lots of smaller files over NFS instead. That was novel.
- thekhatribharat 8y ago
  Well, probably they meant network congestion. Network congestion depends on both time distribution and link distribution of data, which differs widely with the choice of network protocols and network topologies (even if total data flowing through network remains the same)
  - discordianfish 8y ago
    Yep. I'll admit, I didn't crunch the numbers but routing torrent traffic isn't cheap. I haven't found a definitive resources but AFAIK/IIRC torrent uses at least an order of magnitude more packets per second for the same throughput as HTTP.
    - justinsaccount 8y ago
      > torrent uses at least an order of magnitude more packets per second for the same throughput as HTTP. This is total nonsense. This would mean that instead of using 1200+ byte packets, bittorrent uses 120 byte packets to transfer data. This is easily disproven by looking at the implementation of any client or just by looking at the traffic and seeing that it uses 1200+ byte packets for sending chunks around.
  - jsight 8y ago
    That is definitely possible, but in my scenario the discussion wasn't that deep. It was more like the difference between NFS sending lots of small files and HTTP sending a tarball. The real differences were always going to be in the details of the implementation, not the choice of protocol.