6 ms·

I love BT and think it's great, but I always wonder: Is it a net (for the bandwidth of whole internet) gain? I believe the major advantage is to transfer the c

by berbec 8y ago

I love BT and think it's great, but I always wonder: Is it a net (for the bandwidth of whole internet) gain? I believe the major advantage is to transfer the cost of distrubutio away from the original host and distrubute it among those downloading IT. Does it reduce bandwidth used appreciably or just shuffle it around?

dan-robertson 8y ago
It also leads to more balance between upload/download. This is perhaps less good on typical broadband which usually has a much higher bandwidth for downloads but a typical datacentre will have a much greater upload demand to download (there’s a reason that downloading to an aws box is much cheaper than uploading). On the other hand, big networks can be more efficient by caching things near the edge so that the total “bit miles” (I.e. the total over all packets sent of distance traveled * bits of data) are decreased.
- jjeaff 8y ago
  I'd say BitTorrent is likely just as efficient or much more so at keeping distances short. At least, it optimizes by taking the fastest available peers. Your typical big torrent file could have thousands of peers. That's more endpoints than most any other big CDN and more likely to have a few peers really close to you. Heck, it could even download from someone on your local network if they happened to be serving up the file you are looking for.
flatline 8y ago
Centralization will always be more efficient, generally speaking. Access to a centralized channel for distribution can be prohibitive for many reasons - cost and censorship being the foremost in my mind.
- StudentStuff 8y ago
  I'm doubtful of this premise, especially in the context of peering disputes. Here in the states, Netflix had/has serious issues with Comcast/Verizon/Centurylink not peering with their upstream providers sufficently to handle the number of video streams that needed to be carried. This was not an issue with the last mile connection itself, but a middle mile peering problem. From the testing I have done, on certain carriers that play games like this (Deutsch Telekom in Germany, Centurylink, Verizon & others in the US, and most Asian/South American ISPs) torrenting updates to users is significantly faster than an HTTPS download from Amazon, Google Cloud & the like. Once you get a handful of seeders internal to the network, the other downloaders in that network start to get the torrent significantly faster than they would over traditional HTTPS. If the content isn't extremely time sensitive (eg: live sports games, video calls) and is of a noteworthy size, using P2P tech to speed up downloads is not a bad idea.
- kalleboo 8y ago
  Theoretically BitTorrent should be more efficient, since peers nearby can send data directly (e.g. within an ISP or regional node), whereas a central server will always have to transfer a longer distance and involve more infrastructure. E.g. with my ISP, the bottleneck isn't in the last mile (that's all GPON fiber) but rather in the national network and their transit to the internet. If more transfer was made to local-to-local, it would reduce the amount of traffic that needs to go cross-country or out on the internet.
  - bb88 8y ago
    I don't think that's true. Unless bittorrent understands the topology of a network, it will never be more efficient than a structure that takes advantage of the topology. E.g. Netflix makes servers that can be distributed to the ISP's and can serve content to those consumers directly from the ISP itself, and not over the ISP's outgoing link. E.g.#2 Multicast clients register to receive multicast streams and the routers know how to propagate multicast.
    - jstanley 8y ago
      It doesn't have to understand the topology, it can just understand which peers can serve bytes fastest. (I don't know if it does this, but it's been around long enough that I'd be surprised if it doesn't).
      - bb88 8y ago
        That doesn't make it more efficient. It might make it faster at the expense of using your ISP's outgoing bandwidth multiple times over. The most efficient network distribution is one where the data only travels each network link once for all destinations. This is the way multicast works, e.g.
    - T-Winsnes 8y ago
      I think it can be done simpler than that. When the client/server has a choice of multiple peers to connect to, it can pick the one with the lower latency. That way you end up picking peers that are closer and will help lessen the load on the overall network. Not perfect, but a pretty good 90% solution
Retric 8y ago
For widespread distribution it’s a massive net reduction in traffic across internet backbones relative to centralized cervers. If files go from network A > B > C to two users on network C they can send packets to each other without going to networks A or B. The protocol has quite a bit of overhead, so things can be worse among a small number of clients. It’s also worse than having caching servers on each network.
- wmf 8y ago
  But people haven't used centralized servers to distribute large static data for years; they use CDNs which have massive capacity close to users.
  - AnthonyMouse 8y ago
    If the data in the CDN is being downloaded by multiple users in each location then there would be that many local peers with it in a distributed network. If the peers give preference to e.g. lowest ping then they're equally efficient. If the data is unpopular then the CDN is less efficient because it transfers a copy to each node even if nobody ever wants it.
  - Retric 8y ago
    Networks are fractal in nature. Just became you have one or more CDN servers inside Comcast’s network does not mean you’re as close to every machine as two neighbors exchanging files. From a business standpoint it does not matter much, but from a pure network perspective it’s likely worse.
  - realusername 8y ago
    CDNs are also centralized servers, you just have more of them.
  - zormino 8y ago
    Many large downloads (at least for some games) driven not through your browser will also by default have a peer to peer option enabled. Battle.net comes to mind for me.
toast0 8y ago
Bit torrent is probably more absolute bits sent per byte downloaded than http, just because there's the management traffic. Ignoring that, bit torrent is likely to do the bulk transfer over different links than a download from a central server. This seems likely to result in an overall higher number of bytes downloaded, but it's hard to model. It can also lead to congestion in parts of the network that were not normally congested.
- em-bee 8y ago
  doesn't it also help get around already congested routes?

by berbec 8y ago

dan-robertson 8y ago

It also leads to more balance between upload/download. This is perhaps less good on typical broadband which usually has a much higher bandwidth for downloads but a typical datacentre will have a much greater upload demand to download (there’s a reason that downloading to an aws box is much cheaper than uploading). On the other hand, big networks can be more efficient by caching things near the edge so that the total “bit miles” (I.e. the total over all packets sent of distance traveled * bits of data) are decreased.

jjeaff 8y ago
I'd say BitTorrent is likely just as efficient or much more so at keeping distances short. At least, it optimizes by taking the fastest available peers. Your typical big torrent file could have thousands of peers. That's more endpoints than most any other big CDN and more likely to have a few peers really close to you. Heck, it could even download from someone on your local network if they happened to be serving up the file you are looking for.

flatline 8y ago

Centralization will always be more efficient, generally speaking. Access to a centralized channel for distribution can be prohibitive for many reasons - cost and censorship being the foremost in my mind.

StudentStuff 8y ago
I'm doubtful of this premise, especially in the context of peering disputes. Here in the states, Netflix had/has serious issues with Comcast/Verizon/Centurylink not peering with their upstream providers sufficently to handle the number of video streams that needed to be carried. This was not an issue with the last mile connection itself, but a middle mile peering problem. From the testing I have done, on certain carriers that play games like this (Deutsch Telekom in Germany, Centurylink, Verizon & others in the US, and most Asian/South American ISPs) torrenting updates to users is significantly faster than an HTTPS download from Amazon, Google Cloud & the like. Once you get a handful of seeders internal to the network, the other downloaders in that network start to get the torrent significantly faster than they would over traditional HTTPS. If the content isn't extremely time sensitive (eg: live sports games, video calls) and is of a noteworthy size, using P2P tech to speed up downloads is not a bad idea.
kalleboo 8y ago
Theoretically BitTorrent should be more efficient, since peers nearby can send data directly (e.g. within an ISP or regional node), whereas a central server will always have to transfer a longer distance and involve more infrastructure. E.g. with my ISP, the bottleneck isn't in the last mile (that's all GPON fiber) but rather in the national network and their transit to the internet. If more transfer was made to local-to-local, it would reduce the amount of traffic that needs to go cross-country or out on the internet.
- bb88 8y ago
  I don't think that's true. Unless bittorrent understands the topology of a network, it will never be more efficient than a structure that takes advantage of the topology. E.g. Netflix makes servers that can be distributed to the ISP's and can serve content to those consumers directly from the ISP itself, and not over the ISP's outgoing link. E.g.#2 Multicast clients register to receive multicast streams and the routers know how to propagate multicast.
  - jstanley 8y ago
    It doesn't have to understand the topology, it can just understand which peers can serve bytes fastest. (I don't know if it does this, but it's been around long enough that I'd be surprised if it doesn't).
    - bb88 8y ago
      That doesn't make it more efficient. It might make it faster at the expense of using your ISP's outgoing bandwidth multiple times over. The most efficient network distribution is one where the data only travels each network link once for all destinations. This is the way multicast works, e.g.
  - T-Winsnes 8y ago
    I think it can be done simpler than that. When the client/server has a choice of multiple peers to connect to, it can pick the one with the lower latency. That way you end up picking peers that are closer and will help lessen the load on the overall network. Not perfect, but a pretty good 90% solution

Retric 8y ago

For widespread distribution it’s a massive net reduction in traffic across internet backbones relative to centralized cervers. If files go from network A > B > C to two users on network C they can send packets to each other without going to networks A or B. The protocol has quite a bit of overhead, so things can be worse among a small number of clients. It’s also worse than having caching servers on each network.

wmf 8y ago
But people haven't used centralized servers to distribute large static data for years; they use CDNs which have massive capacity close to users.
- AnthonyMouse 8y ago
  If the data in the CDN is being downloaded by multiple users in each location then there would be that many local peers with it in a distributed network. If the peers give preference to e.g. lowest ping then they're equally efficient. If the data is unpopular then the CDN is less efficient because it transfers a copy to each node even if nobody ever wants it.
- Retric 8y ago
  Networks are fractal in nature. Just became you have one or more CDN servers inside Comcast’s network does not mean you’re as close to every machine as two neighbors exchanging files. From a business standpoint it does not matter much, but from a pure network perspective it’s likely worse.
- realusername 8y ago
  CDNs are also centralized servers, you just have more of them.
- zormino 8y ago
  Many large downloads (at least for some games) driven not through your browser will also by default have a peer to peer option enabled. Battle.net comes to mind for me.

toast0 8y ago

Bit torrent is probably more absolute bits sent per byte downloaded than http, just because there's the management traffic. Ignoring that, bit torrent is likely to do the bulk transfer over different links than a download from a central server. This seems likely to result in an overall higher number of bytes downloaded, but it's hard to model. It can also lead to congestion in parts of the network that were not normally congested.

em-bee 8y ago
doesn't it also help get around already congested routes?