3 ms·

Reminds me of those "alternative" bittorrent clients that did neat things by, basically, behaving in selfish ways. The main example behavior I remember was tha

Reminds me of those "alternative" bittorrent clients that did neat things by, basically, behaving in selfish ways. The main example behavior I remember was that these clients would sort of cheat by downloading the file chunks in sequential order (so they could start playing the media sooner) rather than in random order (which is a big part of what makes bittorrent a pretty effective protocol/algorithm)

fmajid 3y ago
That's pretty much what Google is doing with QUIC and BBR, aka HTTP/3. Tragedy of the Commons in the making.
nix0n 3y ago
If multiple users download in sequential order, wouldn't that actually make it pretty likely that the previous downloader would have the chunk that the next downloader is looking for? Or is there a specific advantage to random order?
- the8472 3y ago
  see http://bittorrent.org/bittorrentecon.pdf http://bittorrent.org/bittorrentecon.pdf section 2.4.2
  - nix0n 3y ago
    Thanks! That answers my question. For anyone who doesn't feel like clicking on a PDF, the section title is "Rarest First".
- TonyTrapp 3y ago
  With clients constantly appearing and disappearing, the probability that you will be able to finish the download will be much higher if everyone has a random collection of file parts. Imagine the original seeder goes away after one day. Everyone is stuck at 99% because they downloaded in sequential order. Nobody has the last missing part. This would be much less likely if the clients had downloaded random parts instead, because every part would be roughly equally likely to be available in the end.
FeepingCreature 3y ago
If lots of people are watching the file sequentially, then sequential order is close to optimal. Consider a Bittorrent-based lifestream. Clients should focus on getting and distributing the chunks that everyone wants.
- JohnBooty 3y ago
  Bittorrent does livestreams? Did not know that. But for "normal" Bittorrent usage, oh hell naaaaah. Where you are trying to actually obtain a complete file, that can be a disaster. What happens is the chunks at the end of the file will be much rarer than the chunks at the beginning. Getting the blocks sequentially offers a small convenience benefit (you can start viewing a video while the torrent is still downloading) but if significant numbers of people did that it would harm every other use case including your own ability to actually download the entire file. If you have a surplus of seeders this isn't a problem, but for sparsely-seeded torrents that is not good.
  - zamadatix 3y ago
    This implies the reason most sparsely seeded torrents have chunk availability is because the transient leechers just happen to arrive consistently in time to hand off to one another. I think reality is sparse torrents tend to be kept alive by a few seeders slowly kicking out data (most likely because they are seeding a ton of other stuff too). Sequential or not, if you and another guy are planning on dumping a few seconds after finishing the torrent then it doesn't really matter if that is a chunk near the end or a random chunk - either way you're both trying to squeeze the last chunk out of the seeder. Densely seeded torrents may be slightly different and play out more like you describe, but it doesn't really matter as much there as the first parts can be sourced from those watching sequentially and the end parts left to those that actually seed after getting a full file.
    - JohnBooty 3y ago
      This implies the reason most sparsely seeded torrents have chunk availability is because the transient leechers just happen to arrive consistently in time to hand off to one another. No. I'm not sure where you got that idea. This is a pathological and unrealistic example but please try to extrapolate to more realistic scenarios. Imagine a file with three pieces. 200 torrent users have zero pieces, 100 torrent users have the first piece, 50 have the first and second piece, and "Joe" is the sole seeder that has all three. Obviously, assuming Joe doesn't have godlike bandwidth, there are 350 users who want that third piece and only a single Joe and there is going to be contention. In this pathological situation things will work itself out as soon as supply of the third piece increases, assuming people don't kill their clients as soon as they hit 100%, but things will be slower than they need to be at first. What I think you're trying to point out is that there are situations where sequential torrent downloads aren't harmful to download speed or overall availability. Yes, that is true. If seeder bandwidth for a given piece is more than sufficient for leecher demand for that piece then... yes, no harm done. As another user posted, http://bittorrent.org/bittorrentecon.pdf http://bittorrent.org/bittorrentecon.pdf section 2.4.2 "Rarest first" You're free to disagree with the design of bittorrent, obviously.

fmajid 3y ago

That's pretty much what Google is doing with QUIC and BBR, aka HTTP/3. Tragedy of the Commons in the making.

nix0n 3y ago

If multiple users download in sequential order, wouldn't that actually make it pretty likely that the previous downloader would have the chunk that the next downloader is looking for? Or is there a specific advantage to random order?

the8472 3y ago
see http://bittorrent.org/bittorrentecon.pdf http://bittorrent.org/bittorrentecon.pdf section 2.4.2
- nix0n 3y ago
  Thanks! That answers my question. For anyone who doesn't feel like clicking on a PDF, the section title is "Rarest First".
TonyTrapp 3y ago
With clients constantly appearing and disappearing, the probability that you will be able to finish the download will be much higher if everyone has a random collection of file parts. Imagine the original seeder goes away after one day. Everyone is stuck at 99% because they downloaded in sequential order. Nobody has the last missing part. This would be much less likely if the clients had downloaded random parts instead, because every part would be roughly equally likely to be available in the end.

FeepingCreature 3y ago

If lots of people are watching the file sequentially, then sequential order is close to optimal. Consider a Bittorrent-based lifestream. Clients should focus on getting and distributing the chunks that everyone wants.

JohnBooty 3y ago
Bittorrent does livestreams? Did not know that. But for "normal" Bittorrent usage, oh hell naaaaah. Where you are trying to actually obtain a complete file, that can be a disaster. What happens is the chunks at the end of the file will be much rarer than the chunks at the beginning. Getting the blocks sequentially offers a small convenience benefit (you can start viewing a video while the torrent is still downloading) but if significant numbers of people did that it would harm every other use case including your own ability to actually download the entire file. If you have a surplus of seeders this isn't a problem, but for sparsely-seeded torrents that is not good.
- zamadatix 3y ago
  This implies the reason most sparsely seeded torrents have chunk availability is because the transient leechers just happen to arrive consistently in time to hand off to one another. I think reality is sparse torrents tend to be kept alive by a few seeders slowly kicking out data (most likely because they are seeding a ton of other stuff too). Sequential or not, if you and another guy are planning on dumping a few seconds after finishing the torrent then it doesn't really matter if that is a chunk near the end or a random chunk - either way you're both trying to squeeze the last chunk out of the seeder. Densely seeded torrents may be slightly different and play out more like you describe, but it doesn't really matter as much there as the first parts can be sourced from those watching sequentially and the end parts left to those that actually seed after getting a full file.
  - JohnBooty 3y ago
    This implies the reason most sparsely seeded torrents have chunk availability is because the transient leechers just happen to arrive consistently in time to hand off to one another. No. I'm not sure where you got that idea. This is a pathological and unrealistic example but please try to extrapolate to more realistic scenarios. Imagine a file with three pieces. 200 torrent users have zero pieces, 100 torrent users have the first piece, 50 have the first and second piece, and "Joe" is the sole seeder that has all three. Obviously, assuming Joe doesn't have godlike bandwidth, there are 350 users who want that third piece and only a single Joe and there is going to be contention. In this pathological situation things will work itself out as soon as supply of the third piece increases, assuming people don't kill their clients as soon as they hit 100%, but things will be slower than they need to be at first. What I think you're trying to point out is that there are situations where sequential torrent downloads aren't harmful to download speed or overall availability. Yes, that is true. If seeder bandwidth for a given piece is more than sufficient for leecher demand for that piece then... yes, no harm done. As another user posted, http://bittorrent.org/bittorrentecon.pdf http://bittorrent.org/bittorrentecon.pdf section 2.4.2 "Rarest first" You're free to disagree with the design of bittorrent, obviously.