3 ms·
> My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes. Kinda. You do create a new torrent, but y
by Lammy 2y ago
> My understanding is that there's no standard way to update a torrent file; you re-roll a new file with the changes.
Kinda. You do create a new torrent, but you distribute it in a way that to a swarm member is functionally equivalent to updating an old one. Check out BEP-0039 and BEP-0046 which respectively cover the HTTP and DHT mechanisms for updating torrents:
https://www.bittorrent.org/beps/bep_0039.html https://www.bittorrent.org/beps/bep_0039.html
https://www.bittorrent.org/beps/bep_0046.html https://www.bittorrent.org/beps/bep_0046.html
If that updated torrent is a BEP-0052 (v2) torrent it will hash per-file, and so the updated v2 torrent will have identical hashes for files which aren't changed: https://www.bittorrent.org/beps/bep_0052.html https://www.bittorrent.org/beps/bep_0052.html
This combines with BEP-0038 so the updated torrent can refer to the infohash of the older torrent with which it shares files, so if you already have the old one you only have to download files that have changed: https://www.bittorrent.org/beps/bep_0038.html https://www.bittorrent.org/beps/bep_0038.html
- woodruffw 2y agoThat’s very cool! That addresses the basic update issue, although I would be surprised if there was a production-ready Ruby library for torrents that included these. The state of HTTP(S) in Ruby is sad enough :-) (There’s also still the state/seeding problem and its collision with user expectations around brew getting faster, or at least not any slower.)
- Lammy 2y agoI agree with you about package manager usage patterns being a poor fit for seeding by end users. I definitely wouldn't want my computer to participate. I could see institutional seeders doing it as a way to donate bandwidth though, like a CDN that's built into the distribution protocol instead of getting load-balanced to Microsoft's nearest PoP when hitting a GitHub `ghcr.io` URI like Homebrew does today. Or even better, use that as an HTTP Seed (BEP-0019) to combine benefits of both :) https://www.bittorrent.org/beps/bep_0019.html https://www.bittorrent.org/beps/bep_0019.html
- woodruffw 2y agoYeah, something at the institutional layer makes sense. Thank you for sharing these links! (My skepticism around whether this makes sense for Homebrew might be obscuring it, but I’m generally quite a big fan of distributed/P2P protocols, and I strongly believe that existing CDN decencies in packaging ecosystems are a risk that needs mitigating.)
- hackingonempty 2y agoThere isn't an update issue. BitTorrent metadata are hashes, they get updated and distributed at the same time as the current URLs and hashes in the same file or maybe in a similarly named file right next to it in the same pull request. There is no state/seeding problem. The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.
- woodruffw 2y agoThe adjacent thread observes that there is an update issue, just one that has a technical solution. > The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin. So some kind of hybrid scheme, which (to me) implies the worst of both worlds: clients are still going to hammer upstreams on package updates (since client traffic isn’t uniform), and every client pays a bunch of peering overhead that doesn’t pay off until the files are “hot.” In other words, upstreams still need to plan for the same amount of capacity, and clients have to do more work. (The adjacent thread observes that none of this is necessary if CDNs or other large operators do this between themselves, rather than involving clients. That seems strictly preferable to me.)