4 ms·

That’s very cool! That addresses the basic update issue, although I would be surprised if there was a production-ready Ruby library for torrents that included t

That’s very cool! That addresses the basic update issue, although I would be surprised if there was a production-ready Ruby library for torrents that included these. The state of HTTP(S) in Ruby is sad enough :-) (There’s also still the state/seeding problem and its collision with user expectations around brew getting faster, or at least not any slower.)

Lammy 2y ago
I agree with you about package manager usage patterns being a poor fit for seeding by end users. I definitely wouldn't want my computer to participate. I could see institutional seeders doing it as a way to donate bandwidth though, like a CDN that's built into the distribution protocol instead of getting load-balanced to Microsoft's nearest PoP when hitting a GitHub `ghcr.io` URI like Homebrew does today. Or even better, use that as an HTTP Seed (BEP-0019) to combine benefits of both :) https://www.bittorrent.org/beps/bep_0019.html https://www.bittorrent.org/beps/bep_0019.html
- woodruffw 2y ago
  Yeah, something at the institutional layer makes sense. Thank you for sharing these links! (My skepticism around whether this makes sense for Homebrew might be obscuring it, but I’m generally quite a big fan of distributed/P2P protocols, and I strongly believe that existing CDN decencies in packaging ecosystems are a risk that needs mitigating.)
hackingonempty 2y ago
There isn't an update issue. BitTorrent metadata are hashes, they get updated and distributed at the same time as the current URLs and hashes in the same file or maybe in a similarly named file right next to it in the same pull request. There is no state/seeding problem. The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.
- woodruffw 2y ago
  The adjacent thread observes that there is an update issue, just one that has a technical solution. > The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin. So some kind of hybrid scheme, which (to me) implies the worst of both worlds: clients are still going to hammer upstreams on package updates (since client traffic isn’t uniform), and every client pays a bunch of peering overhead that doesn’t pay off until the files are “hot.” In other words, upstreams still need to plan for the same amount of capacity, and clients have to do more work. (The adjacent thread observes that none of this is necessary if CDNs or other large operators do this between themselves, rather than involving clients. That seems strictly preferable to me.)

Lammy 2y ago

I agree with you about package manager usage patterns being a poor fit for seeding by end users. I definitely wouldn't want my computer to participate. I could see institutional seeders doing it as a way to donate bandwidth though, like a CDN that's built into the distribution protocol instead of getting load-balanced to Microsoft's nearest PoP when hitting a GitHub `ghcr.io` URI like Homebrew does today. Or even better, use that as an HTTP Seed (BEP-0019) to combine benefits of both :) https://www.bittorrent.org/beps/bep_0019.html https://www.bittorrent.org/beps/bep_0019.html

woodruffw 2y ago
Yeah, something at the institutional layer makes sense. Thank you for sharing these links! (My skepticism around whether this makes sense for Homebrew might be obscuring it, but I’m generally quite a big fan of distributed/P2P protocols, and I strongly believe that existing CDN decencies in packaging ecosystems are a risk that needs mitigating.)

hackingonempty 2y ago

There isn't an update issue. BitTorrent metadata are hashes, they get updated and distributed at the same time as the current URLs and hashes in the same file or maybe in a similarly named file right next to it in the same pull request. There is no state/seeding problem. The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin.

woodruffw 2y ago
The adjacent thread observes that there is an update issue, just one that has a technical solution. > The client downloads from the same https url as always but uses peers on an as-available basis to speed things up and reduce load on the origin. So some kind of hybrid scheme, which (to me) implies the worst of both worlds: clients are still going to hammer upstreams on package updates (since client traffic isn’t uniform), and every client pays a bunch of peering overhead that doesn’t pay off until the files are “hot.” In other words, upstreams still need to plan for the same amount of capacity, and clients have to do more work. (The adjacent thread observes that none of this is necessary if CDNs or other large operators do this between themselves, rather than involving clients. That seems strictly preferable to me.)