5 ms·

I wish apt, dnf/rpm, flatpak, etc utilized a decentralized distribution option, like IPFS or BEP46 Mutable Torrents. It would be neat if the project leads seede

by xd1936 2y ago

I wish apt, dnf/rpm, flatpak, etc utilized a decentralized distribution option, like IPFS or BEP46 Mutable Torrents. It would be neat if the project leads seeded new package update hashes, volunteers ran seedboxes instead of http mirrors, and clients had (default-on?) seeding of package binaries in addition to only downloading. It would be neat to see the open source community contributing to support each other's experience.

spookie 2y ago
At the end of the day, there's a trust issue. The way distribution works nowadays, mitigates a lot of those. Making it decentralised would be a step backwards.
- ciupicri 2y ago
  Aren't the RPMs signed?
- BSDobelix 2y ago
  You know that packages are signed right? That's why everyone can be a libreoffice or ArchLinux Mirror...or Fedora?
  - deleted 2y ago
    [deleted]
  - aaomidi 2y ago
    And because of this, these mirrors are often non https so your ISP can actually intercept and provide the data to you directly. This is why torrents would actually work fine in this model.
    - Cyphase 2y ago
      Where intercept usually means point DNS for the mirror domains to the ISP's local mirror, and ISP could mean your cloud provider.
    - BSDobelix 2y ago
      >ISP can actually intercept and provide the data to you directly What a nice gesture! However my mirror-servers are https plus rsync (for some projects). But are there not some ISP's that block torrent-traffic completely?
      - aaomidi 2y ago
        BitTorrent is hard to block completely from my understanding of how the protocol works, but I may be wrong.
    - crest 2y ago
      Unless you can break the SHA1 hashes for the torrent chunks and the global hash torrents also authenticate the content. If you have a valid "info" dictionary at least.
  - adobrawy 2y ago
    A digital signature does not protect you against a malicious actor who starts distributing outdated, vulnerable versions to you.
    - BSDobelix 2y ago
      How do i know you don't know what your talking about?....a mystery ;)
      - adobrawy 2y ago
        I mean replay attack and freeze attack as described in https://doi.org/10.1145/1455770.1455841 https://doi.org/10.1145/1455770.1455841 . It's very likely that I'm not up to date on mitigations in individual package managers.
        BSDobelix 2y ago
        >all of these package managers have vulnerabilities that can be exploited by a man-in-the-middle or a malicious mirror. Well, that's how software is, but that article is from 2008 and things have gotten a lot better in the meantime (I think archlinux wasn't even signing their packages back then). If the distribution's private keyring isn't compromised and you don't have third-party repos, your packages are as trustworthy as your distribution team (and upstream).
- Cyphase 2y ago
  Packages can be and generally are signed and verified.
candiddevmike 2y ago
This would make it so much easier for end users to cache/host a mirror than the myriad of tools and hacks like squid. I'm sure this will happen once the Linux community standardizes on a package format /s.
j-bos 2y ago
Why aren't packages based on torrents? Most distros come with a torrent client.
- utensil4778 2y ago
  Honestly, default behavior should be to at least share package files on the local network. But sharing to the wider internet should be fairly trivial in <current year>, we have no shortage of technologies that accomplish this. I wish more systems had this kind of feature. I have a fat fiber connection, I'd be thrilled to pop up an unofficial mirror for something like a Linux distro. I've tried mirroring Linux ISO torrents, but it seems almost nobody ever downloads from a torrent, so I end up never actually uploading any of these images.
- Joe_Cool 2y ago
  Torrents don't handle frequent updates very well. You'd have tons of outdated torrent floating around. They work well for archives and installation media though that doesn't change that often.
- enasterosophes 2y ago
  Torrents tank when there is lack of popularity. Linux can be configured for a wide variety of purposes, from IoT driver to gaming workstation, so not all packages are equally popular with all people, and just because a package is unpopular doesn't mean it isn't important. If I'm the one person in the world running a service that everyone else relies on, then the packages which support that service are important even if no one else uses them. Therefore to use torrents requires one of two choices, neither of which are really viable when you think about it. Either: * Require everyone who uses the distribution to torrent releases of the full set of packages for the distribution. (And remember that packages regularly get updated even in LTS releases, due to backporting security fixes etc.) It means everyone using the distro really needs to be running dozens or hundreds of torrents. * Everyone torrents only the packages they need, so access to packages becomes up to popular vote instead of actual importance. Torrents are fine for isos but intractible for ongoing package management.
  - 2y ago
Joe_Cool 2y ago
You could update Archlinux with pacman over ipfs from around 2015 until last year. https://github.com/ipfs/notes/issues/84 https://github.com/ipfs/notes/issues/84 Then the mirror became too slow and couldn't handle the amount of package data and was shut down: https://github.com/RubenKelevra/pacman.store https://github.com/RubenKelevra/pacman.store It worked rather well and even automatically mirrored the packages on my LAN. Maybe it'll be back some day.
- yjftsjthsd-h 2y ago
  I'm not quite following (including trying to skim your links) - what mirror? I thought the point was that it was decentralized and should scale arbitrarily?
  - Joe_Cool 2y ago
    It worked like this: rsync2ipfs-cluster fetched the packages from a rsync mirror into an IPFS-MFS-Folder updated the link in IPNS and then added this folder to a running IPFS instance/cluster. All of those packages are then available for pacman under the (unchanging) IPNS name and the packages themselves are stored in IPFS as usual files. All you had to was add `http://x86-64.archlinux.pkg.pacman.store.ipns.localhost:8080/$repo http://x86-64.archlinux.pkg.pacman.store.ipns.localhost:8080...` to your mirror list after running your ipfs node (or use a public ipfs gateway). The website sadly is down now. It had a nicer explanation. None of this still works now however. EDIT: the folder/list of files was managed by the user hosting the project. The files/packages themselves were distributed. That's just how IPFS/IPNS works. Somebody has to put them into IPFS. EDIT2: technically the example I posted is a DNSLink which points to a (now gone) TXT record which points to an IPNS name (which is a mutable pointer) that contained the directory of all current archlinux packages. I hope that was comprehensible.
    - yjftsjthsd-h 2y ago
      Ah, so when you say "Then the mirror became too slow and couldn't handle the amount of package data and was shut down" you're talking about the side that added packages to IPFS and did the initial seeding. That's what I wasn't following, thx.
XorNot 2y ago
I have a local MITM Squid proxy on my home network to try and alleviate this problem, but it's why the prevalence of TLS with no protocol design for how we do proxying has gotten pretty annoying. I did try doing something fun with network-interface scripts and avahi to make this more dynamic - the idea was my desktop and laptop both ran squid proxies, and then dynamically set peer proxies if they detected each other on the network. It didn't work great given the overhead. It really feels like there should be a better way to do this in general, and without having to break TLS directly for things which are not really "secret".

by xd1936 2y ago

spookie 2y ago

At the end of the day, there's a trust issue. The way distribution works nowadays, mitigates a lot of those. Making it decentralised would be a step backwards.

ciupicri 2y ago
Aren't the RPMs signed?
BSDobelix 2y ago
You know that packages are signed right? That's why everyone can be a libreoffice or ArchLinux Mirror...or Fedora?
- deleted 2y ago
  [deleted]
- aaomidi 2y ago
  And because of this, these mirrors are often non https so your ISP can actually intercept and provide the data to you directly. This is why torrents would actually work fine in this model.
  - Cyphase 2y ago
    Where intercept usually means point DNS for the mirror domains to the ISP's local mirror, and ISP could mean your cloud provider.
  - BSDobelix 2y ago
    >ISP can actually intercept and provide the data to you directly What a nice gesture! However my mirror-servers are https plus rsync (for some projects). But are there not some ISP's that block torrent-traffic completely?
    - aaomidi 2y ago
      BitTorrent is hard to block completely from my understanding of how the protocol works, but I may be wrong.
  - crest 2y ago
    Unless you can break the SHA1 hashes for the torrent chunks and the global hash torrents also authenticate the content. If you have a valid "info" dictionary at least.
- adobrawy 2y ago
  A digital signature does not protect you against a malicious actor who starts distributing outdated, vulnerable versions to you.
  - BSDobelix 2y ago
    How do i know you don't know what your talking about?....a mystery ;)
    - adobrawy 2y ago
      I mean replay attack and freeze attack as described in https://doi.org/10.1145/1455770.1455841 https://doi.org/10.1145/1455770.1455841 . It's very likely that I'm not up to date on mitigations in individual package managers.
      - BSDobelix 2y ago
        >all of these package managers have vulnerabilities that can be exploited by a man-in-the-middle or a malicious mirror. Well, that's how software is, but that article is from 2008 and things have gotten a lot better in the meantime (I think archlinux wasn't even signing their packages back then). If the distribution's private keyring isn't compromised and you don't have third-party repos, your packages are as trustworthy as your distribution team (and upstream).
Cyphase 2y ago
Packages can be and generally are signed and verified.

candiddevmike 2y ago

This would make it so much easier for end users to cache/host a mirror than the myriad of tools and hacks like squid. I'm sure this will happen once the Linux community standardizes on a package format /s.

j-bos 2y ago

Why aren't packages based on torrents? Most distros come with a torrent client.

utensil4778 2y ago
Honestly, default behavior should be to at least share package files on the local network. But sharing to the wider internet should be fairly trivial in <current year>, we have no shortage of technologies that accomplish this. I wish more systems had this kind of feature. I have a fat fiber connection, I'd be thrilled to pop up an unofficial mirror for something like a Linux distro. I've tried mirroring Linux ISO torrents, but it seems almost nobody ever downloads from a torrent, so I end up never actually uploading any of these images.
Joe_Cool 2y ago
Torrents don't handle frequent updates very well. You'd have tons of outdated torrent floating around. They work well for archives and installation media though that doesn't change that often.
enasterosophes 2y ago
Torrents tank when there is lack of popularity. Linux can be configured for a wide variety of purposes, from IoT driver to gaming workstation, so not all packages are equally popular with all people, and just because a package is unpopular doesn't mean it isn't important. If I'm the one person in the world running a service that everyone else relies on, then the packages which support that service are important even if no one else uses them. Therefore to use torrents requires one of two choices, neither of which are really viable when you think about it. Either: * Require everyone who uses the distribution to torrent releases of the full set of packages for the distribution. (And remember that packages regularly get updated even in LTS releases, due to backporting security fixes etc.) It means everyone using the distro really needs to be running dozens or hundreds of torrents. * Everyone torrents only the packages they need, so access to packages becomes up to popular vote instead of actual importance. Torrents are fine for isos but intractible for ongoing package management.
- 2y ago

Joe_Cool 2y ago

You could update Archlinux with pacman over ipfs from around 2015 until last year. https://github.com/ipfs/notes/issues/84 https://github.com/ipfs/notes/issues/84 Then the mirror became too slow and couldn't handle the amount of package data and was shut down: https://github.com/RubenKelevra/pacman.store https://github.com/RubenKelevra/pacman.store It worked rather well and even automatically mirrored the packages on my LAN. Maybe it'll be back some day.

yjftsjthsd-h 2y ago
I'm not quite following (including trying to skim your links) - what mirror? I thought the point was that it was decentralized and should scale arbitrarily?
- Joe_Cool 2y ago
  It worked like this: rsync2ipfs-cluster fetched the packages from a rsync mirror into an IPFS-MFS-Folder updated the link in IPNS and then added this folder to a running IPFS instance/cluster. All of those packages are then available for pacman under the (unchanging) IPNS name and the packages themselves are stored in IPFS as usual files. All you had to was add `http://x86-64.archlinux.pkg.pacman.store.ipns.localhost:8080/$repo http://x86-64.archlinux.pkg.pacman.store.ipns.localhost:8080...` to your mirror list after running your ipfs node (or use a public ipfs gateway). The website sadly is down now. It had a nicer explanation. None of this still works now however. EDIT: the folder/list of files was managed by the user hosting the project. The files/packages themselves were distributed. That's just how IPFS/IPNS works. Somebody has to put them into IPFS. EDIT2: technically the example I posted is a DNSLink which points to a (now gone) TXT record which points to an IPNS name (which is a mutable pointer) that contained the directory of all current archlinux packages. I hope that was comprehensible.
  - yjftsjthsd-h 2y ago
    Ah, so when you say "Then the mirror became too slow and couldn't handle the amount of package data and was shut down" you're talking about the side that added packages to IPFS and did the initial seeding. That's what I wasn't following, thx.

XorNot 2y ago

I have a local MITM Squid proxy on my home network to try and alleviate this problem, but it's why the prevalence of TLS with no protocol design for how we do proxying has gotten pretty annoying. I did try doing something fun with network-interface scripts and avahi to make this more dynamic - the idea was my desktop and laptop both ran squid proxies, and then dynamically set peer proxies if they detected each other on the network. It didn't work great given the overhead. It really feels like there should be a better way to do this in general, and without having to break TLS directly for things which are not really "secret".