3 ms·

The nix store seems on the face of it like a perfect use case for something like IPFS

rkeene2 3y ago
My AppFS project [0] is basically this, a content-addressable remote package-oriented system. [0] https://AppFS.net/ https://AppFS.net/
pxc 3y ago
There has been work ongoing to support this in Nix for a long time, actually. Maybe this will help push that effort along, or direct more resources towards it. See: https://github.com/NixOS/rfcs/pull/133 https://github.com/NixOS/rfcs/pull/133
nlitened 3y ago
Or torrents
shp0ngle 3y ago
IPFS in not magic you can just sprinkle on things, say “p2p! decentralised!” and be done. you still need either pay someone to “pin it”, which is more expensive than dumping it on s3. Pinning services (that’s a thing) are all sketchy (as the big ones are all somehow connected to NFTs) and if you do it yourself it’s actually harder than just storing it
- mindslight 3y ago
  As a NixOS user (who doesn't even use the binaries), I would happily pin NixOS packages! The point of IPFS is to decouple the names of resources from what's serving them. It's exactly what is needed for distributed libre software projects where many are willing to donate some resources, but few are able to donate all the resources.
  - joepie91_ 3y ago
    But the problem here isn't distribution, it's durable storage. Something along the lines of Tahoe-LAFS would be far more suitable.
    - mindslight 3y ago
      I had missed that the storage fees were significantly larger than the traffic fees. I still don't know if Tahoe is appropriate though. It's been a while since I looked at it, but doesn't it work with a fixed set of storage nodes? That wouldn't work for nodes coming/going where people can join organically.
esjeon 3y ago
Yeah, it doesn't make sense for the team to keep hosting every single state of the distro, just to boost building some random dudes' old/peculiar setup. It's almost like Nix team is providing an artifact hosting service. Nix team should expire old versions from their repository, and let people pin what they need on IPFS.
hamandcheese 3y ago
Unfortunately it currently isn't, because nix packages are not content addressed. They are "input addressed", which means that when requesting the contents of a derivation, you have to trust the source to give you the right bytes. There are efforts underway to make things content addressed. This doesn't solve the trust problem -- you still have to trust someone to tell you which inputs map to which content address -- but it does help with the distribution problem. Mirrors no longer have to be trusted. Trust and distribution become decoupled.
- sitkack 3y ago
  If you went "meta-nix" and had a nix package hierarchy that was just constructed out of operations and inline literal binary blocks, then the results of those purely functional operations (splice, insert, f(n), swap, etc) could be content-addressed and deduplicated. Views of purely functional data transformations of f_tran(n)(addr_start:addr_end), but if you symbolize it you, you make it generic. The problem is that Nix isn't Haskell enough. It also sounds like if they stored the cache on a deduping file system that their storage costs would plummet.
  - kaba0 3y ago
    > If you went "meta-nix" and had a nix package hierarchy that was just constructed out of operations and inline literal binary blocks, then the results of those purely functional operations (splice, insert, f(n), swap, etc) could be content-addressed and deduplicated. How do you build with an arbitrary build tool then? You have to call out to gcc/any other compiler in the process.
    - sitkack 3y ago
      It ads another layer of indirection, you can still consume the artifacts of the process using normal tooling.
      - kaba0 3y ago
        I might be missing something — how do you decide a priori building what will be the output hash?

My AppFS project [0] is basically this, a content-addressable remote package-oriented system. [0] https://AppFS.net/ https://AppFS.net/

pxc 3y ago

There has been work ongoing to support this in Nix for a long time, actually. Maybe this will help push that effort along, or direct more resources towards it. See: https://github.com/NixOS/rfcs/pull/133 https://github.com/NixOS/rfcs/pull/133

nlitened 3y ago

Or torrents

shp0ngle 3y ago

IPFS in not magic you can just sprinkle on things, say “p2p! decentralised!” and be done. you still need either pay someone to “pin it”, which is more expensive than dumping it on s3. Pinning services (that’s a thing) are all sketchy (as the big ones are all somehow connected to NFTs) and if you do it yourself it’s actually harder than just storing it

mindslight 3y ago
As a NixOS user (who doesn't even use the binaries), I would happily pin NixOS packages! The point of IPFS is to decouple the names of resources from what's serving them. It's exactly what is needed for distributed libre software projects where many are willing to donate some resources, but few are able to donate all the resources.
- joepie91_ 3y ago
  But the problem here isn't distribution, it's durable storage. Something along the lines of Tahoe-LAFS would be far more suitable.
  - mindslight 3y ago
    I had missed that the storage fees were significantly larger than the traffic fees. I still don't know if Tahoe is appropriate though. It's been a while since I looked at it, but doesn't it work with a fixed set of storage nodes? That wouldn't work for nodes coming/going where people can join organically.

esjeon 3y ago

Yeah, it doesn't make sense for the team to keep hosting every single state of the distro, just to boost building some random dudes' old/peculiar setup. It's almost like Nix team is providing an artifact hosting service. Nix team should expire old versions from their repository, and let people pin what they need on IPFS.

hamandcheese 3y ago

Unfortunately it currently isn't, because nix packages are not content addressed. They are "input addressed", which means that when requesting the contents of a derivation, you have to trust the source to give you the right bytes. There are efforts underway to make things content addressed. This doesn't solve the trust problem -- you still have to trust someone to tell you which inputs map to which content address -- but it does help with the distribution problem. Mirrors no longer have to be trusted. Trust and distribution become decoupled.

sitkack 3y ago
If you went "meta-nix" and had a nix package hierarchy that was just constructed out of operations and inline literal binary blocks, then the results of those purely functional operations (splice, insert, f(n), swap, etc) could be content-addressed and deduplicated. Views of purely functional data transformations of f_tran(n)(addr_start:addr_end), but if you symbolize it you, you make it generic. The problem is that Nix isn't Haskell enough. It also sounds like if they stored the cache on a deduping file system that their storage costs would plummet.
- kaba0 3y ago
  > If you went "meta-nix" and had a nix package hierarchy that was just constructed out of operations and inline literal binary blocks, then the results of those purely functional operations (splice, insert, f(n), swap, etc) could be content-addressed and deduplicated. How do you build with an arbitrary build tool then? You have to call out to gcc/any other compiler in the process.
  - sitkack 3y ago
    It ads another layer of indirection, you can still consume the artifacts of the process using normal tooling.
    - kaba0 3y ago
      I might be missing something — how do you decide a priori building what will be the output hash?