3 ms·
When you think of Guix as a source distribution first and a binary distribution second this makes sense. The build farm is just another user that builds everyt
by rekado 6y ago
When you think of Guix as a source distribution first and a binary distribution second this makes sense. The build farm is just another user that builds everything from source. If it didn't catch up with the builds because someone just changed a package definition then you might be requesting a binary that hasn't yet been built.
Slow download appears to depend a lot on location to an extent that we suspect peering problems. The main build farm's head node is connected to the internet from a well-connected research campus with dual 10G. Multiple tests have concluded that the configuration from server-side software, to network hardware of the server, down to networking in the data centre is sane.
Maybe we'll have to bite the bullet and (again) use a CDN to better serve users that inexplicably experience slow connections.
- maxeonyx 6y agoWhat about integrating (and encouraging users to opt into) peer-to-peer distribution? If I understand correctly, this would work for Guix and Nix. I'm not using Guix, but if I was I would be happy to do this. I have no data cap on my home internet, and my PC is usually on.
- rekado 6y agoThere was early work on using IPFS as a substitute distribution mechanism, but the API changed and nobody has picked up the existing work yet. But I agree that this would be a great feature and I hope someone will feel motivated to pick up and assemble the pieces.
- thom 6y agoWhat are the security implications of this?
- mbakke 6y agoThe implementation requires that you have at least one authorized substitute server advertising the same hash. In simplified terms, if ci.guix.gnu.org advertises a substitute for /gnu/store/abc123-foo, with the checksum "xyz789" (and the cryptographic signature of that advertisement checks out), your daemon can safely download that file over P2P.
- thom 6y agoAh, I think I'm misunderstanding the intent here. Clearly P2P distribution of checksummed binaries can be safe, I was just wondering if there were a solution to the build farm being behind. It seems like you can't really trust the first build of any artifact unless it comes from a central source.
- mbakke 6y agoThere have been discussions of an "N of P" distribution, i.e. if 80% of available peers (or substitute servers) advertise the same build result, then treat it as safe. I expect that both will be implemented, and the choice left up to the user.
- thom 6y agoUnless I'm massively underestimating the number of Guix users, that seems quite easy to exploit.
- snicker7 6y agoWith regards to downloading speeds, the issue seems to be downloading many packaging as opposed to large packages. Each package incurs some noticeable overhead, which is painful when upgrades consist of dozens or hundreds of upgraded packages.