3 ms·
Programs potentially duplicating dependencies at a quadratic rate is an equally abject failure. It's the other extreme of the dependency sharing spectrum. Share
by max-privatevoid 2y ago
Programs potentially duplicating dependencies at a quadratic rate is an equally abject failure. It's the other extreme of the dependency sharing spectrum. Share, but don't overshare. If two programs share a dependency, and that dependency is the exact same (by content or by its inputs, when using a pure build system), they may share the dependency. If not, you install the dependency "twice" (it's not really twice, because it's not the same dependency, so two dependencies, each installed once).
- forrestthewoods 2y ago> at a quadratic rate wat? Worst case is linear. Every program duplicates everything it needs. Which honestly is just totally fine. How is this quadratic?
- max-privatevoid 2y agoQuadratic in the sense that, in the worst case, you have N programs sharing the same set of M dependencies, requiring N*M components worth of space. With sharing, it would only be N+M.
- forrestthewoods 2y agoAh yeah. Feels like you could argue either way! In any case I don’t think library duplication is a meaningful issue in most cases. As evidenced by everyone using Docker which duplicates libraries! Feels like this could be totally solved at the filesystem level with copy-on-write de-duplication. The best of both worlds!
- max-privatevoid 2y agoFS deduplication solves the problem at the storage level, but not at the transfer level. The components would still be downloaded multiple times if the downloader doesn't have a way to know what has already been downloaded. Docker's way of doing things is basically an admittance of defeat by lazy engineers. We can do better than that.