7 ms·
While acknowledging one does not "have to" have so many dependencies, the prevalence of this npm-esque type of practice is one of the two things that destroyed
by ergonaught 2y ago
While acknowledging one does not "have to" have so many dependencies, the prevalence of this npm-esque type of practice is one of the two things that destroyed all of my interest in Rust.
- kibwen 2y agoHow many transitive dependencies is the right number for a database?
- jandrewrogers 2y agoHonestly, current best practice puts that number right around zero, which you see for ambitious implementations. A non-obvious issue is that database engines have peculiar requirements for how libraries are designed and implemented which almost no conventional library satisfies. To make matters worse, two different database implementations may have different requirements in this regard, so you can't even share libraries between databases. There are no black boxes in good database engines.
- almostdeadguy 2y agoCompression libraries, OpenSSL, ICU, etc. are all common dependencies for databases. Looking at the dependencies list (https://gist.github.com/tisonkun/06550d2dcd9cf6551887ee6305e92edc https://gist.github.com/tisonkun/06550d2dcd9cf6551887ee6305e...) I see plenty of reasonable things like: * Base64/checksum/compression encoding libraries * Encryption/hash libraries * Platform-specific bindings (likely conditional dependencies) * Bit hacking/casting/zero-copy libraries like bytemuck, zerocopy, zero-vec, etc. * "Small"/stack allocated data structure libraries (smallvec, tinystr, etc.) * Unicode libraries There are certainly things that would add bloat too, but I think it's silly to pretend like everything here is something a database engine would need custom implementations of.
- jandrewrogers 2y agoI think you'd be surprised how many of these things are custom implementations in databases. The main motivation is performance. Databases tend to have detailed and well-specified constraints on each use case for data structures and algorithms that can be used to codegen narrowly optimized implementations. You can do significantly better than generic library codecs or data structures in most cases, those implementations lack the context and metaprogramming hooks to make it feasible. Combine this with the challenge of implementations being async, non-allocating, compatible with explicitly paged memory, etc and it generally becomes worth the effort. You'll find more libraries used at the periphery for integration and compatibility where it matters less but not in the core.
- 0x457 2y agoPretty sure it's not due to performance, but due to age of most database code bases and in some cases licensing. How annoying it is to have dependencies in C and C++ also probably a contributing factor. I'd rather an author pulls in a tinyvec/serde than tries to make a bespoke implementation.
- jandrewrogers 2y agoYou are entitled to your take, but it is a demonstration that you’ve never been anywhere near a modern database engine code base. You can’t defend your opinion. It is what people with no experience would say. That’s fine, I have not objection to hot takes, but don’t conflate your limited experience with reality.
- 0x457 2y agoNever claimed that I was an expert or anything. I interpolated my opinion from my experience with large C code bases. I get it that you work on a geospatial database and understand this field better than I do, but why so rude and dismissive? Work on that.
- kibwen 2y ago> current best practice puts that number right around zero In the case where the answer is "zero", then that means that one does not actually need a package manager at all, in which case the features of the package manager are not relevant to the choice of language. This would imply that the parent commenter has no need to reject Rust.
- orf 2y agoit's completely stupid to measure "number of dependencies" in absolute numbers. Lots of packages have a `-macros` or `-derive` transient dependency, meaning a single dependency can end up coutning as 3 additional dependencies. Rust makes it simple to split packages into workspaces - for example, regex[1] consists of `regex-automata` and `regex-syntax` packages. This composition and separation of concerns is a sign of good design, and not an npm-esque hellhole. 1. https://crates.io/crates/regex/1.11.1/dependencies https://crates.io/crates/regex/1.11.1/dependencies
- rectang 2y agoI suppose you could say that the audit burden scales linearly with the number of module publishers, with a small additional amount on every release point to confirm that the publisher is still who they purport to be and hasn't been compromised. This is assuming that the audit consists of validating dependency authorship, and not the more labor-intensive approach of reviewing dependency code.
- brabel 2y agoJust tried to look at what some macro was generating using cargo-expand. It requires a LOT of dependencies. Took like 5 minutes to compile it all (run `cargo install cargo-expand` if you want to try). I almost aborted because the description of the crate says "Wrapper around rustc -Zunpretty=expanded." so I had expected the simplest possible crate to do that.
- PittleyDunkin 2y ago> Took like 5 minutes to compile it all TBF this has nothing to do with dependency complexity and everything to do with semantic complexity. You could easily do this without using any dependencies at all. unless you're downloading dependencies during the build or something like that, of course.
- PittleyDunkin 2y agoWhat are you comparing this to? Do you have positive examples? This seems to be a general dependancy management issue unrelated to rust—the reason C++ has this is that C++ also lacks any concept of dependencies, so people kind of just make do with modifying what packages are already integrated into the build process. This certainly doesn't imply you should trust boost (or the standard library, or whatever people use this decade, or xz, or whatever).
- sealeck 2y agoRust dependencies tend to be pretty high quality in my experience. Maintained by experts and offer new improvements over state-of-the-art. But if you compare to C/C++ at least with Rust you _can_ but aren't required to use dependencies. In C/C++ if you want to, it's a _massive_ pain.
- rectang 2y agoI care less about the quality of the dependencies than about the burden of protecting against supply chain attacks when there are a lot of dependencies.
- larusso 2y agoNothing stops you from vendoring them into your repo and hand update each. But how would you do this in c++? Write everything from scratch? I mean rust doesn’t stopp you there [edit] typos
- whodev 2y agoThank you. As someone who works in cybersecurity and works closely with our developers, a lot of them tend to inherently trust third-party code with no auditing of the supply chain. I am always fighting that while yes, we don't need to reinvent the wheel and libraries/packages are important, our organzation and developers need to be aware of what we are bringing into our network and our codebase.
- kibwen 2y agoAs someone who also works in cybersecurity, we use Rust extensively and are sunsetting all of our C code. We use third-party dependencies judiciously and never deploy anything without auditing it. It's great that Rust facilitates this convenient ecosystem, and it's to Rust's benefit, not Rust's detriment, that the ecosystem exists.
- whodev 2y ago> We use third-party dependencies judiciously and never deploy anything without auditing it. This is how I think it should be of course. Like I said, I'm not against the use of third-party code or dependencies, I'm against using them without performing any audit of that code.
- klysm 2y agoThis take is utter nonsense to me - just don't use them...
- deleted 2y ago[deleted]
- FullGarden_S 2y agoimagine if one dependency is GPL lol With over 600 dependencies, the probability goes up and up.