4 ms·

> In Rust every non-trivial library pulls in 10s or even 100s of dependencies. You're exaggerating here. The most recent project I've been working on pulls in

> In Rust every non-trivial library pulls in 10s or even 100s of dependencies. You're exaggerating here. The most recent project I've been working on pulls in 6 dependencies. The anyhow crate has no dependencies, regex 3 (recursively!), clap and csv each 8. Only handlebars and palette pull in 10s of dependencies, and I can trim a fair few dependencies of palette by opting out of named color support (dropping the phf crate for perfect hash functions).

avar 5y ago
It's typical for very large C projects to have perhaps 2-5 dependencies that aren't libc, often something very basic such as zlib, curl or openssl. A rust CSV parser has 8 dependencies?
- throwaway894345 5y ago
  A Go program wouldn't even need curl, zlib, or openssl as it has equivalent implementations in its stdlib.
- steveklabnik 5y ago
  The CSV crate has five of them: https://crates.io/crates/csv/1.1.6/dependencies https://crates.io/crates/csv/1.1.6/dependencies
- jcranmer 5y ago
  Three of those dependencies are not entirely unreasonable: one is the actual "core" CSV implementation (with one recursive dependency), one is the dependency for the standard "this interface allows you to serialize/deserialize arbitrary data to arbitrary formats", and one crate that adds traits to make &[u8]/Vec<u8> work much more like &str/String (which has 4 recursive dependencies, although two of those are already mentioned). The last two dependencies are crates that provide accelerated routines for serializing integers and floating points, which are IMHO somewhat excessive, but if it's easy to pull in such dependencies, why not? It would be helpful in places where CSV serialization might actually be a bottleneck. There are two key differences between large projects in ecosystems that have working package managers and those in C/C++. For starters, the basic working 'package' tends to be smaller. Consider something like LLVM (just LLVM itself, none of its subprojects like Clang). This you'd count as "one project" without any dependencies (at least, no required dependencies that aren't vendored, liked zlib and googletest). But if you were to make it in something like Rust, it would likely consist of a dozen or more packages: the core IR, analysis passes, command-line tools, transformations, codegen (with each target likely being its own package!). With such fragmentation, you'd have dozens of dependencies that are essentially nothing more than the 'project' itself. But more importantly is that, because dependencies are hard to add, code duplication is the norm; it's only when what you want to do is too impossible to duplicate that you resort to a dependency. For any large C++ project, I can virtually guarantee that there will exist some form of 'standard library++' dependency in the code that will contain at least an intrusive linked list, better hashtable implementation, std::vector-lookalike that avoids heap allocation for small lists, replacement for iostream, helpers for process creation, custom string formatting routines, and something to make dealing with segfaults sane. In a package ecosystem, each of those kinds of things would be a separate dependency. Is it really necessary for everybody who wants to write something dealing with graphs to have write their own graph-based data structure, as opposed to just using the ecosystem-standard graph library (e.g., networkx in Python, or petgraph in Rust)? Or to have everybody that wants to use a hashtable to have to write their own implementation of a notoriously tricky data structure?
  - avar 5y ago
    I wasn't attempting to make a value-judgement about whether something /should/ have these dependencies, but only to point out that if you think this is comparable to C-based projects you seemed to be pretty out of touch with common practices in that domain. I take your point that NIH is a large thing driving this sort of divergence in many languages, with some (such as Common Lisp, and C) being more notorious than others. But I'd say it's just as common in other languages for programmers to pile on needless dependencies. E.g. how many Rust/Node.js/Java etc. projects are using some "standard library++" to e.g. get better memory behavior for strings, but for whom string allocation overhead is never ever going to be an issue? It would be an interesting metric to check what % of the code in a given library is commonly used by its consumers across languages. Even if you only used 1% you're implicitly assuming some of the maintenance overhead for 100% of it. E.g. if it won't build on an older toolchain that's now a hassle for someone packaging your program, even if it's in the 99% you don't use.
dmz73 5y ago
druid v0.7.0 (\druid\druid) druid-derive v0.4.0 (proc-macro) (\druid\druid-derive) proc-macro2 v1.0.32 unicode-xid v0.2.2 quote v1.0.10 proc-macro2 v1.0.32 () syn v1.0.81 proc-macro2 v1.0.32 () quote v1.0.10 () unicode-xid v0.2.2 [dev-dependencies] druid v0.7.0 (\druid\druid) () float-cmp v0.8.0 trybuild v1.0.52 glob v0.3.0 lazy_static v1.4.0 serde v1.0.130 serde_derive v1.0.130 (proc-macro) proc-macro2 v1.0.32 () quote v1.0.10 () syn v1.0.81 () serde_json v1.0.69 itoa v0.4.8 ryu v1.0.5 serde v1.0.130 () termcolor v1.1.2 winapi-util v0.1.5 winapi v0.3.9 winapi-x86_64-pc-windows-gnu v0.4.0 toml v0.5.8 serde v1.0.130 () druid-shell v0.7.0 (\druid\druid-shell) anyhow v1.0.45 cfg-if v1.0.0 instant v0.1.12 cfg-if v1.0.0 keyboard-types v0.5.0 bitflags v1.3.2 kurbo v0.8.2 arrayvec v0.7.2 lazy_static v1.4.0 piet-common v0.5.0-pre1 cfg-if v1.0.0 piet v0.5.0-pre1 kurbo v0.8.2 () unic-bidi v0.9.0 matches v0.1.9 unic-ucd-bidi v0.9.0 unic-char-property v0.9.0 unic-char-range v0.9.0 unic-char-range v0.9.0 unic-ucd-version v0.9.0 unic-common v0.9.0 piet-direct2d v0.5.0-pre1 associative-cache v1.0.1 dwrote v0.11.0 lazy_static v1.4.0 libc v0.2.107 winapi v0.3.9 () wio v0.2.2 winapi v0.3.9 () piet v0.5.0-pre1 () utf16_lit v2.0.2 winapi v0.3.9 () wio v0.2.2 () png v0.17.2 bitflags v1.3.2 crc32fast v1.2.1 cfg-if v1.0.0 deflate v0.9.1 adler32 v1.2.0 encoding v0.2.33 encoding-index-japanese v1.20141219.5 encoding_index_tests v0.1.4 encoding-index-korean v1.20141219.5 encoding_index_tests v0.1.4 encoding-index-simpchinese v1.20141219.5 encoding_index_tests v0.1.4 encoding-index-singlebyte v1.20141219.5 encoding_index_tests v0.1.4 encoding-index-tradchinese v1.20141219.5 encoding_index_tests v0.1.4 miniz_oxide v0.4.4 adler v1.0.2 [build-dependencies] autocfg v1.0.1 scopeguard v1.1.0 time v0.3.5 tracing v0.1.29 cfg-if v1.0.0 pin-project-lite v0.2.7 tracing-attributes v0.1.18 (proc-macro) proc-macro2 v1.0.32 () quote v1.0.10 () syn v1.0.81 () tracing-core v0.1.21 lazy_static v1.4.0 winapi v0.3.9 () wio v0.2.2 () [dev-dependencies] piet-common v0.5.0-pre1 () static_assertions v1.1.0 test-env-log v0.2.7 (proc-macro) proc-macro2 v1.0.32 () quote v1.0.10 () syn v1.0.81 () tracing-subscriber v0.2.25 ansi_term v0.12.1 winapi v0.3.9 () chrono v0.4.19 libc v0.2.107 num-integer v0.1.44 num-traits v0.2.14 [build-dependencies] autocfg v1.0.1 [build-dependencies] autocfg v1.0.1 num-traits v0.2.14 () winapi v0.3.9 () lazy_static v1.4.0 matchers v0.0.1 regex-automata v0.1.10 regex-syntax v0.6.25 regex v1.5.4 regex-syntax v0.6.25 serde v1.0.130 () serde_json v1.0.69 () sharded-slab v0.1.4 lazy_static v1.4.0 smallvec v1.7.0 thread_local v1.1.3 once_cell v1.8.0 tracing v0.1.29 () tracing-core v0.1.21 () tracing-log v0.1.2 lazy_static v1.4.0 log v0.4.14 cfg-if v1.0.0 tracing-core v0.1.21 () tracing-serde v0.1.2 serde v1.0.130 () tracing-core v0.1.21 () unicode-segmentation v1.8.0 fluent-bundle v0.15.2 fluent-langneg v0.13.0 unic-langid v0.9.0 unic-langid-impl v0.9.0 tinystr v0.3.4 fluent-syntax v0.11.0 thiserror v1.0.30 thiserror-impl v1.0.30 (proc-macro) proc-macro2 v1.0.32 () quote v1.0.10 () syn v1.0.81 () intl-memoizer v0.5.1 type-map v0.4.0 rustc-hash v1.1.0 unic-langid v0.9.0 () intl_pluralrules v7.0.1 tinystr v0.3.4 unic-langid v0.9.0 () rustc-hash v1.1.0 self_cell v0.10.1 smallvec v1.7.0 unic-langid v0.9.0 () fluent-langneg v0.13.0 () fluent-syntax v0.11.0 () fnv v1.0.7 instant v0.1.12 () tracing v0.1.29 () tracing-subscriber v0.2.25 () unic-langid v0.9.0 () unicode-segmentation v1.8.0 xi-unicode v0.3.0 [dev-dependencies] float-cmp v0.8.0 open v1.7.1 winapi v0.3.9 () piet-common v0.5.0-pre1 () pulldown-cmark v0.8.0 bitflags v1.3.2 memchr v2.4.1 unicase v2.6.0 [build-dependencies] version_check v0.9.3 tempfile v3.1.0 cfg-if v0.1.10 rand v0.7.3 getrandom v0.1.16 cfg-if v1.0.0 rand_chacha v0.2.2 ppv-lite86 v0.2.15 rand_core v0.5.1 getrandom v0.1.16 () rand_core v0.5.1 () remove_dir_all v0.5.3 winapi v0.3.9 () winapi v0.3.9 () test-env-log v0.2.7 (proc-macro) () tracing-subscriber v0.2.25 () druid-derive v0.4.0 (proc-macro) (\druid\druid-derive) () druid-shell v0.7.0 (\druid\druid-shell) ()

avar 5y ago

It's typical for very large C projects to have perhaps 2-5 dependencies that aren't libc, often something very basic such as zlib, curl or openssl. A rust CSV parser has 8 dependencies?

throwaway894345 5y ago
A Go program wouldn't even need curl, zlib, or openssl as it has equivalent implementations in its stdlib.
steveklabnik 5y ago
The CSV crate has five of them: https://crates.io/crates/csv/1.1.6/dependencies https://crates.io/crates/csv/1.1.6/dependencies
jcranmer 5y ago
Three of those dependencies are not entirely unreasonable: one is the actual "core" CSV implementation (with one recursive dependency), one is the dependency for the standard "this interface allows you to serialize/deserialize arbitrary data to arbitrary formats", and one crate that adds traits to make &[u8]/Vec<u8> work much more like &str/String (which has 4 recursive dependencies, although two of those are already mentioned). The last two dependencies are crates that provide accelerated routines for serializing integers and floating points, which are IMHO somewhat excessive, but if it's easy to pull in such dependencies, why not? It would be helpful in places where CSV serialization might actually be a bottleneck. There are two key differences between large projects in ecosystems that have working package managers and those in C/C++. For starters, the basic working 'package' tends to be smaller. Consider something like LLVM (just LLVM itself, none of its subprojects like Clang). This you'd count as "one project" without any dependencies (at least, no required dependencies that aren't vendored, liked zlib and googletest). But if you were to make it in something like Rust, it would likely consist of a dozen or more packages: the core IR, analysis passes, command-line tools, transformations, codegen (with each target likely being its own package!). With such fragmentation, you'd have dozens of dependencies that are essentially nothing more than the 'project' itself. But more importantly is that, because dependencies are hard to add, code duplication is the norm; it's only when what you want to do is too impossible to duplicate that you resort to a dependency. For any large C++ project, I can virtually guarantee that there will exist some form of 'standard library++' dependency in the code that will contain at least an intrusive linked list, better hashtable implementation, std::vector-lookalike that avoids heap allocation for small lists, replacement for iostream, helpers for process creation, custom string formatting routines, and something to make dealing with segfaults sane. In a package ecosystem, each of those kinds of things would be a separate dependency. Is it really necessary for everybody who wants to write something dealing with graphs to have write their own graph-based data structure, as opposed to just using the ecosystem-standard graph library (e.g., networkx in Python, or petgraph in Rust)? Or to have everybody that wants to use a hashtable to have to write their own implementation of a notoriously tricky data structure?
- avar 5y ago
  I wasn't attempting to make a value-judgement about whether something /should/ have these dependencies, but only to point out that if you think this is comparable to C-based projects you seemed to be pretty out of touch with common practices in that domain. I take your point that NIH is a large thing driving this sort of divergence in many languages, with some (such as Common Lisp, and C) being more notorious than others. But I'd say it's just as common in other languages for programmers to pile on needless dependencies. E.g. how many Rust/Node.js/Java etc. projects are using some "standard library++" to e.g. get better memory behavior for strings, but for whom string allocation overhead is never ever going to be an issue? It would be an interesting metric to check what % of the code in a given library is commonly used by its consumers across languages. Even if you only used 1% you're implicitly assuming some of the maintenance overhead for 100% of it. E.g. if it won't build on an older toolchain that's now a hassle for someone packaging your program, even if it's in the 99% you don't use.

dmz73 5y ago

druid v0.7.0 (\druid\druid) druid-derive v0.4.0 (proc-macro) (\druid\druid-derive) proc-macro2 v1.0.32 unicode-xid v0.2.2 quote v1.0.10 proc-macro2 v1.0.32 () syn v1.0.81 proc-macro2 v1.0.32 () quote v1.0.10 () unicode-xid v0.2.2 [dev-dependencies] druid v0.7.0 (\druid\druid) () float-cmp v0.8.0 trybuild v1.0.52 glob v0.3.0 lazy_static v1.4.0 serde v1.0.130 serde_derive v1.0.130 (proc-macro) proc-macro2 v1.0.32 () quote v1.0.10 () syn v1.0.81 () serde_json v1.0.69 itoa v0.4.8 ryu v1.0.5 serde v1.0.130 () termcolor v1.1.2 winapi-util v0.1.5 winapi v0.3.9 winapi-x86_64-pc-windows-gnu v0.4.0 toml v0.5.8 serde v1.0.130 () druid-shell v0.7.0 (\druid\druid-shell) anyhow v1.0.45 cfg-if v1.0.0 instant v0.1.12 cfg-if v1.0.0 keyboard-types v0.5.0 bitflags v1.3.2 kurbo v0.8.2 arrayvec v0.7.2 lazy_static v1.4.0 piet-common v0.5.0-pre1 cfg-if v1.0.0 piet v0.5.0-pre1 kurbo v0.8.2 () unic-bidi v0.9.0 matches v0.1.9 unic-ucd-bidi v0.9.0 unic-char-property v0.9.0 unic-char-range v0.9.0 unic-char-range v0.9.0 unic-ucd-version v0.9.0 unic-common v0.9.0 piet-direct2d v0.5.0-pre1 associative-cache v1.0.1 dwrote v0.11.0 lazy_static v1.4.0 libc v0.2.107 winapi v0.3.9 () wio v0.2.2 winapi v0.3.9 () piet v0.5.0-pre1 () utf16_lit v2.0.2 winapi v0.3.9 () wio v0.2.2 () png v0.17.2 bitflags v1.3.2 crc32fast v1.2.1 cfg-if v1.0.0 deflate v0.9.1 adler32 v1.2.0 encoding v0.2.33 encoding-index-japanese v1.20141219.5 encoding_index_tests v0.1.4 encoding-index-korean v1.20141219.5 encoding_index_tests v0.1.4 encoding-index-simpchinese v1.20141219.5 encoding_index_tests v0.1.4 encoding-index-singlebyte v1.20141219.5 encoding_index_tests v0.1.4 encoding-index-tradchinese v1.20141219.5 encoding_index_tests v0.1.4 miniz_oxide v0.4.4 adler v1.0.2 [build-dependencies] autocfg v1.0.1 scopeguard v1.1.0 time v0.3.5 tracing v0.1.29 cfg-if v1.0.0 pin-project-lite v0.2.7 tracing-attributes v0.1.18 (proc-macro) proc-macro2 v1.0.32 () quote v1.0.10 () syn v1.0.81 () tracing-core v0.1.21 lazy_static v1.4.0 winapi v0.3.9 () wio v0.2.2 () [dev-dependencies] piet-common v0.5.0-pre1 () static_assertions v1.1.0 test-env-log v0.2.7 (proc-macro) proc-macro2 v1.0.32 () quote v1.0.10 () syn v1.0.81 () tracing-subscriber v0.2.25 ansi_term v0.12.1 winapi v0.3.9 () chrono v0.4.19 libc v0.2.107 num-integer v0.1.44 num-traits v0.2.14 [build-dependencies] autocfg v1.0.1 [build-dependencies] autocfg v1.0.1 num-traits v0.2.14 () winapi v0.3.9 () lazy_static v1.4.0 matchers v0.0.1 regex-automata v0.1.10 regex-syntax v0.6.25 regex v1.5.4 regex-syntax v0.6.25 serde v1.0.130 () serde_json v1.0.69 () sharded-slab v0.1.4 lazy_static v1.4.0 smallvec v1.7.0 thread_local v1.1.3 once_cell v1.8.0 tracing v0.1.29 () tracing-core v0.1.21 () tracing-log v0.1.2 lazy_static v1.4.0 log v0.4.14 cfg-if v1.0.0 tracing-core v0.1.21 () tracing-serde v0.1.2 serde v1.0.130 () tracing-core v0.1.21 () unicode-segmentation v1.8.0 fluent-bundle v0.15.2 fluent-langneg v0.13.0 unic-langid v0.9.0 unic-langid-impl v0.9.0 tinystr v0.3.4 fluent-syntax v0.11.0 thiserror v1.0.30 thiserror-impl v1.0.30 (proc-macro) proc-macro2 v1.0.32 () quote v1.0.10 () syn v1.0.81 () intl-memoizer v0.5.1 type-map v0.4.0 rustc-hash v1.1.0 unic-langid v0.9.0 () intl_pluralrules v7.0.1 tinystr v0.3.4 unic-langid v0.9.0 () rustc-hash v1.1.0 self_cell v0.10.1 smallvec v1.7.0 unic-langid v0.9.0 () fluent-langneg v0.13.0 () fluent-syntax v0.11.0 () fnv v1.0.7 instant v0.1.12 () tracing v0.1.29 () tracing-subscriber v0.2.25 () unic-langid v0.9.0 () unicode-segmentation v1.8.0 xi-unicode v0.3.0 [dev-dependencies] float-cmp v0.8.0 open v1.7.1 winapi v0.3.9 () piet-common v0.5.0-pre1 () pulldown-cmark v0.8.0 bitflags v1.3.2 memchr v2.4.1 unicase v2.6.0 [build-dependencies] version_check v0.9.3 tempfile v3.1.0 cfg-if v0.1.10 rand v0.7.3 getrandom v0.1.16 cfg-if v1.0.0 rand_chacha v0.2.2 ppv-lite86 v0.2.15 rand_core v0.5.1 getrandom v0.1.16 () rand_core v0.5.1 () remove_dir_all v0.5.3 winapi v0.3.9 () winapi v0.3.9 () test-env-log v0.2.7 (proc-macro) () tracing-subscriber v0.2.25 () druid-derive v0.4.0 (proc-macro) (\druid\druid-derive) () druid-shell v0.7.0 (\druid\druid-shell) ()