8 ms·

> when building a large-ish package, it only makes sense to use existing packages rather than reinventing the wheel. Yes, for instance one of `reqwest`'s depen

> when building a large-ish package, it only makes sense to use existing packages rather than reinventing the wheel. Yes, for instance one of `reqwest`'s dependencies is `base64`. We can argue both that re-inventing base64 in every big library is pointless, and also that a language doesn't need base64 in its stdlib because C and C++ don't have it. There's no right way to cut this cake - Many a language has included something in the standard library, and then struggled to fix a broken interface once everything depended on it. (Like JS picking UTF-16 for strings) Many a language has also failed to include something in the stdlib and caused every application to suffer and re-invent it. (Like C++ not really saying anything about Unicode strings, which means Qt and everyone else has their own little take on it.) And the suffering is only bigger if you make dependencies artificially hard to add. Rust's approach is more flexible, at the cost of building the universe from scratch every time you make a new workspace. Personally I don't mind it. At least it's not like the problems I've heard from Haskell land. Moving from futures to async in Rust was fairly smooth.

tptacek 5y ago
Rust should include Base64 in the standard library, and languages should continue to standardize on UTF-8. Doesn't seem hard to work out.
- burntsushi 5y ago
  You're nitpicking one example. Putting base64 into std makes the higher level comment go from complaining about 100 dependencies to complaining about 99 dependencies. So maybe it's harder to work out than you think.
  - tptacek 5y ago
    I don't have an opinion about the broader dependency problem in Rust. The comment to which I responded was hosting an internal debate about Base64 and UTF-8. I think that debate is easy to resolve: use UTF-8, put Base64 in the stdlib. Do you disagree?
    - burntsushi 5y ago
      The comment you responded to was using base64 as an example. That's why your comment is a nitpick in this conversation. And yeah, I doubt I would support including base64 in std unless there were a compelling reason to do so. I can't think of one. And yes, use UTF-8. Not contesting that.
      - tptacek 5y ago
        I guess I'd just say that the aggregation of these "keep base64 out of std" decisions are why every Rust thingy I put together ends up with a zillion dependencies. There might be an upside to that (I have not detected it), but there's an obvious downside. Rust gets a bunch of stuff right that other languages missed. I don't think this one of them. I think Go's strategy of having a pretty-ok version of most of the table-stakes things developers do is probably the right one, just like I think sum types and option/result are better than error types and `nil`. I share your irritation at people who drop into the middle of discussions like these to nitpick random language war arguments, but this is a thread that is actually about Rust dependency hell, and the comment I responded to, I think, sort of diagnoses why Rust has that "problem" (if you think it's a problem --- I do, but whatever). For whatever this is worth, by the way, languages that have Base64 in std appear to include Ruby, Javascript, Python, Clojure, Java, Scala, Go, Elixir, Nim, and Swift. OCaml, C++ and Haskell do not.
        pixel_fcker 5y ago
        What’s the “obvious” downside?
        tptacek 5y ago
        Having to find, download, review, and build a bunch of extra dependencies (and surveil all the dependencies you do explicitly want to make sure they're not pulling in fucky code to do table-stakes things that the stdlib could just stdize.) Again: these are downsides, but not dispositions. On balance there could be good reasons to have a terse, svelte, balletic std. I'm just saying, there's clearly a downside to the decision.
- kibwen 5y ago
  The process for proposing new modules for Rust's stdlib is pretty streamlined. Small API additions can be proposed directly via PR to rust-lang/rust, whereas large API additions can be brought up for discussion in rust-lang/rfcs. If anybody thinks that base64 deserves to be in Rust's stdlib, I encourage them to propose it (to my knowledge nobody ever has).
awsthro00945 5y ago
>and also that a language doesn't need base64 in its stdlib because C and C++ don't have it I don't follow this logic. If we just wanted to do everything the way C/C++ does it, we'd be using C/C++, and Rust wouldn't exist.
- pjmlp 5y ago
  True, although my only complain with C++ is the subculture that to this day pushes for C types and unsafe by default. For anything else I don't have any major complaint.
  - secondcoming 5y ago
    who is pushing for 'unsafe by default'?
    - pjmlp 5y ago
      Anyone that uses char * instead of std::array for example.
      - yakubin 5y ago
        std::array will instantiate function templates for all used array sizes, lengthen compilation time, and be hard to use across separately-compiled code without putting code into headers. It has its uses, but more often I'd prefer T[N] with bounds-checking, and, in another world, a builtin variably-sized, non-resizable array type; basically a sort of absl::FixedArray[1], but with the semantics of operator[] and at() reversed, so that bounds-checking is the default. Sadly, nothing like that is included in the standard, so won't be a lingua-franca. [1]: <https://github.com/abseil/abseil-cpp/blob/master/absl/container/fixed_array.h https://github.com/abseil/abseil-cpp/blob/master/absl/contai...>
        pjmlp 5y ago
        On Visual Studio it is relatively easy to enable bounds checking for operator[](). https://docs.microsoft.com/en-us/visualstudio/debugger/how-to-use-native-run-time-checks?view=vs-2019 https://docs.microsoft.com/en-us/visualstudio/debugger/how-t... That is the thing, the standard doesn't require bounds checking for operator()[], while it does require for at(), but it doesn't forbid it either, it is up to the implementations.
hardwaregeek 5y ago
Yeah in the debate about dependencies, people often frame it as more dependencies == bad and less dependencies == good. Except more/less dependencies is actually more or less explicit dependencies. Fact of the matter is that you need a crazy amount of dependencies to write most modern software. You need ways to handle Unicode, to connect to databases, to write async code, to handle dates, etc. The question is whether that code is going to come from a monolithic Swiss Army Knife of a library (aka a standard library), a bunch of packages, or some code that everybody copies into their codebase. In which case the packages option doesn't seem so bad. At least they're explicitly versioned, tracked for vulnerabilities and can be updated at a granular level.
- pjmlp 5y ago
  Because when working in large corporations one ends up having to write license checker tools and having humans go through those packages, alongside legal, to validate what is allowed to appear on the final product, and to vendor them on the internal package repository.

tptacek 5y ago

Rust should include Base64 in the standard library, and languages should continue to standardize on UTF-8. Doesn't seem hard to work out.

burntsushi 5y ago
You're nitpicking one example. Putting base64 into std makes the higher level comment go from complaining about 100 dependencies to complaining about 99 dependencies. So maybe it's harder to work out than you think.
- tptacek 5y ago
  I don't have an opinion about the broader dependency problem in Rust. The comment to which I responded was hosting an internal debate about Base64 and UTF-8. I think that debate is easy to resolve: use UTF-8, put Base64 in the stdlib. Do you disagree?
  - burntsushi 5y ago
    The comment you responded to was using base64 as an example. That's why your comment is a nitpick in this conversation. And yeah, I doubt I would support including base64 in std unless there were a compelling reason to do so. I can't think of one. And yes, use UTF-8. Not contesting that.
    - tptacek 5y ago
      I guess I'd just say that the aggregation of these "keep base64 out of std" decisions are why every Rust thingy I put together ends up with a zillion dependencies. There might be an upside to that (I have not detected it), but there's an obvious downside. Rust gets a bunch of stuff right that other languages missed. I don't think this one of them. I think Go's strategy of having a pretty-ok version of most of the table-stakes things developers do is probably the right one, just like I think sum types and option/result are better than error types and `nil`. I share your irritation at people who drop into the middle of discussions like these to nitpick random language war arguments, but this is a thread that is actually about Rust dependency hell, and the comment I responded to, I think, sort of diagnoses why Rust has that "problem" (if you think it's a problem --- I do, but whatever). For whatever this is worth, by the way, languages that have Base64 in std appear to include Ruby, Javascript, Python, Clojure, Java, Scala, Go, Elixir, Nim, and Swift. OCaml, C++ and Haskell do not.
      - pixel_fcker 5y ago
        What’s the “obvious” downside?
        tptacek 5y ago
        Having to find, download, review, and build a bunch of extra dependencies (and surveil all the dependencies you do explicitly want to make sure they're not pulling in fucky code to do table-stakes things that the stdlib could just stdize.) Again: these are downsides, but not dispositions. On balance there could be good reasons to have a terse, svelte, balletic std. I'm just saying, there's clearly a downside to the decision.
kibwen 5y ago
The process for proposing new modules for Rust's stdlib is pretty streamlined. Small API additions can be proposed directly via PR to rust-lang/rust, whereas large API additions can be brought up for discussion in rust-lang/rfcs. If anybody thinks that base64 deserves to be in Rust's stdlib, I encourage them to propose it (to my knowledge nobody ever has).

awsthro00945 5y ago

>and also that a language doesn't need base64 in its stdlib because C and C++ don't have it I don't follow this logic. If we just wanted to do everything the way C/C++ does it, we'd be using C/C++, and Rust wouldn't exist.

pjmlp 5y ago
True, although my only complain with C++ is the subculture that to this day pushes for C types and unsafe by default. For anything else I don't have any major complaint.
- secondcoming 5y ago
  who is pushing for 'unsafe by default'?
  - pjmlp 5y ago
    Anyone that uses char * instead of std::array for example.
    - yakubin 5y ago
      std::array will instantiate function templates for all used array sizes, lengthen compilation time, and be hard to use across separately-compiled code without putting code into headers. It has its uses, but more often I'd prefer T[N] with bounds-checking, and, in another world, a builtin variably-sized, non-resizable array type; basically a sort of absl::FixedArray[1], but with the semantics of operator[] and at() reversed, so that bounds-checking is the default. Sadly, nothing like that is included in the standard, so won't be a lingua-franca. [1]: <https://github.com/abseil/abseil-cpp/blob/master/absl/container/fixed_array.h https://github.com/abseil/abseil-cpp/blob/master/absl/contai...>
      - pjmlp 5y ago
        On Visual Studio it is relatively easy to enable bounds checking for operator[](). https://docs.microsoft.com/en-us/visualstudio/debugger/how-to-use-native-run-time-checks?view=vs-2019 https://docs.microsoft.com/en-us/visualstudio/debugger/how-t... That is the thing, the standard doesn't require bounds checking for operator()[], while it does require for at(), but it doesn't forbid it either, it is up to the implementations.

hardwaregeek 5y ago

Yeah in the debate about dependencies, people often frame it as more dependencies == bad and less dependencies == good. Except more/less dependencies is actually more or less explicit dependencies. Fact of the matter is that you need a crazy amount of dependencies to write most modern software. You need ways to handle Unicode, to connect to databases, to write async code, to handle dates, etc. The question is whether that code is going to come from a monolithic Swiss Army Knife of a library (aka a standard library), a bunch of packages, or some code that everybody copies into their codebase. In which case the packages option doesn't seem so bad. At least they're explicitly versioned, tracked for vulnerabilities and can be updated at a granular level.

pjmlp 5y ago
Because when working in large corporations one ends up having to write license checker tools and having humans go through those packages, alongside legal, to validate what is allowed to appear on the final product, and to vendor them on the internal package repository.