6 ms·

Without getting into Rust’s standard library inclusion philosophy, I would note that UTF-8 and Base64 are quite different cases, and shouldn’t be combined in th

Without getting into Rust’s standard library inclusion philosophy, I would note that UTF-8 and Base64 are quite different cases, and shouldn’t be combined in this way. String encoding is pervasive, and if the standard library didn’t have strings or didn’t specify the encoding, there would be much grief throughout the ecosystem, because you can’t just patch it in at the library level, it’s a language-level feature. So string encoding is a choice that has to be made. Base64, on the other hand, is just a bunch of functions, nothing special about it that means it has to be in the standard library.

tptacek 5y ago
It does need to be in the standard library, because everyone needs it, almost nobody has special needs for it, and if it's not in the stdlib then you have to find, download, review, and build it and its dependencies, and that is problematic.
- chrismorgan 5y ago
  > It does need to be in the standard library When I say that something “has to be in the standard library”, I mean that it can’t be implemented outside the standard library. That’s certainly not the case here. You’re using an outright bad definition of “need” here—subjective opinion rather than objective requirement. > because everyone needs it This is factually wildly wrong. I wrote a fair bit more here but decided it wasn’t helpful. Précis: web stuff tends to load it indirectly (though amusingly most of the time actually not use it, so that Base64 code won’t actually end up in your binary), but it’s not terribly common outside of internet stuff to reach for Base64. I’ll leave just one more remark about Base64: once things are in the standard library, breaking changes can no longer be made; the base64 crate is still experiencing breaking changes (<https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md https://github.com/marshallpierce/rust-base64/blob/master/RE...>, 0.12 and 0.13 were last year and 0.20 is not released), largely for performance reasons. Please don’t just call the thin-std approach “problematic” without acknowledging that the alternative is at least as problematic, just with a different set of caveats.
  - masklinn 5y ago
    > This is factually wildly wrong. I wrote a fair bit more here but decided it wasn’t helpful. Précis: web stuff tends to load it indirectly (though amusingly most of the time actually not use it, so that Base64 code won’t actually end up in your binary), but it’s not terribly common outside of internet stuff to reach for Base64. It also seems to be distinctly losing popularity as time goes on, as the alphabet is not restricted enough to be easy for a human to sight-read/copy, and a wider alphabet will have as good an ASCII channel compatibility.
    - afiori 5y ago
      (begin rant) My issue with base64 is that + and / are terrible character for the alphabet and that the final padding = can be omitted so that base64 strings cannot be safely concatenated (as whitespace is ignored)
      - chrismorgan 5y ago
        See also https://en.wikipedia.org/wiki/Base64#Variants_summary_table https://en.wikipedia.org/wiki/Base64#Variants_summary_table. base64url is common, with - and _ instead of + and /. As for concatenating, that’s not valid even in variants where = padding is mandatory: padding marks the end and is not allowed to occur mid-string. But I don’t think this is a problem, either: transparently concatenating Base64 strings is not generally a useful operation; I can’t think of any valid use case for it off the top of my head, I’m curious why you might think it useful. Base64 is a transfer encoding sort of a thing, and concatenating blobs while in their transfer encoding just… isn’t something that you do.
  - tptacek 5y ago
    I feel like the fact that the Base64 crate is still experiencing breaking changes adds to my point rather than detracting from it.
    - chrismorgan 5y ago
      It’s experiencing breaking changes in the name of improving performance and flexibility. This is precisely the reason why you don’t want it in the standard library, because that would have either prevented these improvements from being invented, or fractured the ecosystem between std::base64 and base64 in just your urllib/urllib2/urllib3/requests sort of way. As it stands, by being external, people can depend on base64 0.11, 0.12, 0.13, &c. and it all works fine, and over time they’ll tend to migrate to the latest version (dealing with the generally minor breaking changes involved) but it doesn’t matter so much.
- burntsushi 5y ago
  Are you referring to base64 here? If so, then you are very very wrong. I've written lots of Rust programs over the years and have needed base64 precisely once IIRC.

tptacek 5y ago

It does need to be in the standard library, because everyone needs it, almost nobody has special needs for it, and if it's not in the stdlib then you have to find, download, review, and build it and its dependencies, and that is problematic.

chrismorgan 5y ago
> It does need to be in the standard library When I say that something “has to be in the standard library”, I mean that it can’t be implemented outside the standard library. That’s certainly not the case here. You’re using an outright bad definition of “need” here—subjective opinion rather than objective requirement. > because everyone needs it This is factually wildly wrong. I wrote a fair bit more here but decided it wasn’t helpful. Précis: web stuff tends to load it indirectly (though amusingly most of the time actually not use it, so that Base64 code won’t actually end up in your binary), but it’s not terribly common outside of internet stuff to reach for Base64. I’ll leave just one more remark about Base64: once things are in the standard library, breaking changes can no longer be made; the base64 crate is still experiencing breaking changes (<https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md https://github.com/marshallpierce/rust-base64/blob/master/RE...>, 0.12 and 0.13 were last year and 0.20 is not released), largely for performance reasons. Please don’t just call the thin-std approach “problematic” without acknowledging that the alternative is at least as problematic, just with a different set of caveats.
- masklinn 5y ago
  > This is factually wildly wrong. I wrote a fair bit more here but decided it wasn’t helpful. Précis: web stuff tends to load it indirectly (though amusingly most of the time actually not use it, so that Base64 code won’t actually end up in your binary), but it’s not terribly common outside of internet stuff to reach for Base64. It also seems to be distinctly losing popularity as time goes on, as the alphabet is not restricted enough to be easy for a human to sight-read/copy, and a wider alphabet will have as good an ASCII channel compatibility.
  - afiori 5y ago
    (begin rant) My issue with base64 is that + and / are terrible character for the alphabet and that the final padding = can be omitted so that base64 strings cannot be safely concatenated (as whitespace is ignored)
    - chrismorgan 5y ago
      See also https://en.wikipedia.org/wiki/Base64#Variants_summary_table https://en.wikipedia.org/wiki/Base64#Variants_summary_table. base64url is common, with - and _ instead of + and /. As for concatenating, that’s not valid even in variants where = padding is mandatory: padding marks the end and is not allowed to occur mid-string. But I don’t think this is a problem, either: transparently concatenating Base64 strings is not generally a useful operation; I can’t think of any valid use case for it off the top of my head, I’m curious why you might think it useful. Base64 is a transfer encoding sort of a thing, and concatenating blobs while in their transfer encoding just… isn’t something that you do.
- tptacek 5y ago
  I feel like the fact that the Base64 crate is still experiencing breaking changes adds to my point rather than detracting from it.
  - chrismorgan 5y ago
    It’s experiencing breaking changes in the name of improving performance and flexibility. This is precisely the reason why you don’t want it in the standard library, because that would have either prevented these improvements from being invented, or fractured the ecosystem between std::base64 and base64 in just your urllib/urllib2/urllib3/requests sort of way. As it stands, by being external, people can depend on base64 0.11, 0.12, 0.13, &c. and it all works fine, and over time they’ll tend to migrate to the latest version (dealing with the generally minor breaking changes involved) but it doesn’t matter so much.
burntsushi 5y ago
Are you referring to base64 here? If so, then you are very very wrong. I've written lots of Rust programs over the years and have needed base64 precisely once IIRC.