4 ms·

I assume "rust fanboy" is me. Please stop it. This is the fourth time you're making this unsubstantiated claim. Yes, I backed up my argument with "the crates

I assume "rust fanboy" is me. Please stop it. This is the fourth time you're making this unsubstantiated claim. Yes, I backed up my argument with "the crates I am using" (that is, all the crates in the entire dependency tree of any crate I've used in the last year), which is at least somewhat representative of the crates that folks use, as opposed to your wild claims that have zero backup (or are backed up with one or two mentions of crates, where subsequently the use of unsafe is justified by me or others). Here's an example of one of these discussions: https://news.ycombinator.com/item?id=14276129 https://news.ycombinator.com/item?id=14276129 . Here's another: https://news.ycombinator.com/item?id=13280347 https://news.ycombinator.com/item?id=13280347 . In the second one I actually admit that I was surprised at the number of crates doing unchecked indexing, that's definitely not denial. I still maintained there that it was quite manageable and not some crazy pervasive problem as you make it out to be, but you've literally never provided evidence otherwise. That's not denial. As I've mentioned before, I'm genuinely interested in finding these cases so that they can be fixed (or replaced with well-audited general purpose unsafe abstraction libraries -- e.g. a cheap ASCII-within-utf8 handling library that lets you do things like munch ascii bytes while parsing without extra overhead (or whatever the use cases are -- I'm not yet clear on the set of use cases out there which is why I'm interested to hear more). I'm not "denying everything", and other people have even pointed this out (https://news.ycombinator.com/item?id=14277634 https://news.ycombinator.com/item?id=14277634) in the discussions we've had. ----- FWIW, IIRC UTF8 stuff has been a perf bottleneck in the url crate. It's not premature (file a bug, though). That said, we might be able to write that code differently to avoid it or have a more general way of doing it with clearer invariants (I'll think about it!). The ordermap crate does implement a hash map without unsafe code (leveraging vec). The stdlib hashmap uses unsafe because its design (which has different tradeoffs wrt ordermap) needs to be able to have buckets be empty (where whether or not the bucket is empty is based on a nontrivial set of invariants from the robin hood based design) without using something like an Option which has extra memory and CPU overhead. Optimizers are not smart enough to figure out the robin hood invariants, so this is not premature either. ---- I no longer believe you're making these claims in good faith, so I'm not really interested in discussing this further with you. I just want to counter the wild claims here for the benefit of other folks reading this. If other folks wish to constructively point out cases of unsafe being used this way, I'd love to hear about it, both to get a better idea of the ecosystem, and to help improve it. I do want the community to move towards less unsafe code.

Animats 9y ago
Note lack of links to code above. Please provide a list of the basic Rust crates which contain no unsafe code. Can you write useful programs using only those crates, not counting ones which need foreign function calls to talk to the OS?
- openasocket 9y ago
  > Note lack of links to code above. He provided links to several other threads where he explicitly has made a list. In this one https://news.ycombinator.com/item?id=13280347 https://news.ycombinator.com/item?id=13280347 he says he looked at ~600 crates he actually uses, and found 70 used unsafe code. Those 70 are given here: https://gist.github.com/Manishearth/6a9367a7d8772e095629e82184f05ad4 https://gist.github.com/Manishearth/6a9367a7d8772e095629e821... with an explanation of how and why they use unsafe. In that comment, he even addresses that he's suprised the number is that high and that work can be done to lower it. You are seriously misrepresenting his contributions to this discussion.
burntsushi 9y ago
> IIRC UTF8 stuff has been a perf bottleneck in the url crate Right, and this isn't surprising at all. It's not at all obvious to me that it's premature. It's really easy for from_utf8 to show up in a profile, and I've had occasion to either work around it safely (perhaps by staying in `&[u8]` land) or by resorting to `unsafe`. I did the latter in the CSV crate for managing `StringRecord`s.[1] The record is stored contiguously in memory, which means that even if the entire record is valid UTF-8 as a whole, then each individual field may not be. Checking UTF-8 validity on each field or on access leads to a slow down. (There are benchmarks.) It's much nicer to code a fast path that checks if the entire record is all ASCII. (Because then you can assume each field is valid UTF-8.) [1] - https://github.com/BurntSushi/rust-csv/blob/34e957e63bb92853ba32c314f613d948e8a3c012/src/string_record.rs#L26-L32 https://github.com/BurntSushi/rust-csv/blob/34e957e63bb92853...
- dbaupp 9y ago
  I have found it very interesting to use the safe function when debug_asserts are on and the unsafe one otherwise (or more generally, some abstraction over safe/unsafe variants that compiles away to nothing in the optimised case). This plays really well with fuzzing/quickcheck (etc) too. My favourite example of doing this personally is: https://github.com/Aatch/ramp/pull/48 https://github.com/Aatch/ramp/pull/48

Animats 9y ago

Note lack of links to code above. Please provide a list of the basic Rust crates which contain no unsafe code. Can you write useful programs using only those crates, not counting ones which need foreign function calls to talk to the OS?

openasocket 9y ago
> Note lack of links to code above. He provided links to several other threads where he explicitly has made a list. In this one https://news.ycombinator.com/item?id=13280347 https://news.ycombinator.com/item?id=13280347 he says he looked at ~600 crates he actually uses, and found 70 used unsafe code. Those 70 are given here: https://gist.github.com/Manishearth/6a9367a7d8772e095629e82184f05ad4 https://gist.github.com/Manishearth/6a9367a7d8772e095629e821... with an explanation of how and why they use unsafe. In that comment, he even addresses that he's suprised the number is that high and that work can be done to lower it. You are seriously misrepresenting his contributions to this discussion.

burntsushi 9y ago

> IIRC UTF8 stuff has been a perf bottleneck in the url crate Right, and this isn't surprising at all. It's not at all obvious to me that it's premature. It's really easy for from_utf8 to show up in a profile, and I've had occasion to either work around it safely (perhaps by staying in `&[u8]` land) or by resorting to `unsafe`. I did the latter in the CSV crate for managing `StringRecord`s.[1] The record is stored contiguously in memory, which means that even if the entire record is valid UTF-8 as a whole, then each individual field may not be. Checking UTF-8 validity on each field or on access leads to a slow down. (There are benchmarks.) It's much nicer to code a fast path that checks if the entire record is all ASCII. (Because then you can assume each field is valid UTF-8.) [1] - https://github.com/BurntSushi/rust-csv/blob/34e957e63bb92853ba32c314f613d948e8a3c012/src/string_record.rs#L26-L32 https://github.com/BurntSushi/rust-csv/blob/34e957e63bb92853...

dbaupp 9y ago
I have found it very interesting to use the safe function when debug_asserts are on and the unsafe one otherwise (or more generally, some abstraction over safe/unsafe variants that compiles away to nothing in the optimised case). This plays really well with fuzzing/quickcheck (etc) too. My favourite example of doing this personally is: https://github.com/Aatch/ramp/pull/48 https://github.com/Aatch/ramp/pull/48