5 ms·

Why do you say premature? Maps, for example, being such a crucial and widely-used data type, should be optimized as much as possible. There's nothing premature

by eridius 10y ago

Why do you say premature? Maps, for example, being such a crucial and widely-used data type, should be optimized as much as possible. There's nothing premature about using `unsafe` to build a highly-efficient Map.

Animats 10y ago
Let's see the benchmarks justifying the use of "unsafe" for maps. Maybe there's a better way to do it without much of a performance penalty. It's a good way to find out what optimizations the compiler is missing. It may even turn out that unsafe code written early no longer is a performance win, since the Rust compiler is getting better at optimizing out redundant subscript checks. When you start looking through Rust libraries, "unsafe" turns up way too often.
- leshow 10y ago
  Unsafe rarely turns up in any of the libraries I've used. Where are you getting this from?
- Manishearth 10y ago
  (The hashmap that is used in the winning rust benchmarksgame entry is 100% safe, fwiw) > When you start looking through Rust libraries, "unsafe" turns up way too often. You keep making this claim without substantiation. Yes, there is some level of unnecessary unsafe, but certainly not "way too often". I recall going through all the crates in my .cargo and finding very little unnecessary unsafe, and showing you the audit: https://news.ycombinator.com/item?id=13280347 https://news.ycombinator.com/item?id=13280347 Please stop throwing around this claim without substantiation.
  - yazaddaruvala 10y ago
    Do you think there is some room for education here? Would it be useful for Crates.io to show some "coverage" metric? eg. "unsafe lines = 5%" or "unsafe instructions = 0.01%" I think that would be a pretty easy way to put this convo to bed yea? Have there been talks about this before? Is it worth me opening an issue/RFC on Cargo?
    - Manishearth 10y ago
      I don't think it's worth making changes like that to satisfy random misconceptions unless they are widespread. There are good reasons for having an unsafe-code flag on a crate. This is not one of those. This has been discussed in the past and the main concern IIRC is that this may make people afraid to use crates that contain unsafe code even if it is well audited. There's a finer balance you want to strike here.
  - Animats 10y ago
    We've seen this unsubstantiated denial before on YC, at "Why I’m dropping Rust"[1] and "Rust sucks if I fail to write X".[2] I once started going through the Rust library packages and listed uses of "unsafe". Try doing that. The previous discussions made a few things clear: - The big design-level problems with data structure safety are 1) partially initialized arrays, and 2) backlinks. The first is needed for growing arrays in Vec, and the second is needed for doubly-linked lists and some kinds of trees. It's very hard to handle either of those in safe Rust. A very small number of packages need unsafe code for those functions, and those should be tightly controlled. - Foreign code remains a problem, and is inherently unsafe when calling unsafe languages. The most downloaded Rust crate is "libc". What could possibly go wrong there? Was it really a good idea to import unsafe "strcpy" into Rust? pub fn strcpy(dst: *mut c_char, src: *const c_char) -> *mut c_char; - There's a lot of use of "unsafe" code that's not strictly necessary. You find unsafe code like "from_utf8_unchecked", which then turns up in the JSON decoder at (https://github.com/rust-lang-deprecated/rustc-serialize/blob/master/src/json.rs https://github.com/rust-lang-deprecated/rustc-serialize/blob...). There are no comments on the safety of that. Is there some way to create bad JSON, get bad UTF-8 into a string, and cause trouble further upstream? I don't know, but somebody "optimized" there, and created a potential problem. I could give many more examples. Most of these problems are fixable. They're not inherent in Rust. Fixing them is important to Rust's credibility. If Rust is going to replace C++, which it should, the holes have to be plugged. It only takes one hole to create a security vulnerability. Denial is not a river in Egypt. [1] https://news.ycombinator.com/item?id=12474445 https://news.ycombinator.com/item?id=12474445 [2] https://news.ycombinator.com/item?id=13670366 https://news.ycombinator.com/item?id=13670366
    - crzwdjk 10y ago
      > which then turns up in the JSON decoder at (https://github.com/rust-lang-deprecated/rustc-serialize/blob... https://github.com/rust-lang-deprecated/rustc-serialize/blob...). There are no comments on the safety of that. Maybe there's a reason it's in a repository labeled "deprecated", namely that it's deprecated. The replacement serialization framework, serde, has exactly one appearance of "unsafe", in a function that is only compiled when you explicitly enable the "unstable" feature flag and that in fact appears to be a reasonably safe use of unsafe.
  - lossolo 10y ago
    > winning rust benchmarksgame entry C is winning that benchmark, around 2x faster than Rust. http://benchmarksgame.alioth.debian.org/u64q/regexredux.html http://benchmarksgame.alioth.debian.org/u64q/regexredux.html
    - Manishearth 10y ago
      Wrong benchmark. I'm talking about k-nucleotide, which tests hashmaps. http://benchmarksgame.alioth.debian.org/u64q/knucleotide.html http://benchmarksgame.alioth.debian.org/u64q/knucleotide.htm... (And I was only really talking of the comparison of rust v rust, not rust v c, i.e. that ordermap is faster than the built in hashmap despite not using unsafe)
      - lossolo 10y ago
        Ah, sorry, you were writing about hashmaps and I was writing regexes in my IDE in background while reading this thread from time to time, probably that's why I've checked regex instead of hashmap benchmark. I even wrote this benchmark in language not listed on benchmarks game lately to compare the speed. Brain can work in interesting ways!

by eridius 10y ago

Animats 10y ago

Let's see the benchmarks justifying the use of "unsafe" for maps. Maybe there's a better way to do it without much of a performance penalty. It's a good way to find out what optimizations the compiler is missing. It may even turn out that unsafe code written early no longer is a performance win, since the Rust compiler is getting better at optimizing out redundant subscript checks. When you start looking through Rust libraries, "unsafe" turns up way too often.

leshow 10y ago
Unsafe rarely turns up in any of the libraries I've used. Where are you getting this from?
Manishearth 10y ago
(The hashmap that is used in the winning rust benchmarksgame entry is 100% safe, fwiw) > When you start looking through Rust libraries, "unsafe" turns up way too often. You keep making this claim without substantiation. Yes, there is some level of unnecessary unsafe, but certainly not "way too often". I recall going through all the crates in my .cargo and finding very little unnecessary unsafe, and showing you the audit: https://news.ycombinator.com/item?id=13280347 https://news.ycombinator.com/item?id=13280347 Please stop throwing around this claim without substantiation.
- yazaddaruvala 10y ago
  Do you think there is some room for education here? Would it be useful for Crates.io to show some "coverage" metric? eg. "unsafe lines = 5%" or "unsafe instructions = 0.01%" I think that would be a pretty easy way to put this convo to bed yea? Have there been talks about this before? Is it worth me opening an issue/RFC on Cargo?
  - Manishearth 10y ago
    I don't think it's worth making changes like that to satisfy random misconceptions unless they are widespread. There are good reasons for having an unsafe-code flag on a crate. This is not one of those. This has been discussed in the past and the main concern IIRC is that this may make people afraid to use crates that contain unsafe code even if it is well audited. There's a finer balance you want to strike here.
- Animats 10y ago
  We've seen this unsubstantiated denial before on YC, at "Why I’m dropping Rust"[1] and "Rust sucks if I fail to write X".[2] I once started going through the Rust library packages and listed uses of "unsafe". Try doing that. The previous discussions made a few things clear: - The big design-level problems with data structure safety are 1) partially initialized arrays, and 2) backlinks. The first is needed for growing arrays in Vec, and the second is needed for doubly-linked lists and some kinds of trees. It's very hard to handle either of those in safe Rust. A very small number of packages need unsafe code for those functions, and those should be tightly controlled. - Foreign code remains a problem, and is inherently unsafe when calling unsafe languages. The most downloaded Rust crate is "libc". What could possibly go wrong there? Was it really a good idea to import unsafe "strcpy" into Rust? pub fn strcpy(dst: *mut c_char, src: *const c_char) -> *mut c_char; - There's a lot of use of "unsafe" code that's not strictly necessary. You find unsafe code like "from_utf8_unchecked", which then turns up in the JSON decoder at (https://github.com/rust-lang-deprecated/rustc-serialize/blob/master/src/json.rs https://github.com/rust-lang-deprecated/rustc-serialize/blob...). There are no comments on the safety of that. Is there some way to create bad JSON, get bad UTF-8 into a string, and cause trouble further upstream? I don't know, but somebody "optimized" there, and created a potential problem. I could give many more examples. Most of these problems are fixable. They're not inherent in Rust. Fixing them is important to Rust's credibility. If Rust is going to replace C++, which it should, the holes have to be plugged. It only takes one hole to create a security vulnerability. Denial is not a river in Egypt. [1] https://news.ycombinator.com/item?id=12474445 https://news.ycombinator.com/item?id=12474445 [2] https://news.ycombinator.com/item?id=13670366 https://news.ycombinator.com/item?id=13670366
  - crzwdjk 10y ago
    > which then turns up in the JSON decoder at (https://github.com/rust-lang-deprecated/rustc-serialize/blob... https://github.com/rust-lang-deprecated/rustc-serialize/blob...). There are no comments on the safety of that. Maybe there's a reason it's in a repository labeled "deprecated", namely that it's deprecated. The replacement serialization framework, serde, has exactly one appearance of "unsafe", in a function that is only compiled when you explicitly enable the "unstable" feature flag and that in fact appears to be a reasonably safe use of unsafe.
- lossolo 10y ago
  > winning rust benchmarksgame entry C is winning that benchmark, around 2x faster than Rust. http://benchmarksgame.alioth.debian.org/u64q/regexredux.html http://benchmarksgame.alioth.debian.org/u64q/regexredux.html
  - Manishearth 10y ago
    Wrong benchmark. I'm talking about k-nucleotide, which tests hashmaps. http://benchmarksgame.alioth.debian.org/u64q/knucleotide.html http://benchmarksgame.alioth.debian.org/u64q/knucleotide.htm... (And I was only really talking of the comparison of rust v rust, not rust v c, i.e. that ordermap is faster than the built in hashmap despite not using unsafe)
    - lossolo 10y ago
      Ah, sorry, you were writing about hashmaps and I was writing regexes in my IDE in background while reading this thread from time to time, probably that's why I've checked regex instead of hashmap benchmark. I even wrote this benchmark in language not listed on benchmarks game lately to compare the speed. Brain can work in interesting ways!