3 ms·
I assume you're asking why the "simple" Rust version is slower than the "simple" Go version? I wrote the simple Rust program. I've also been writing Go for a l
by burntsushi 6y ago
I assume you're asking why the "simple" Rust version is slower than the "simple" Go version?
I wrote the simple Rust program. I've also been writing Go for a long time. It's tough to say precisely why, but here are some guesses:
* The simple variants, by virtue of being simple, do a lot of extra allocation. Go, because of its GC, might be able to do a bit better here. (I believe the Rust program also does more allocations than the Go program.)
* In Rust, all strings are UTF-8 validated. In Go, they are not. In Go, strings are only conventionally UTF-8.
* A good portion of these programs is spent interacting with a hashmap. Rust's default hashing algorithm is chosen to prevent HashDoS attacks[1] at the expense of slower hashing. I actually don't know whether Go's hashmap does the same. So I'm highlighting it here as a potential difference that perhaps someone else can elaborate on.
[1] - https://doc.rust-lang.org/std/collections/struct.HashMap.html https://doc.rust-lang.org/std/collections/struct.HashMap.htm...
- hu3 6y agoHey! The great burntsushi himself answering me. Thanks for taking the time to clarify specifics. I appreciate your attention. I guess in Go strings are just a slice of bytes? Isn't it possible to do the same with Rust for this benchmark?
- burntsushi 6y agoStrings are just slices of bytes in Rust too. Both languages use UTF-8 as their internal representation. The difference is that Rust requires its strings to be valid UTF-8. Go does not. Either choice is reasonable. The downside of requiring valid UTF-8 is that in order to read bytes from a file, you have to do a validation check on them to ensure they are UTF-8 before returning a string back. This is an additional cost. But this is not the only additional cost in the Rust program. I outlined a few others. > Isn't it possible to do the same with Rust for this benchmark? This benchmark has two programs for each language. The "simple" and "optimized" variant. The "simple" version is supposed to be written in an idiomatic style for that language. Taking extra steps to make tweaks and optimize the code is, I think, against the spirit of the challenge. Obviously, this is a very fuzzy concept, and everyone can make up their own mind on the extent to which this framing is useful. (I think it is, personally, especially when you also allow for a second submission that tries to make the program fast.)
- hu3 6y agoTo clarify my question, can Rust read files and split their strings without checking UTF8 correctness? That could speedup the algorithm. In Go I have used this [1] in the past when I had to validate UTF8 encoding but on hot paths where I'm sure UTF8 is valid (coming from sanitized database data for example) I skipped that part. [1] https://golang.org/pkg/unicode/utf8/#Valid https://golang.org/pkg/unicode/utf8/#Valid
- unionpivo 6y agoyes, you can operate on raw byte strings (you write them like b'this is ASCI' ), and then it would look like go or c code.
- burntsushi 6y ago> To clarify my question, can Rust read files and split their strings without checking UTF8 correctness? That could speedup the algorithm. That's what every single Rust program in this benchmark does, except for the simple variant.
- praveenperera 6y agoI had a specific question about the rust implementation. ordered.sort_by(|&(_, cnt1), &(_, cnt2)| cnt2.cmp(&cnt1)); would produce the same result as what was in the blog post: ordered.sort_by(|&(_, cnt1), &(_, cnt2)| cnt1.cmp(&cnt2).reverse()); But would avoiding the `reverse` call make it any faster, or is that a zero cost abstraction?
- burntsushi 6y agoI didn't check the codegen or anything, but: 1) Yes, almost certainly zero-cost. 2) This isn't a hot part of the program. See also: https://old.reddit.com/r/rust/comments/m5ix0s/performance_comparison_counting_words_in_python/gr099cg/ https://old.reddit.com/r/rust/comments/m5ix0s/performance_co...
- komuW 6y ago> prevent HashDoS attacks[1] at the expense of slower hashing. I actually don't know whether Go's hashmap does the same. From looking at the code[1], it seems like it does. From[2] it looks like it is only for platforms that have AES. 1. https://github.com/golang/go/blob/7bfe32f39c59056c49f5776b104beaf09b010088/src/runtime/map.go#L1001-L1003 https://github.com/golang/go/blob/7bfe32f39c59056c49f5776b10... 2. https://github.com/golang/go/issues/9365 https://github.com/golang/go/issues/9365