4 ms·

Do you mean this post? https://www.reddit.com/r/Python/s/raofvsKCiz https://www.reddit.com/r/Python/s/raofvsKCiz That speedup was disputed in the comments. I ha

Do you mean this post? https://www.reddit.com/r/Python/s/raofvsKCiz https://www.reddit.com/r/Python/s/raofvsKCiz That speedup was disputed in the comments. I haven’t tried it myself though

masklinn 3y ago
https://www.reddit.com/r/Python/comments/mgi4op/comment/gswghvi/ https://www.reddit.com/r/Python/comments/mgi4op/comment/gswg... I’d use “thoroughly disproven” rather than “disputed”. I’m sure the hash function can be changed, but as various comments noted: - the benchmark was nonsensical - cpython caches string hashes, and “symbols” are interned, so outside of dynamic attribute access from dynamically constructed strings each hash for attribute purposes or namespace lookup is computed once per process - and finally (though probably not the biggest issue) xxhash is known for being mostly useful on larger sizes (>128 bytes, although you can find better hashes (city IIRC) up to 512 or so) Much like Rust, CPython uses siphash as its default, it’s a pretty good all rounder though not the fastest. It actually used to use FNV before HashDOS. CPython does suffer from the inability of users to configure hash functions since it’s an object property rather than a container property.
- tialaramex 3y ago
  Rust is doing something even more subtle here than just making the hash function a container property. There are two inter-related Rust traits, Hash is a trait which types implement to explain abstractly how to hash that type, but it's written in terms of a Hasher trait so you can drop in a different function with the same API. The Rust standard library provides a derive macro for Hash, so most people can just gesture vaguely at their custom type and have it hashed correctly for any hash functions they need. The naïve approach here easily ends up doing a bad job either colliding hashes for correlated objects in a surprising way or emitting different hashes for equivalent objects because people who make a user defined type probably aren't hashing experts.
  - masklinn 3y ago
    We can get into the weeds about the details, but what I'm talking about is mostly that in Python (and in most languages really, AFAIK Ruby, Java, or C# are the same to list just a few) objects have to return the hash itself, so the hash function is fixed by the type. In Rust, the Hash trait is only used to feed data to a hasher such that the type can decide what should be hashed. The creation of the hasher is done by the collection, and thus provides better opportunities / flexibility in customising the hash function.
    - tialaramex 3y ago
      The derive macro is also key here. Have you ever looked at the machinery needed for Java's URI type to have a halfway useful hashCode implementation? It's pretty elaborate, there's just no way the average programmer will do that work correctly for their own types even ignoring the desire to allow different hash functions. Rust programmers are almost always able to just write #[derive(Hash)] and not worry about it.

Do you mean this post? https://www.reddit.com/r/Python/s/raofvsKCiz https://www.reddit.com/r/Python/s/raofvsKCiz That speedup was disputed in the comments. I haven’t tried it myself though

masklinn 3y ago

https://www.reddit.com/r/Python/comments/mgi4op/comment/gswghvi/ https://www.reddit.com/r/Python/comments/mgi4op/comment/gswg... I’d use “thoroughly disproven” rather than “disputed”. I’m sure the hash function can be changed, but as various comments noted: - the benchmark was nonsensical - cpython caches string hashes, and “symbols” are interned, so outside of dynamic attribute access from dynamically constructed strings each hash for attribute purposes or namespace lookup is computed once per process - and finally (though probably not the biggest issue) xxhash is known for being mostly useful on larger sizes (>128 bytes, although you can find better hashes (city IIRC) up to 512 or so) Much like Rust, CPython uses siphash as its default, it’s a pretty good all rounder though not the fastest. It actually used to use FNV before HashDOS. CPython does suffer from the inability of users to configure hash functions since it’s an object property rather than a container property.

tialaramex 3y ago
Rust is doing something even more subtle here than just making the hash function a container property. There are two inter-related Rust traits, Hash is a trait which types implement to explain abstractly how to hash that type, but it's written in terms of a Hasher trait so you can drop in a different function with the same API. The Rust standard library provides a derive macro for Hash, so most people can just gesture vaguely at their custom type and have it hashed correctly for any hash functions they need. The naïve approach here easily ends up doing a bad job either colliding hashes for correlated objects in a surprising way or emitting different hashes for equivalent objects because people who make a user defined type probably aren't hashing experts.
- masklinn 3y ago
  We can get into the weeds about the details, but what I'm talking about is mostly that in Python (and in most languages really, AFAIK Ruby, Java, or C# are the same to list just a few) objects have to return the hash itself, so the hash function is fixed by the type. In Rust, the Hash trait is only used to feed data to a hasher such that the type can decide what should be hashed. The creation of the hasher is done by the collection, and thus provides better opportunities / flexibility in customising the hash function.
  - tialaramex 3y ago
    The derive macro is also key here. Have you ever looked at the machinery needed for Java's URI type to have a halfway useful hashCode implementation? It's pretty elaborate, there's just no way the average programmer will do that work correctly for their own types even ignoring the desire to allow different hash functions. Rust programmers are almost always able to just write #[derive(Hash)] and not worry about it.