11 ms·

Go 1.21 may have a clear(x) builtin

• halpmeh 4y ago
  The issue with NaN equality is interesting, but is that really why they're adding a clear(x) builtin? What if you want to remove a single-NaN key from a map? clear(x) seems like a band-aid at best if the Go team is strictly trying to fix removing NaN keys in maps.
  • erk__ 4y ago
    The way Rust dels with this issue is by saying that the Key must implement both `Hash` and `Eq` where Eq is a strict equality compared to `PartialEq` which allows the equality relation to be incomplete. While integers implement both PartialEq and Eq, floating point values only implement PartialEq. https://doc.rust-lang.org/std/collections/struct.HashMap.html https://doc.rust-lang.org/std/collections/struct.HashMap.htm... https://doc.rust-lang.org/std/cmp/trait.Eq.html https://doc.rust-lang.org/std/cmp/trait.Eq.html https://doc.rust-lang.org/std/cmp/trait.PartialEq.html https://doc.rust-lang.org/std/cmp/trait.PartialEq.html There is also a partial ordering relation `PartialOrd` (and `Ord` for complete relations) Floating point values only implement the partial relation as well because of NaN's. This means that it is a bit harder to sort them, though the floating point standard also has a seperate complete comparison relation with some extra rules which you can use instead. https://doc.rust-lang.org/std/primitive.f64.html#method.total_cmp https://doc.rust-lang.org/std/primitive.f64.html#method.tota...
    • duped 4y ago
      This is one of those rough edges where Rust is more annoying than helpful imo. It's also inconsistent. I'd rather panic on NaN comparisons than complicate the fundamental comparison traits. Treat it like integer overflow or array bounds access. We don't have different traits for indexing that might fail or overflow that might happen. There are methods on the trait to handle those cases when you care about them. But that said, this feels like an XY issue. If you're comparing hashes of floats as keys somewhere you don't want IEEE 754 defined equivalence. You almost certainly want bit equivalence, which means the keys should be type punned to u32/u64. Classic example is memoizing function calls with floats. You don't want to compare the semantic values of the arguments, but the literal bits. For floats that's not the same.
      • moomin 4y ago
        No you don’t, you want to use the hardware defined equality function. And that one is IEEE 754 compliant.
        • anonymoushn 4y ago
          How are you so sure what other people want? Using the hardware defined equality function for integers is reasonable a lot of the time.
        • duped 4y ago
          My point is sometimes I want NaN to compare with NaN as true when I'm using a float as a key in a table.
          • tialaramex 4y ago
            You are welcome to build a type which has this property and declares that it is Eq. But Rust's built-in floating point types f32 and 64 do not have that property.
        • jeroenhd 4y ago
          In a hash table, I want my equality function to be based on the actual value, not the super special comparison for a specific type. If I write a UniqueNumber that just returns false on the == operator, I'd be stupid but the hash map should still work. Java has a separate getHashCode() and equals() method for a reason. Either provide a hash key override or use the raw bits for deep compares. Custom equality algorithms usually don't make sense for arbitrary key-value stores.
          • tialaramex 4y ago
            Because of the Pigeonhole Principle what you're describing is nonsense. In Rust you can make UniqueNumber, indeed misfortunate::OnewayGreater is such a type. misfortunate::Maxwell even more so. However of course the HashMap can't meaningfully "work" for this type. Rust promises your program doesn't have Undefined Behaviour despite such a prank, but it might spin forever when you try to insert this type into a map for example, a well defined but undesirable behaviour brought on by your poor choices. You seem to imagine that Java's hash map doesn't need to compare types, that it can just use getHashCode() -- however because of the Pigeonhole Principle that can't actually work. And NaN != NaN isn't a "custom equality algorithm" it's literally how the floating point numbers are defined in your CPU.
    • tialaramex 4y ago
      Rust's HashMap and similar containers have clear() and many of them have drain() Drain is interesting, because it resolves the other problem if things are in the collection which can never be retrieved from it. Drain gives you each of the things in the collection, one at a time, to do with as you wish, the collection is now empty. So this means even if the collection has a dozen SillyNonsenses in it, all of which insist they're not the one you were looking for whenever you go looking for a SillyNonsense in the collection, you get all twelve of them out in Drain, and can examine them as you wish. Of course if you just wanted to fish one thing out and then throw the rest away, you can do that, once you drop the result of Drain the rest of the things being drained are dropped.
      • iudqnolq 4y ago
        I thought HashMap behaved in an undefined-but-safe manner if an implementation of Hash or Eq is incorrect. Are you saying there's a reasonable way to put misbehaving types in a HashMap? (I'm pretty sure I recognize your username as someone with way more rust experience than me. For the crowd an implementation of Eq for floats that used the standard comparison operators would be "incorrect", which is why the stdlib doesn't provide one)
        • tialaramex 4y ago
          If your key type K doesn't obey the rules explained in HashMap ("if two keys are equal, their hashes must be equal") then Rust doesn't promise HashMap<K,V>'s functions will do something useful (although it will still be safe if your type is safe). So no this isn't a license to put silly things in hash maps. Non-useful but safe consequences might include spinning indefinitely. But if you have put something daft in a hash map, drain() should definitely get it back out, so there's that.
• martisch 4y ago
  > This for loop is less efficient than clearing a map in one operation For maps with keys that are reflexive with == the Go compiler already optimizes the range loop to a single efficient runtime map clear call: https://go-review.googlesource.com/c/go/+/110055 https://go-review.googlesource.com/c/go/+/110055
  • tapirl 4y ago
    Yes, for most cases, the for-loop is as efficient as the proposed "clear" function. I am not sure whether or not this is still true if there are NaN keys in the cleared map.
• inglor 4y ago
  Interestingly JavaScript handles this correctly and specified Object.is equality that is different than `===` equality around NaNs so `const m = new Map(); m.set(NaN, 3); m.get(NaN)` returns 3 and `.delete` similarly works.
• ch_123 4y ago
  I'm curious why this is not implemented as a method on the map type (and others like list) instead of being a top-level builtin. I suppose it is consistent with other collection operations such as append and len... which I guess makes me wonder why those are builtins as well.
  • esprehn 4y ago
    The answer is lack of generics in all previous versions of Go. There's no way to express a generic interface to have methods for map or slice (which are not interfaces, but compiler magic instead). All the operations on builtin types must be implemented as globals handled by the compiler.
    • hesdeadjim 4y ago
      Been using Go since the early days, and 'make/append' always felt clunky and out of place. Especially when you could just use literals for a map/slice...
      • chimeracoder 4y ago
        > Been using Go since the early days, and 'make/append' always felt clunky and out of place. Especially when you could just use literals for a map/slice... Most people do use the literals for creating the empty map/slice. "make" is mostly useful when specifying a known length or capacity, which saves unnecessary allocations and improves performance.
  • urzaj 4y ago
    From the go FAQ: > Why is len a function and not a method? > We debated this issue but decided implementing len and friends as functions was fine in practice and didn't complicate questions about the interface (in the Go type sense) of basic types. I imagine similar reasoning applies to "clear" here. https://go.dev/doc/faq#methods_on_basics https://go.dev/doc/faq#methods_on_basics
    • 4y ago
• akira2501 4y ago
  You've already got zero and negative zero as an outlying case. I'm not sure why anyone would feel comfortable using floats as keys in a map. To anyone who has done this.. why? What was the use case?
  • craigching 4y ago
    Go has no built-in set data structure, you use map if you don’t want to bring in a third party. Maybe this is why?
    • Ferret7446 4y ago
      A set of floats sounds like an equally questionable idea as a map of float keys, for the exact same reason.
  • deleted 4y ago
    [deleted]
  • wallstop 4y ago
    For a game I'm making, I have a helper function that takes in a specified time (as a float) and returns a (heap based) wait instruction. The times are literals sprinkled throughout the code base of callers of this function, so I maintain a cache of time -> wait instruction to reduce allocations.
    • iudqnolq 4y ago
      Wouldn't your life be much simplier if you'd represented time as integer milliseconds? I totally get it being too late now, but I think that's an argument for making floats a little bit unpleasant to start working with.
• zmj 4y ago
  C# had the same NaN behavior until recently: https://learn.microsoft.com/en-us/dotnet/core/compatibility/core-libraries/7.0/equals-nan https://learn.microsoft.com/en-us/dotnet/core/compatibility/...
  • rawling 4y ago
    Oof, I don't like that Matrix3x2 type. "It's a 3x2 matrix, but we're going to pretend it has a last column going 0/0/1 so we can get a determinant or multiply them together, but when we negate or add or subtract them it's still going to be 0/0/1."
    • _old_dude_ 4y ago
      It's the classical representation of affine transforms (translate, rotate, scale, etc) using homogeneous coordinates. https://en.wikipedia.org/wiki/Transformation_matrix#Affine_transformations https://en.wikipedia.org/wiki/Transformation_matrix#Affine_t...
      • rawling 4y ago
        Ah, thanks. I figured it might be something like this but the documentation is very dry and doesn't seem to mention it.
• kangalioo 4y ago
  Would anything actually break if NaN == NaN equalled true?
  • moomin 4y ago
    Yes, in surprising places. But the most obvious thing is that comparing float values would become significantly more expensive.
    • derriz 4y ago
      Could you explain how/why?
      • crote 4y ago
        Virtually all architectures have floats implemented in hardware - including NaN behavior. This means things like IEEE 754 float comparison is done in a single instruction, executed in a single cycle. Explicitly checking for NaNs would (at least) double the cycle count in cases where that branch is not taken, and having to deal with the custom comparison is even worse. This is not helped by the fact that there isn't a single NaN value: there are both positive and negative NaNs, and there are 23 bits which are usually ignored but for which some values do have specific meaning. To make it even worse, modern CPUs include vector extensions, allowing you to operate on multiple values at once. With AVX-512, you can compare 32 floats per clock cycle. I would not be surprised if switching to a custom comparison made some edge cases over 100x slower.
        • anonymoushn 4y ago
          Comparing the raw bits using instructions intended for integers is quite fast.
          • masklinn 4y ago
            NaN is not a single value, it's a multitude of them: a NaN is defined by all exponent bits being set to 1, so in a double there are 2^53 different possible NaN bit patterns (though only 2^52 values, because the sign bit is not part of the "NaN value", and possibly half that if you interpret signaling as a variable property of a fixed value)
• yawaramin 4y ago
  OK, now I understand why OCaml's Float.compare function compares NaNs as equal to each other: https://discuss.ocaml.org/t/assertions-involving-nan/10762/4 https://discuss.ocaml.org/t/assertions-involving-nan/10762/4
• _old_dude_ 4y ago
  In Java, the primitive type double uses the IEEE-754 semantics and java.lang.Double uses the bits by bits comparison semantics [1] so List<Double>.remove() works correctly. [1] https://docs.oracle.com/en/java/javase/19/docs/api/java.base/java/lang/Double.html https://docs.oracle.com/en/java/javase/19/docs/api/java.base...
  • uluyol 4y ago
    This won't help you in a future version of Java where you can write List<double>, right?
    • masklinn 4y ago
      No. What will help you is that `clear()` is part of `Java.until.Collection` so just about every container has it.
    • _old_dude_ 4y ago
      It depends if List<double> is an alias for List<@PleaseSpecialize @NotNull Double> or not.
• derriz 4y ago
  Java has a specific hack for this which I discovered by accident a few years ago. Normally, given (primitive) double values d0 and d1, then boxing them does not affect equality testing - i.e. d0 == d1 if and only if Double.valueOf(d0).equals(Double.valueOf(d1)). However if d0 and d1 are both NaN, then the boxed versions ARE considered equal while d0 == d1 is false. This inconsistency infuriated me when I discovered it but the Javadoc for Double.equals explicitly states that this anomaly is there to "allow hash tables to work properly".
  • bobbylarrybobby 4y ago
    Using floats as hash keys is insane, no?
    • metadat 4y ago
      Yes, why would you ever want a map with floating point keys? I'm struggling to think of a valid use case where there is no better alternative. Any design utilizing this language "feature" seems masochistic and begging to get sliced by sheet metal edges.
      • masklinn 4y ago
        > Yes, why would you ever want a map with floating point keys? Because it's a cache and you didn't specifically handle that edge case.
        • malcolmgreaves 4y ago
          I don't know what precisely the poster meant. However, from my perspective, I see it as a simpler design to explicitly hash float values into keys.
          • masklinn 4y ago
            That... still requires the foresight to see that edge case, and being in a situation where you're not colliding your float hashes with random integers?
• unwind 4y ago
  That was weird. Makes sense once you know about NaN's incomparability, but still surprising. I had to try it in Python (3.10.4, Windows on x86), it worked fine: >>> import math >>> a={math.nan: 1} >>> a {nan: 1} >>> del(a[math.nan]) >>> a {}
  • vhcr 4y ago
    This works as expected only because id(math.nan) is always the same, if you build the nan dynamically, it will fail: >>> d = {float('nan'): 1} >>> d {nan: 1} >>> del d[float('nan')] Traceback (most recent call last): File "<stdin>", line 1, in <module> KeyError: nan
  • masklinn 4y ago
    It's because cpython shortcuts through identity: https://github.com/python/cpython/blob/main/Objects/object.c#L724-L731 https://github.com/python/cpython/blob/main/Objects/object.c... /* Quick result when objects are the same. Guarantees that identity implies equality. */ if (v == w) { if (op == Py_EQ) return 1; else if (op == Py_NE) return 0; } `math.nan` is always itself, and thus this shortcut is taken and the key works.
  • deleted 4y ago
    [deleted]
• chubot 4y ago
  Meh using floating point as keys for maps in any language is just asking for trouble -- it’s not just NaN I don’t think there’s any real use case for it I’d say clear() is good for clarity, and that’s it
  • mprime1 4y ago
    > I don’t think there’s any real use case for it Histogram where you want to keep a count for each float value you’ve seen (maybe rounded to some precision to reduce the number of buckets) Not disagreeing with your comment, just saying it’s not uncommon
    • iudqnolq 4y ago
      Why not just multiply each key by ten to the power of your desired precision, then convert to int? If you're rounding anyway it's basically the same amount of work.
      • mprime1 4y ago
        You can certainly map floats to int ‘bin number’ which is the index into an array. No need for a map. But the ‘naive’ histogram is an example of a map using float keys.
    • chubot 4y ago
      Hm I don't agree that example means it's "not uncommon" -- why would you want to hash the float rather than the bucket number? Hashing the floats themselves isn't going to give you a useful histogram. Is there a codebase that hashes float to a useful effect? If it exists, I'd be interested to see it
• kazinator 4y ago
  This looks like catching up to GNU Awk, which allows "delete a" to clear an associative array. That's an extension over POSIX, which specifies "delete a[key]".
• karmakaze 4y ago
  Usage of NaN as hash key reminds me of the two uses of NULL in SQL. One is as unknown values which are never equal to anything even other NULL values. Another use is as a key for aggregate grouping. In that case the entry represents the aggregate for all the unknown values which aren't equal but still grouped together. Different uses have different meanings so invalid in one use does not invalidate other uses.
• kazinator 4y ago
  > This for loop is less efficient than clearing a map in one operation That is not obvious; a compiler could have a pattern match for that exact AST pattern, and transform it to a delete operation. (Except for that pesky issue where two fail to be equivalent due to NaN keys.) Quick and dirty, not entirely correct proof-of-concept in TXR Lisp: 1> (macroexpand '(my-dohash (k v some.obj.hash) (print [some.obj.hash k]))) (dohash (k v some.obj.hash ()) (print [some.obj.hash k])) 2> (macroexpand '(my-dohash (k v some.obj.hash) (del [some.obj.hash k]))) (clearhash hash) Impl: (defmacro my-dohash ((kvar vvar hash : result) . body) (if-match @(require ((del [@hash @kvar])) (null result)) body ^(clearhash hash) ^(dohash (,kvar ,vvar ,hash ,result) ,*body)))
  • coder543 4y ago
    https://news.ycombinator.com/item?id=33673792 https://news.ycombinator.com/item?id=33673792
• anontrot 4y ago
  Hold on a sec, the whole clearing map scenario. Is it for reusing within your fn or is it trying to free memory? If latter, I’ve just been hoping GC does its job
• yencabulator 4y ago
  I'm more surprised they went with a new built-in instead of making the compiler recognize something like m = make(map[A]B, cap(m)) just like it already recognized for k := range m { delete(m, k) } and many other similar idioms. (cap because len is not quite the same -- note, cap is not currently defined on maps) EDIT: Likely because that's an assigment on m, not a mutation of it, so it can't be done e.g. in a function that gets m as argument.