3 ms·
Super simple to detect and try again.
by erikerikson 5mo ago
Super simple to detect and try again.
- jandrewrogers 5mo agoA collision is simple to detect but it requires you to actually check, which is expensive at scale. The entire point of UUIDv4 is that you don't have to check for collisions because it should never happen. But if you don't check and it does happen you are in UB territory which is generally very bad. A risk of collision before it happens is non-trivial to detect but this is really what you'd want.
- erikerikson 5mo agoOnly expensive if you have unsorted keys or lack an index. Neither of which are unscalable.
- orf 5mo agoAKA centralising a decentralised identifier generator?
- erikerikson 5mo agoThere are better approaches like pre -avoiding collisions but generating tends to be more expensive than checking.
- orf 5mo agoIn what world is generating a UUID more expensive than checking for duplicates? at any scale? Walk me through that please
- erikerikson 5mo agoYeah, that was a little sloppy but it's generating is more expensive than not generating. In more words, generating an id and validating uniqueness is more expensive than only validating uniqueness.
- soni96pl 5mo agoUUID is used where checking is difficult, think distributed devices offline at a plantation. How could checking be easier in that case? It would require infrastructure that doesn't exist. There are many other cases where it's easier to handle collisions.
- erikerikson 5mo agoAgreed. While it's an uncommon scenario it is a significant case and UUID shouldn't be used in that case because as you write checking doesn't work well. Better to use an alternative such as coordinate/reserve monotonic producer IDs that get paired with monotonic, per-producer monotonic sequence IDs to produce guaranteed unique, well-ordered IDs. [edit: in IOT it's common to issue x509 certificates as a type of authentication which could be used instead of using producer IDs. Solutions always have to be paired to use case.]
- tommoneytools 5mo agoWild edge case. Curious if they ever found the root cause.
- erikerikson 5mo agoWelcome to Hacker News. I suspect you meant to reply to a different comment. Regardless the most plausible speculation I've read here is that the RNG used to generate the UUID is low quality.
- keeganpoppen 5mo agoexactly lmao. that is exactly what is being presented as "scalable <full stop>". sigh.
- erikerikson 5mo agoNo one has yet defined the scale but almost all of the real world scenarios people are actually encountering would be handled by either of the offered solutions.
- keeganpoppen 5mo agoas would be a million things that don’t involve using uuids at all. it literally means nothing to say “this can be handled by centralizing your uuid store”… if you do that, you DON’T NEED UUIDS. THAT IS THE POINT!
- erikerikson 5mo agoYou seem to be insisting that UUID meets the promise of it's name, despite the long standing documentation to the contrary. Is that your intention?
- jandrewrogers 5mo agoYou must have missed the “at scale” part. There is nothing inexpensive about extra network hops, cache misses, and page faults implied by your solution. Indexing at scale is almost always lossy for performance reasons. The location where you insert a new record is frequently not the same location as where you have to search for an existing record. It is resource amplification all the way down. In a lot of systems that index these keys the cost of that check is several times that of doing a blind insert.
- erikerikson 5mo agoNo I didn't miss it. DynamoDb works fine, using CQRS if necessary.
- keeganpoppen 5mo agoliterally the whole point of randomly generating UUIDs is that you don't need to check for collision. that's what the "U"s are for. that is the abstraction that is supposedly being provided. "using <insert Amazon AWS Certification Test Answer #7>" is not in any way a "scalable solution" for that with no other context. nor is just throwing out <random Martin Fowler concept #27>. the whole point is that it is a global (well, per name, "universal") abstraction that can, in practice, have holes that make it so you can't use it "universal"-ly.
- erikerikson 5mo agoI totally appreciate what you are complaining about. It's always been part of the documentation for a UUID. Having had Martin Fowler as a colleague and meeting with him weekly for a bit, I'd expect him to nod along with what I've written. It's standard knowledge and part of the technical corpus. As is actually distributed unique ID generation which is also not hard.
- keeganpoppen 5mo agowell it’s especially not hard when you turn it into a completely different problem
- squirrellous 5mo agoIn this specific case. In the case of trace IDs (an example of which is [1]) where the equivalent of UUIDs are explicitly used to avoid coordination, it’s hard to imagine how you’d reliably detect and retry. [1] https://news.ycombinator.com/item?id=48033853 https://news.ycombinator.com/item?id=48033853
- erikerikson 5mo agoA lot of databases have a uniqueness constraint that is basically a register level compare and replace. Others have a if_not_exists which is nearly the same. If you're not targeting a serious throughput use case, it's enough. If you are then there are lots of solutions/alternatives that completely avoid coordination. On the other hand, maybe tracing protocols are robust to out of order delivery. If that won't do them sequence numbers tied to monotonic sequence IDs should be plenty. If not then I'd need very serious conversations to be convinced you're not wasting everyone's time