10 ms·
TIL: Versions of UUID and when to use them
- MrDarcy 2y agoJust use v7. Cue the security experts who say otherwise…
- wongarsu 2y agoUse v4 if creation date could conceivably be sensitive information of if you depend on your uuids being completely unguessable. Otherwise use v7
- stavros 2y agoIf we want a v7, shouldn't we use a ULID instead?
- wongarsu 2y agoWhen we didn't have UUIDv7, ULID was great. But now that we have v7 it's the more widely supported alternative. And apart from v7 setting the UUID version bits and having a different default representation they are not that different.
- stavros 2y agoOh, I didn't realize v7 was newer than ULID, thanks.
- 6512398 2y agoBe careful if you're using python. The python uuid standard library doesn't have V7 yet, and there is a package called uuid7 which is unmaintained, and not in compliance with the latest standard. That's using nanosecond time precision rather than millisecond, which means the leading bits are larger than they are meant to be. If you use that unmaintained uuid7 package and later change to the correct implementation your uuid7 will go backwards, which is a breaking change considering that monotonicity is a key property of uuidv7.
- bruce511 2y agoI believe your concern, while valid, only affects systems which are currently adding records faster than 1 per nano-second on occasion. Also while technically true - it could technically break monotonicity (records added in the same nanosecond could be out of order) they'll still be all "near the end of the file, likely in the same page" such that performance implications are negligible. As a general rule I would avoid any program making any assumptions about a uuid. Programs should treat it as an opaque binary random value. Doing so avoids any future incompatibilities.
- eropple 2y agoULID's presentation format is probably better for humans, though. You can double-click-to-highlight a ULID; the standard UUID representation doesn't like this. (You can use ULID's presentational tools with UUIDv7, though.)
- wtetzner 2y ago> the standard UUID representation doesn't like this. Yeah, I've gotten in the habit of stripping hyphens from the string representation of UUIDs in a lot of the code I write for that reason.
- beart 2y agoIt doesn't help in other tools, but there is a css rule to help with this. https://developer.mozilla.org/en-US/docs/Web/CSS/user-select https://developer.mozilla.org/en-US/docs/Web/CSS/user-select
- Lammy 2y ago> You can double-click-to-highlight a ULID; the standard UUID representation doesn't like this. You can control this behavior in CSS with `user-select`. Peep my fiddle: https://jsfiddle.net/gLyph5km/ https://jsfiddle.net/gLyph5km/
- eropple 2y agoYup, in a browser you can. In my terminal or my text editor or Slack, I can't.
- fiddlerwoaroof 2y agoAn issue I've always had with UUIDs and ULIDs is there isn't a great way to generate one deterministically, as far as I can tell: for a lot of use-cases, being able to reprocess data and generate identical IDs is really useful and there isn't a standard way that I know of to achieve this.
- exe34 2y agohttps://stackoverflow.com/a/64229385 https://stackoverflow.com/a/64229385
- fiddlerwoaroof 2y agoSure, there are workarounds in various languages, but it would nice to have a standardized hash-based UUID or ULID
- 1986 2y agofrom the article, it sounds like this is V5?
- fiddlerwoaroof 2y agoI missed that because I typically am using ULIDs these days. But, yeah, some standardized format for a hash of message data is what I want.
- IggleSniggle 2y agoIf it's a standardized sequence, then that's no different than just 0, 1, 2, 3 but with different names. If you just want a non-sequential but deterministic sequence, then that's every random number generator that accepts a seed value, and being anymore standardized than that makes zero sense.
- fiddlerwoaroof 2y agoThe problem with autoincrement in this context is you can’t reproduce the right value when replaying the input streams for your stream processing job. Hashing some combination of values and using that as a primary key solves this problem nicely and, when you’re using bitemporal data modeling, makes it easy to correct mistakes. The point of standardization is compatibility, not standardizing the sequence of keys used.
- treve 2y agoWhy are you dismissive of security-related issues?
- tonetegeatinst 2y agoBecause developers don't always consider the security aspect. Not saying this is what he's doing but could also just be due to how complex good software can be to write. Their is a reason cybersecurity or UI/UX or product design isn't always left to the developer. The coder write code that fits certain criteria they are given, then someone down the line might QA check it, fuzz inputs or security review the code. How well this is done depends on the product,market, and environment.
- sgarland 2y agoBecause at least half if not more of the industry sector is charlatans. Similar to how devs love to work on new shiny tech, regardless of its actual applicability to their needs, security folk will often insist that X is insecure due to absurdly unlikely edge case wherein you’re already pwned anyway, while ignoring some basic problem in their org.
- elric 2y agoThat doesn't mean that it's not a valid concern in some cases, and it's good to be aware of it. When you're building something, and need a UUID, and stop think "v7 seems useful, but wait, wasn't there a security thing?", that's a win. Even if the answer turns out to be "not a problem in this case".
- MrDarcy 2y agoI’m not dismissive of security related issues. My comment was a snarky reference to some other HN comments on uuid7 some time ago which I’m having trouble finding now. The comment was lamenting developers using uuid7 for many things, presumably without thought, creating loads of security issues for them. In most cases creation time is not sensitive. Therefore, for most cases uuid7 is the best trade off currently.
- Vecr 2y agoI suggest not using any of the MAC based versions. In theory that could be anything other than v4 and v7, but v1 is the worst. As well as v3, MD5 is horribly broken.
- tashbarg 2y agoMD5 is “broken” as a cryptographic hash function. It still is perfectly fine as a non-cryptographic hash function.
- Vecr 2y agoNot really, it's slower than truncated blake3 for no gain and much loss.
- zerodensity 2y agoBut is it slower than sha1? Which is the alternative if you don't roll your own in V8.
- Vecr 2y agoAbout six times faster compared to sha1. Depends on the hardware/cache environment.
- bigiain 2y agoThere's some gain to be had in that I can reliably expect md5 to be available and compatible with pretty much anything back as far as Perl4 or PHP from the 90s, right up to bleeding edge version of Rust or Clojure or exotic language de jour. Whether that's actually worth anything for a particular use case is a good question, and the answer will mostly be "not just no but HELL NO!"
- slaymaker1907 2y agoYeah, if you really need non-guessability, you should be using the version that’s completely random anyways.
- motohagiography 2y agoWhile I didn't know the details of ones other than 4, the one really useful one missing would be using some SHA256 data with a counter, not unlike PBKDF2. It could be a privacy preserving derived identifier, where you you could loosely prove a given UUID had been derived from a given seed.
- tacitusarc 2y agoLike V3 but with a specifiable hash algorithm.
- kyzer-davis 2y agoThe appendix details how one can do a sha256 style v8 algo that is basically v5 with the algorithm swapped. In theory one could do this for any hashing algorithm. https://www.rfc-editor.org/rfc/rfc9562.html#name-example-of-a-uuidv8-value-n https://www.rfc-editor.org/rfc/rfc9562.html#name-example-of-... Though I am working on a way to solve this problem with UUIDs beyond 128 bits so we don't have to truncate the hash.
- dchest 2y agoIf you really need it, consider UUIDv4 as encoding, and use your cryptographic algorithm to create a 122-bit output and encode it as a UUID. Otherwise, you'd want longer outputs.
- hamasho 2y agoI wish there's a standard for short UUID, like `73WakrfVbNJBaAmhQtEeDv` or `bK7nP9xM`. I mean, it's not UUID cause it can be duplicated somewhere, I just want an ID standart combination of random and short enough to remember.
- kyzer-davis 2y agoWe are working on standardizing an alternate encoding technique for the 128 bit UUID so it can have a shorter text form. You can the discussions here: https://github.com/uuid6/new-uuid-encoding-techniques-ietf-draft https://github.com/uuid6/new-uuid-encoding-techniques-ietf-d...
- jfdjkfdhjds 2y agojust use creation timedate plus auto increment int. and then a small hash with base64 or 37 or whatever is in vogue these days. thats what old timers used before uuid 1. guess we should guerilla standardize something like this as uuid-0 or uuid-deprecated-2.0 for keeping up with the spirit.
- selcuka 2y ago> just use creation timedate plus auto increment int. The problem with auto-increment integer ids is they are not always possible with distributed systems.
- jagrsw 2y agoImagine how many careers have been built on inventing and promoting something, in the end, turned out to be a cleverly encoded output from /dev/urandom.
- zerodensity 2y agoImmagine a world where every framework / API / database had its own incompatible UUID format. Without a standard specification that's where we would end up. Do you want to live in such a world?
- bruce511 2y agoIf you treat the uuid as an opaque binary random value (which is how programs -should- treat it) then variances between versions, or custom versions, have no effect. As long as they gave sufficient randomness etc, from a program perspective they are unique id's. There are already multiple versions in active use (4, 7 and arguably 8) so you really shouldn't be using the uuid as anything but a long-random-value. Yes, the database engine may appreciate one version over another for performance reasons, but that's irrelevant to most developers and programs.
- jagrsw 2y agoForget universally compatible UID formats. Frameworks, APIs, and databases only need consistency within their own ecosystem. Want visually recognizable unique identifiers? JAGRSW-UID-<192bit-input-from-urandom-encoded-in-base64> Need to shave off some bytes? JUID-<192bit-input-from-urandom-encoded-in-base64> Same byte size as UUIDs, arguably more "secure." Can I become an ACM Fellow for solving this problem now? Seriously, these UUID debates are about as sensible as arguing over XML.
- dchest 2y agoIndeed. Also, add many other unique identifier formats (ULID, nanoid, bson ids ...) — in contrast to compatibility complaints here — and you'll see that people just love tinkering with trivial things (me too). What was important during the times when we didn't know how to generate random numbers on computers, perhaps shouldn't be as important today?
- JSDevOps 2y agoInteresting read. You learn something everyday.
- Lammy 2y ago> UUID Version 2 (v2) is reserved for security IDs with no known details. Only no known details if the only document you're reading is the notoriously poorly-specified RFC. Here you go: https://pubs.opengroup.org/onlinepubs/9696989899/chap5.htm#tagcjh_08_02_01_01 https://pubs.opengroup.org/onlinepubs/9696989899/chap5.htm#t... There are also “version 0” UUIDs that you are very unlikely to ever come across but should be noted because they are the source of the reserved bits (via wastefully setting aside an entire octet for Address Family) that later allowed the other “versions” to be specified in a compatible way. Read my research about them here in my UUID library: https://github.com/okeeblow/DistorteD/blob/NEW%E2%80%85SENSATION/Globe%20Glitter/lib/globeglitter/chrono_diver.rb#L61-L141 https://github.com/okeeblow/DistorteD/blob/NEW%E2%80%85SENSA... I decided to support them Because It's Cool™ but still need to figure out how to handle the date rollover of them and the even-older Apollo UIDs: irb> ::GlobeGlitter::from_ncs_time => "#<GlobeGlitter 40639cd25341.02.00.00.e0.4c.18.00.69>" irb> ::GlobeGlitter::from_ncs_time.to_time => 1988-12-21 14:52:02 UTC irb> ::GlobeGlitter::from_aegis_time => "#<GlobeGlitter 00000000-0000-0000-4814-17c8b0080069>" (Proper AEGIS `#to_str` not implemented yet lol)
- kyzer-davis 2y ago> Only no known details if the only document you're reading is the notoriously poorly-specified RFC To be fair, in RFC9562 I did cite two documents that are UUID Version 2 specifications. But RFC4122 was too cryptic for my taste. As for the historical UUID types specified by the 0-7 Variant space: We are starting work on an informational RFC that will help folks understand those. See https://github.com/yocto/draft-yocto-uuid https://github.com/yocto/draft-yocto-uuid if you want to add to some discussions and/or review text. We are still a bit early into the stages but I hope to have some progress soon.
- dlgeek 2y ago> UUID Version 2 (v2) is reserved for security IDs with no known details. I found the details in about 2 minutes: Click the link in the article to take me to the section of RFC 9562 that says it's defined as part of DCE, click the first link in that paragraph to go to the spec, ctrl-f "UUID", then jump to appendix A (deceptively named "Universal Unique Identifier") which has all the details. Is it really too much to ask to CLICK YOUR OWN LINKS?
- octernion 2y agohaha I had the precise same thought process and immediately didn't finish the article since they didn't have much attention to detail. i enjoyed reading the appendix though as a snapshot of time.
- THBC 2y agoProbably written by a language model
- dan-robertson 2y agoI mean, there’s two things that one might learn about. One is reading a specification for what a v2 uuid is. The other is learning information about the circumstances of a UUID’s generation from the uuid. Obviously the sentence was about the latter, as could be determined from the context as the other versions are described as being generated from specific things. I agree the sentence is a bit unclear, but I don’t think it’s misleading or whatever.
- efilife 2y agoUuid 4 is just a random bytes generator that inserts hyphens in specified places. You don't need to use it, you can just generate random bytes yourself and save on space (unnecessary hyphens, version info and so on)
- sweca 2y agoTrue but the appeal for most developers is it's simple to implement. Virtually every language has a UUID library that works in one line or code. Like in Go, it's just uuid.New().String() vs using crypto/rand to read random data, convert it into Base64 of hex... which will take more lines and effort.
- lopkeny12ko 2y agoThis is an unfair argument. Which standard library in Go gives you uuid.New().String()? Anyone can publish a third party library that condenses reading random data, creating an identifier from it, and rendering it as a string into a single line of code API.
- Lammy 2y agoUUIDs are 128-bit numbers and the hyphenated-string representation is only one of many ways to represent that number, sort of like how an IPv4 address is a 32-bit number of which the “dotted-quad” is only one representation. If you are thinking of UUID as a string format then your most fundamental concept of UUID is flawed. Even if you do just want a random identifier (not really the original point of UUID but has become their most popular form) I still think it's cool how random UUIDs have a little flag bit to tell you that it's intended to be random. Useful when one runs across a lone identifier with zero context.
- deleted 2y ago[deleted]
- pajeets 2y agois there something shorter than UUID i hate how long it is something like youtube URLs but guaranteed to be without duplicates
- asperous 2y agoOne advantage of uuids is they can be generated on several distributed systems without having to check with each other that they are unique. Only long ids make this reliable. Youtube ids are random and short, but youtube has to check they are unique when generating them. Maybe one way is to split up a random assignment space and assign to each distributed node, but that would be more complex.
- imron 2y agoAnd then there’s uuid5 which you can use to generate identical unique identifiers across multiple systems without having to check on each other. Very very useful to have in some circumstances.
- wtetzner 2y agoEven UUIDs are not guaranteed to not have duplicates. It's just extremely unlikely, largely due to their length.
- bigiain 2y agoDifferent use cases have differing requirements for uniqueness though. A lot of stuff doesn't need "you'd have to generate 1 billion v4 UUIDs per second for 85 years to have a 50% chance of a single collision." sort of guarantee. You might think your "Uber, but for short term giraffe rental" startup needs that sort of guarantee in the investor demo prototype, but it doesn't. Just use an auto increment in Postgres or MySQL (or an integer primary key column in SQLite). If you fool those investors into pouring the money pipe over you, your first real technical/senior engineer hire is gonna throw all the code you have running on your laptop away anyway. Maybe someone at Meta sat down once and figured "we have 4 billion users, each of who on average has 3 cats that they take a dozen picture of every day, so about 150 billion cat pictures per day. So if we name them using uuids we're still good for almost 50 thousand years before we have a 50% chance of displaying a pic of Fluffy when we should have displayed a pic of Mr Whiskers". Then they promptly ignored the problem (or fixed Zack's code that was using php's hash("md2", $query['catname']) ).
- amarcheschi 2y agoI'm failing at understanding what is the purpose of having uuid2. I didn't even know that more type existed till now. I had only encountered uuid2 when asking xandr to remove my personal data from its database. (discussion about xandr being asked to be investigated in Europe by noyb here https://news.ycombinator.com/item?id=40913915 https://news.ycombinator.com/item?id=40913915) By reading the Wikipedia page I'm failing at understanding why we invented something called universally unique identifier and have different types of it, some of which can be traced back to the original pc. Is it because mixing some Mac codes increase the chance of the uuid2 being randomic or does it have a different reason? For privacy reason, could we just not have a very long identifier with many different chars to choose from so that we have so many combinations that we're almost guaranteed we're using non duplicated uuids?
- Lammy 2y agoTheir original purpose was to identify messages in Apollo's distributed computing architecture. UID and later UUIDs were a reversible way to mark an intersection point between two dimensions. Any two machines would generate the same UID/UUID for the same two inputs, and a recipient of an identified message could reverse the identifier back into the original components. They were designed as labels for ephemeral messages so the two dimensions were time and hardware ID (originally Apollo serial number, later Ethernet hwaddress etc). I think a lot of the confusion can be traced to the very earliest AEGIS implementation where the Apollo engineers started using “canned” (their term, i.e. static or well-known) UIDs to identify filesystems. Over time the popular usage of UUID fully shifted from ephemeral identifiers where duplicates were intentional toward canned identifiers where duplicates were unwanted and the two dimensions were random-and-also-random. The history gets even more complicated because Microsoft hired one of the top Apollo guys to do MSRPC for Windows NT, so there is also “GUID” which differs from UUID in the layout of the fields and is not mixed-endian despite what a lot of sources will tell you. In addition to ephemeral RPC message-identifying GUIDs Microsoft are also in love with canned GUIDs for identifying COM classes, media codecs, and almost anything else that would ever need a well-known identifier. See https://gix.github.io/media-types/ https://gix.github.io/media-types/ for example. Apologies for linking my own repo twice in the same comment section but I started (and need to get back to) compiling the history of all this in the README of my UUID library. Apollo started in 1980 and the Leach/Salz UUID RFC draft didn't happen until 1998 so there is a huge amount unsaid by the modern standards: https://github.com/okeeblow/DistorteD/blob/NEW%E2%80%85SENSATION/Globe%20Glitter/README.md#history https://github.com/okeeblow/DistorteD/blob/NEW%E2%80%85SENSA...
- BohuTANG 2y agoUUID v7's timestamp is a game-changer for Databend. We're using it to quickly locate metadata files on AWS S3 by timestamp, making operations like vacuuming much faster. PR: https://github.com/datafuselabs/databend/pull/16049 https://github.com/datafuselabs/databend/pull/16049
- ElFitz 2y agoIt’s also great for key-value store like DynamoDB. Much cleaner than compound keys prefixed with a timestamp or iso date (or worse: wasting a secondary index on timestamps).
- dan-robertson 2y agoInteresting. I don’t think I’d ever really consider using uuids to look something up by time (I’d probably want an actual timestamp field instead). To me, the big advantage of the uuids being sorted by time is that You get better locality – inserting new entries into an index is probably a mostly-append operation which could be cheaper than a random insert (but may have more contention too; worth considering a hybrid with some random bits before the timestamp to have more sorted ‘shards’) and in many places reads tend to be for more recent data so it can be nice to have the more recent data in one place (and eg pulled into more caches).
- benreesman 2y agoTLDR: UUIDv4 is one of several reasonable choices, but if it ends up mattering: you’re rich and famous and can do arbitrary fixes. Not getting pwned by incrementing href attacks is good, past that, web scale bro.
- deleted 2y ago[deleted]