19 ms·

You Don't Need UUID

129 points by popcalc 3y ago

hot_gril 3y ago
When it comes to random identifiers, the advantage of uuid4 is mainly that it's a standard thing that everyone understands well, and finding a lib that generates one securely won't require any thought. It's never the ideal solution, but it's often good enough.
- ddoolin 3y ago
  Agreed - not to mention the specification of new ones with different features keeps evolving.
  - hot_gril 3y ago
    Yeah I'm not a fan of the non-random UUID specs causing confusion. Seems like they mostly get used accidentally by someone who actually needed uuid4. "uuid5 is newer and better, right?"
- koromak 3y ago
  I would love to use short UUIDs, but every part of my stack can get a long one with a single, trusted import. I agree. If there was a shorter standard, I would use that for most cases.
iaaan 3y ago
The crux of this argument seems to be that UUIDs are too long? Which I disagree with. I can't memorize them, no, and it would be cumbersome to try to say one aloud, but these aren't situations I've ever found myself in. Does it make the URL in the URL bar longer? Yeah, but does that matter?
- hot_gril 3y ago
  It matters aesthetically. A huge site like Amazon might care about that, otherwise it's not a high priority.
  - eternityforest 3y ago
    It somewhat matters aesthetically, but mostly people are going to ignore long random strings. UUIDs are at least visually kind of white noise, companies have no problem+with+links?that=look&like=this, so why do they care about UUIDs?
    - skybrian 3y ago
      Some of us will strip off the crud to get the base URL, to avoid being tracked.
    - bandergirl 3y ago
      apple.com/iphone You can't beat that. Or would you rather receive texts that include the following? https://www.amazon.com/dp/B09V3HZ8B5/?pd_rd_w=DAycy&content-id=amzn1.sym.04e31e32-5e01-4048-8b59-a557891d595f:amzn1.sym.04e31e32-5e01-4048-8b59-a557891d595f&pf_rd_p=04e31e32-5e01-4048-8b59-a557891d595f&pf_rd_r=M1KXM468KNZJV189JBE7&pd_rd_wg=3O1uC&pd_rd_r=e51c1edc-213f-4c6d-b081-e23afa6bf4ab&ref_=vn_s_iwp&qid=1694453553 > companies have no problem+with+links?that=look&like=this Let's not match the lowest common denominator, shall we?
      - 5e92cb50239222b 3y ago
        You only really need this part: https://amazon.com/dp/B09V3HZ8B5 https://amazon.com/dp/B09V3HZ8B5
        bandergirl 3y ago
        Yes but that's irrelevant, I used a real URL that you will receive in your texts. It's real life, it happens and it sucks. Let's not normalize it.
        sieabahlpark 3y ago
        [dead]
        3y ago
- richbell 3y ago
  > Does it make the URL in the URL bar longer? Yeah, but does that matter? Yes. That's one of the author's arguments. ) A simple ID like 3c6n63N is more than enough to represent any product while keeping it readable and making communication easier. A UUID alternative like a73ba12d-1d8b-2516-3aee-4b15e563a835 is just wasteful from an user’s perspective.
  - appletrotter 3y ago
    Maybe it’s one if the author’s arguments, but that doesn’t mean the author is right.
    - richbell 3y ago
      Sure — it's just a weird question to pose when the author specifically provides their opinion on it. It'd be different to say "I don't think it matters".
      - Petersipoi 3y ago
        You're just nitpicking. "Does it matter" is a very reasonable response to someone expressing their opinion.
        richbell 3y ago
        It's not nitpicking. If I say "x matters and here's why" and you respond "yeah but does x matter?", you're either not paying attention or being rude and dismissive.
  - NBJack 3y ago
    There is the rub. Readable for what purpose? Aesthetics? And why? It is meaningless in their example. And what do they mean with communication? Reading it aloud? If the URL is too long to stick in a tweet or message, there are plenty of URL shorteners out there.
    - 3y ago
- NBJack 3y ago
  Agreed. Decorative URL slugs have been successfully used by everything from blogs to Amazon itself for over a decade. If anything, that should be the focus here. Short URLs aren't much of a help, especially when short is relative on something like a mobile device.
- Alupis 3y ago
  > Does it make the URL in the URL bar longer? Yeah, but does that matter? No, it doesn't. If you're using a central system to generate id's (UUID's or other), then you are indeed missing a lot of the benefits of UUID's and might as well use some other ID scheme. The benefits of UUID are many systems can generate them, and your database can eventually accept them, all with a very high guarantee of uniqueness/no-collisions. Not to mention the other nice bits like non-sequential and non-enumerability. Most modern databases have UUID/Binary types/functions to allow for more efficient storage/handling anyway.
  - emodendroket 3y ago
    That’s an excellent point; that can end up being a bottleneck across many systems.
    - refulgentis 3y ago
      What can?
      - marginalia_nu 3y ago
        Unique ID generation.
      - emodendroket 3y ago
        A service responsible for handing out UIDs.
  - crooked-v 3y ago
    For a simple practical example, it's awfully useful for a lot of systems to be able to generate UUIDs in an app while offline (think document creation, etc) and reconcile things later by shooting the blob of content off to the server, while knowing that you only need to handle the dance of getting IDs and updating stuff locally again in your error handling case.
- croes 3y ago
  >Does it make the URL in the URL bar longer? Yeah, but does that matter? Nowadays many links are much longer because of referrer information and tracking parameters. So the UUID doesn't add much.
  - dspillett 3y ago
    They might if several sets of tracking information end up there, because people are blocking the cookies they might otherwise be in for instance, and each set of tracking info has an identifier or few. The length of UUIDs will multiply up quite quickly there. Not that people will care. Once a URL is longer than a few words plus a code most users won't even look, even if they bother that far. You'll be in danger of starting to hit some browser/server/proxy limits though. Not that I mind that sort of information being lost, I'm happy for the incessant tracking of everyone's everything these days needs to go to hell.
  - hot_gril 3y ago
    I noticed a recent change on YouTube where the "share" button now includes extra tracking stuff, making it longer than the actual URL in the address bar. Which is even funnier considering that they bought the youtu.be domain just to make shared URLs shorter. Address bar: https://www.youtube.com/watch?v=SHWhKRKlG98 https://www.youtube.com/watch?v=SHWhKRKlG98 Share button: https://youtu.be/SHWhKRKlG98?si=gwQHHze3UnjTobJg https://youtu.be/SHWhKRKlG98?si=gwQHHze3UnjTobJg
- i_am_jl 3y ago
  >Yeah, but does that matter? I'm really reaching here, but an ID that omits special characters is easier to extract from a url, since it can be double-clicked in a URL bar to select it, whereas a GUID with hyphens forces the user to select the beginning and the end of the string. Realistically I think this matters to developers more than users (I hope your application doesn't force users to interact with GUIDs in the address bar) but that's the only "waste" I can identify. EDIT: Someone elsewhere noted that they can base64 encode their GUIDs to achieve this and I'm seething that I'd never thought to do that
  - noisem4ker 3y ago
    To select an UUID, double click the first segment and drag the selection till the last segment before releasing. No need to aim for the exact beginning and end characters. (Maybe that's what you meant, amd yes, it's still clunkier than simply double clicking, but not much)
  - dheera 3y ago
    > easier to extract from a url, since it can be double-clicked Maybe the double-click logic needs to be less stupid then. Or have it progressive, e.g. double-click = alphanumeric, triple-click = stop at symbols, quadruple-click = stop at slashes only, quintiple-click = the whole URL. Use the device microphone to listen for swear words and adjust algorithm with machine learning accordingly
  - tombert 3y ago
    As you said in your edit, you can of course base64 the GUID, but also don't most GUID libraries have an option to just omit the hyphens? I know .NET allows you to do that, and even correctly parse GUIDs that don't have any hyphens. I usually did that in .NET simply because I thought it was prettier without the hyphens.
  - tzot 3y ago
    > I'm really reaching here, but an ID that omits special characters is easier to extract from a url, since it can be double-clicked in a URL bar to select it, whereas a GUID with hyphens forces the user to select the beginning and the end of the string. The is-hyphen-part-of-a-word issue depends on the text control (browser, UI widget library, …) / terminal you use, which is not universal. In any case, the hyphens/dashes are not required part of the representation, and one can use the “dashless” representation (exactly 32 hex digits) in URLs and in-code without any issues whatsoever, as far as I know; for example, Postgres accepts either “dashful” or “dashless” UUIDs the same.
- thaurelia 3y ago
  > Yeah, but does that matter? It might, e.g. with embedded systems. I can neither confirm nor deny that we once BOFed our own devices because we assumed that 1200 bytes was a big enough buffer to hold a URL.
- AdamH12113 3y ago
  > Does it make the URL in the URL bar longer? Yeah, but does that matter? I appreciate shorter URLs any time I copy and paste them, which always involves looking at them and sometimes involves scrolling to the end to remove tracking- and search-related fluff. 128 bits is an absurd amount for a unique ID within a single system. Even 48 bits is very, very large -- enough to provide a unique ID (MAC address) to every Ethernet device made for something like 100 years. The purpose of a UUID is to provide a negligible probability of collisions between randomly-generated IDs in a global space forever. Almost no applications require that, so most of the time using a UUID is just needlessly taking up space.
  - siva7 3y ago
    That’s not the only major purpose for a UUID and even if it was, it still would be flat out wrong to claim that almost no application requires that.
    - AdamH12113 3y ago
      Maybe I'm missing something, then? I'm having trouble thinking of applications where it seems like a good idea to generate a permanent ID number completely randomly at all, much less applications that need to let the whole world do so. I can see how Windows device drivers have that requirement, but the point of the article is that even a massive site like YouTube (which doesn't want videos to be randomly discoverable via URL manipulation) does not need anywhere close to 128 bits. What are these applications you speak of?
    - paulddraper 3y ago
      The purpose of a universally unique identifier was indeed to be universally unique.
  - hot_gril 3y ago
    Basically, you want UUID's 122 bits (128 minus 6 wasted on variant + version) if you can't afford collisions. If you have some DB enforcing unique IDs at creation time, you can manage with a lot fewer bits if you want shorter URLs or something.
- paulddraper 3y ago
  > these aren't situations I've ever found myself in You've never needed to give an order number, member ID, account ID, invoice number, etc?
emodendroket 3y ago
> A simple ID like 3c6n63N is more than enough to represent any product while keeping it readable and making communication easier. A UUID alternative like a73ba12d-1d8b-2516-3aee-4b15e563a835 is just wasteful from a user’s perspective. I would challenge the premise we appear to be starting from, that the average end user cares to be dealing with any random string of numbers and digits. GUIDs work well, they’re implemented everywhere, and you won’t find out long after you go into production you made some mistake that is going to make it so you have to migrate away from them.
- czechdeveloper 3y ago
  Plus I could easily represent UUID in base64, such as JlEt5BSYe0enwB7nxl5V6g, which makes it shorter if I need that.
  - masklinn 3y ago
    Maybe use a smaller alphabet though, base64’s can be quite hard to read / spell out / reproduce as it includes confusing pairs. While somewhat less dense, base58 or rfc 4648 base32 mitigate these issues.
    - Olreich 3y ago
      And for UUIDs, Base58 and Base64 both require 22 characters, so no density loss! Even Base32 only takes you up to 26 characters.
    - ianburrell 3y ago
      Base32 or Base36 have the advantage of using single case which is easier to read out. And no symbols.
      - stevesimmons 3y ago
        The official Base32 doesn't work for encoding UUID7, which is time-ordered, because its symbols for 0-31 are not in ascending ASCII order. My own preferred format for UUID7 is one I call "id25". It's really Base35 because alphabets of 35 and 36 characters both need 25 characters to represent the 128 bits in a UUID. So I can start with Base36 and take out one of the next most ambiguous character pairs. The result looks something like '0pydgw5pifvapk5zyhmpso5tx'. The two other advantages of using this id25 format rather than UUID7 are: - The id25 format is quite distinct from UUID4. So if you have UUID7s being generated distributedly (rather than centrally) and the UUID7 time-ordering feature is important, it's nice to have a format that a trivial "if '-' in uuid_string..." check will spot. - Because there are no hyphens or symbols, the whole id25 uuid can be selected in web page with a double click. Whereas a uuid will need a mouse movement to get all 5 parts.
ddoolin 3y ago
Sure, you don't need them, but why re-invent the wheel? At least UUIDs are not some one-off function in a helpers file that may need to be amended many times before you get it right, but are backed by a specification, are time-sortable, etc.
- jaywalk 3y ago
  Not all UUIDs are time-sortable.
mattbeck 3y ago
I mean, don't use UUID for user-facing things sure, but that doesn't mean it's not useful in of itself. That's a terrible way to use it is all.
ars 3y ago
Use sequential IDs, but run them through a maximal linear feedback shift register. They will be in a deterministic order, but will appear semi-random to the end user. For things like product IDs or user IDs, etc you don't actually need them to be random. But perhaps you don't want them to simply start counting sequentially.
no_wizard 3y ago
Isn't auto generating UUIDs "fast enough" in most situations that its negligible? In particular databases (where UUIDs taking space was a big concern) they have largely switched to a packed binary format that makes the size of UUIDs over time a non issue for all practical purposes.
- hot_gril 3y ago
  UUIDs are packed in DBs. But they tend to get misused as PKs. In common relational DBs, your PK for every table should probably be bigserial.
  - siva7 3y ago
    No, that shouldn’t be the criteria (common relational db) when deciding to use bigserial v UUID but the overall architecture and requirements of your system.
    - hot_gril 3y ago
      Whatever your overall requirements are, if you've already decided on a relational DB to solve them, you probably want bigserial PKs in said DB. The DBMS is optimized for that (as I've noticed in Postgres), and you don't gain anything by using UUIDs instead there. UUID in a secondary col or elsewhere in your system, that's fine. If you have to look up by UUID from outside and have no foreign keys, UUID PKs can be faster. But that's the classic use case for a non-relational DB.
jackblemming 3y ago
Doesn’t UUID have mechanisms to prevent collisions even if you start two processes simultaneously with the same initial seed? There are human readable tweaks you can make to them that preserve all of the good properties without limiting the pool much though.
- WirelessGigabit 3y ago
  v4 is. v1 relied on the MAC address and the time. https://devblogs.microsoft.com/oldnewthing/20040211-00/?p=40663 https://devblogs.microsoft.com/oldnewthing/20040211-00/?p=40...
swader999 3y ago
There are so many nuances and edge cases not mentioned in the article. The author needs to do several projects in real life before making global pronouncements like this.
- monero-xmr 3y ago
  The lack of humbleness and respect for devs who have came before you and solved all these problems is coming through strong. But I myself was that way 15 years ago. It takes a lot of hard lessons to realize everything new is old and we truly stand on the shoulders of 7000 giants all standing on each others’ shoulders.
  - jholman 3y ago
    That's how I feel about UUIDs themselves, and the people who advocate in favour of them. Interesting duality.
    - afgrant 3y ago
      It’s just a 128-bit value that’s hexadecimal encoded. What can be controversial about that?
  - RockRobotRock 3y ago
    It's good to get a fresh perspective sometimes
  - henvic 3y ago
    Did you even read beyond the title? To say that because of this choice of title, I lack humbleness or respect for peers who came before me is despicable. My title is too presumptive. Yet, the reality for almost everyone who doesn't interface with other systems that already require UUID is that. The brief article gives some points – backed by trustworthy sources – and shows some alternatives without further ado. I didn't spend days writing it. I could cover many more scenarios but didn't invest my time in it. So what?
dgb23 3y ago
An issue that is not solved by either UUIDv4 or the proposed solution (random base58 strings) is indexing performance. Both of those solutions typically make it hard for a DB if you write new entries, assuming you have an index on the ID. In addition it might be more calming to actually be sure that a particular ID is not in use without doing a round-trip. Is it practical to pre-allocate empty entries and reserve a set of them?
- afgrant 3y ago
  Are there modern databases that can’t readily index on a 128-bit value?
  - dvlsg 3y ago
    I don't think it's a matter of not being able to index on it, but that mostly random UUIDs (like v4) can lead to some interesting index fragmentation you have to stay on top of somehow.
  - konschubert 3y ago
    If you use sequential ids, the IDs most in use are the higher ones. So your index is “hot at the top” and you can keep that part in memory. This is only a problem with extremely big indexes.
  - safetytrick 3y ago
    Random ids require random inserts and that is bad for performance. It isn't a deal-breaker in most situations but it is a real cost that you do pay.
  - dgb23 3y ago
    You already got multiple answers explaining the performance issue. Now in many, typical applications you'd have to scale up quite significantly before this becomes a problem. But if your application requires you to store small, new entries very quickly, you'll start to notice this even with moderate scale. Disk persistence is often the bottleneck already, and this might make it worse.
  - 3y ago
mccrory 3y ago
Best part is watching the video and seeing the guy dance at the end over doing it in a single take.
srameshc 3y ago
Read this as I was about to use uuid for id :/
- spion 3y ago
  I would use it anyway. This article misses some subtleties that make uuids really great, as well as some real points against uuids (impact of v4 uuids on index access and uuid v7 as a solution to that)
nwah1 3y ago
I recently found ULID. I like its simplicity. It is sequential. It has a very low probability of collision. And it is of a more reasonable length. Because it encodes the time, theoretically you could use it to grab the CreatedDate of a record without a need for another field. https://github.com/ulid/spec https://github.com/ulid/spec
- hot_gril 3y ago
  Sequential isn't always simple. If your system is distributed in the slightest way (even just multiple threads), it's a bit of a luxury to find sequential guarantees.
  - kiitos 3y ago
    ULIDs can get you rough ordering guarantees even in distributed systems. Precision, of course, requires synchronization, which is usually a non-starter.
lifechoseme123 3y ago
You don't need to tell me I don't need UUID ...because I am going to keep using uuid
kolanos 3y ago
You don't "need" them, but they're almost universally understood and supported which makes them super convenient and a good starting point for many use cases. The point in which the "wasteful" nature of a UUUID really bites you is so far down the road that it is almost not even worth thinking about until you get there. Chances are you won't be dealing in petabytes at all and the 64 bit savings you are fretting over doesn't really matter.
kiitos 3y ago
A UUID is 128 bits, or 16 bytes. This code suggests IDs which are 11 bytes. That's not really comparable. A UUID can also be encoded in any form, it doesn't need to be represented as the dashed string notation which is common, you can just as easily use the base58 alphabet suggested in the post. But the code in the post doesn't encode to base58 correctly. You need to map 58 bits of the input, sequentially, to one character in the alphabet as output. You can't just mod each byte of the input by the alphabet length and use the corresponding alphabet element.
- cbarrick 3y ago
  The base58 code in the post is wrong, but your version isn't right either. Since the alphabet is 58 characters, each character contains log2(58) bits. Since that's not an integer, the encoding process is a bit more complex than just mapping bits to characters in a table. https://digitalbazaar.github.io/base58-spec/#encode https://digitalbazaar.github.io/base58-spec/#encode
  - henvic 3y ago
    Hi. I'm the author of the post here. I agree lines 12-14 doesn't do a proper encoding, but that's not what I was after. Would you still say it's still wrong if you consider that my function is not really encoding the string per se, but using rand.Read to generate entropy for what I want to be the final output (random string with the base58 alphabet)?
    - kiitos 3y ago
      Your IDs are []byte of len=11. Those bytes can be represented in many ways. You can represent them as hex strings via encoding/hex.EncodeToString(id), or base64 strings via encoding/base64.StdEncoding.EncodeToString(id), or base32 strings via encoding/base32.StdEncoding.EncodeToString(id), or etc. Looks like the most used base58 package is https://pkg.go.dev/github.com/btcsuite/btcutil/base58 https://pkg.go.dev/github.com/btcsuite/btcutil/base58, but looking at the implementation [0] I'm not impressed, there's definitely a much better approach. [0] https://github.com/btcsuite/btcutil/blob/v1.0.2/base58/base58.go#L49 https://github.com/btcsuite/btcutil/blob/v1.0.2/base58/base5... But how you encode 11 bytes of data is kind of orthogonal to the important thing, which is that you have 11 bytes of data. Those bytes should be always be stored in memory (in your application, or in your DB, or wherever) as the actual 11 bytes of the ID, and not as a base58 or base64 or JSON or whatever other kind of string that can be decoded to the actual 11 bytes they represent. Likewise, a UUID shouldn't be stored as a string like "64d3f2e0-a4dc-48d3-98ad-7f09eb3b082f", that's a specific encoding of the actual 16 UUID bytes, you should store, process, etc. those bytes directly.
erlendellingsen 3y ago
I usually go for Nano Id for new projects https://github.com/ai/nanoid https://github.com/ai/nanoid
- eagleinparadise 3y ago
  I generate uuidv7 as PK and have a publicId "slug" with a 13 character nanoId to display in the URL not sure if this is duplicative but down the road i figure I can navigate whichever direction i need to go in
- phlogisticfugu 3y ago
  +1 for nanoid, which matches what I used to implement myself as a lib function in new projects
BiteCode_dev 3y ago
If your users needs to communicate an ID orally to you, you already failed in numerous ways.
alwaysbeconsing 3y ago
> Version 4 is completely randomly generated (hence, it has more entropy) and is what most web systems seem to use. It has 16^32 = 2^128 bits that guarantee uniqueness and has an insignificant risk of collision. Small correction here but it is not 128 random bits entirely: 6 bits are reserved for the version marker.
tcoff91 3y ago
UUIDs are great for when you have a distributed system where connectivity between the nodes is intermittent, and every node needs to be able to create new entities without ever having ID collisions. For instance, UUIDs allow mobile apps to create entities while offline.
rdtsc 3y ago
> This solution uses the human-readable base58 encoding scheme. We could encode the 128 bit UUID integer in base58 as well if needed. Its textual representation won't conform to the standard but we'd get the same number of bits. Which would be 2^122 or 2^121 bits of uniqueness not 2^128 if it's a proper UUID. 11base58 character certainly doesn't have the 2^122 bits. So we could decided separately if we could either reduce the number of bits needed and/or use a different encoding. > If you click and buy any of these from Amazon after visiting the links above, I might get a commission from their Affiliate program. :-)
nodesocket 3y ago
I’m not sure if this is accurate any longer, but it was bad practice to use UUID’s as PK’s in PostgreSQL for example due to indexing and performance. Is this still true? It’s really nice in PostgreSQL to just do: id UUID DEFAULT gen_random_uuid()
xcdzvyn 3y ago
> As Tom Scott shows in his video, 11 base58-encoded characters are enough for YouTube to serve content even when considering that private videos should be undiscoverable. Nitpick: Google isn't concerned about the discoverability of private videos; those can only be viewed when granted access. You're thinking of unlisted videos.
- henvic 3y ago
  Fixed! Thanks!
stayfrosty420 3y ago
uuids are ubiquitous and easy and implemented in many languages in the standard library - therefore a dev can get near-guaranteed uniqueness without thinking too hard about it. Until another standard similar to what the article is suggesting becomes widely implemented in standard libraries then uuid isn't going anywhere, although in principle I agree with many of the arguments presented.
pphysch 3y ago
You Don't Need UUID, But You Probably Want It: - Don't invent your own ID datatype (especially 11 byte one), this is almost guaranteed to cause dangerous bugs, because each integration will have to carefully implement/hack it. - Use UUID (preferably the new v7). 128-bit UUID is implemented, for you, pretty much everywhere. - Serial integers still work too, but you should choose them consciously to fit the data model. - Implement "natural keys" if you want pretty/memorable/Cool URLs. Never use non-standard PKs to store custom semantic data, because inevitably you will get garbage PKs that need to be fixed, and migrating a PK value is extremely risky.
afgrant 3y ago
It’s a sales pitch for base58 that I’m not buying at all. According to the code segment, they include numeral “1” but exclude lowercase “l”? 0-9 and a-f are unambiguous and widely understood. They’re more “human readable”.
saulpw 3y ago
I completely agree. UUIDs are almost always evidence of overengineering, especially when user-facing. 64-bits is "enough" of an address space for almost any purpose, even global. And while it seems like speaking an ID over the phone or having to scan it manually is something you'd never have to do, in practice it happens all the time. Cut-and-paste is not always an option on all platforms.
- afgrant 3y ago
  128 bits versus 64 bits is a deal breaker?
  - saulpw 3y ago
    Yes. Even 64 bits is too many, really: at 6 bits per human-recognizable token, it's more than 10 tokens, which is outside the range of what almost any human can keep in their working memory. You can't even hold it in your head long enough to type it in a different window. 128 bits is completely beyond that; when confronted with 128-bits like a UUID, people just give up. Seriously, try actually typing in a UUID sometime. Even with the grouping it's incredibly difficult. Between 64-bits and 128-bits you discover that you absolutely need an automated way to transmit the data, and then how do you get it to that airgapped computer?
    - WA 3y ago
      The more important requirement is that the id should not be guessable and not easily be brute-forced (example: YouTube private video listings). Now, what's the minimum required entropy to make it unguessable but also make it short enough to have people tell the id over the phone or something?
insanitybit 3y ago
I think there's a better "You Don't Need UUID", which is that you probably aren't going to generate so many integers per second that you need to do so in a way that's perfectly collision free. For example, you can update a postgres integer 1000s of times per second, sequentially, and 100s of 1000s of times per second if you allow for gaps/ unordered sequencing. The reason to choose a UUID is because it doesn't require a db. Totally fair. But you probably already have a db lying around if you're generating UUIDs to begin with? Or because you need an unguessable token, which a UUID is great for. But if you have a postgres db around somewhere consider just having a counters table and using that. It'll be fast enough for almost anyone, it'll return smaller tokens, and it'll return tokens that are sequentially allocated (in range. These are nice properties to have!
- SoftTalker 3y ago
  Database "sequence" types that are used to create integer IDs often no not make any guarantees about being gap-free. This can be a problem for people who don't realize this, and use them for something that is legally required to be gap-free, such as invoice numbers in some jurisdictions. Unguessability is a nice layer to have in your security strategy, especially if you're passing IDs to and from a web page, but if that's all you rely on you're not doing enough.
  - insanitybit 3y ago
    > Database "sequence" types that are used to create integer IDs often no not make any guarantees about being gap-free. Yep, the docs from Postgres are very clear that `serial` has gaps. But you can also just create a counter table and manually manage the sequence if you have strict serial requirements. Of course, if you have serial requirements a UUID would already not be viable.
siva7 3y ago
> A UUID alternative like a73ba12d-1d8b-2516-3aee-4b15e563a835 is just wasteful from an user’s perspective. The argumentation in this article is pretty poor from my experience. A UUID isn’t meant to be handled by the non-technical end user. The end user usually doesn’t and shouldn’t care about the URL. I can assure there are bigger architectural problems in your design if your user has to care about accessible internal ids.
- olalonde 3y ago
  Does this help see what's wrong with UUIDs in URLs? A GitHub issue URL in a parallel universe: https://github.com/a73ba12d-1d8b-2516-3aee-4b15e563a835/e16957bf-7d7f-41f1-97f4-98c93a6c3540/issues/4e4e9133-bcdb-45b9-8dfe-9e951846e3c6
  - siva7 3y ago
    I see bad url design for other reasons but nothing wrong with UUIDs, so no it doesn’t help me see the argument.
    - olalonde 3y ago
      What do you mean by bad url design? Because it's using UUIDs? That was my point.
      - JaredCliff 3y ago
        Not OP but it's obvious to me that by bad url design, they mean that you could have used usernames and issue numbers in the URL but you chose to go with random IDs where there was no need for it.
  - j-pb 3y ago
    A fairer comparison would be that in that universe the url would be: https://github.com/4e4e9133-bcdb-45b9-8dfe-9e951846e3c6 or even urn:uuid:4e4e9133-bcdb-45b9-8dfe-9e951846e3c6 with the right infrastructure.
wnolens 3y ago
My experience: If you're storing data in a database keyed by a GUID, you get a nicer random distribution in an index, avoiding hot partitions when your workload is seemingly random.
Exuma 3y ago
YoU dOnT nEeD _____ AKA I will make broad sweeping stupid headlines
Waterluvian 3y ago
I think is an “engineer thinks harder about UX” kind of problem. UUIDs are fine for what they’re good at. Try to avoid making them user-facing because, yeah, they’re ugly. I also wouldn’t hand 128 bit integers to the user if I could avoid it. Don’t forget: UUIDs are not dash-separated strings. They’re integers. You can render them differently if those dashes are sucky.
Ameo 3y ago
I like UUIDs aesthetically. They're also an instant queue that something is an "entity" and familiar to people for that purpose. Databases and lots of other things also have native support for them which makes them even more appealing to use. Of course, there are plenty of reasons to avoid them like the article discusses. I'd almost certainly not criticize someone for choosing smaller IDs as long as security isn't an issue.
henvic 3y ago
Author here. I posted this because I've witnessed many systems in companies I've worked for where our end-users needed UUID to communicate with it (technical support, customer ID, etc.) in a way that makes communication harder. We could've used another shorter ID scheme, which would be fine. The good thing about UUID is that it's omnipresent. From what I've heard, it's this lengthy (2^32) because it was hard to guarantee uniqueness when it was conceived in the telecom industry. The length is overkill, and per se, that's fine, but the fact that it dampers communication is awful. That all said, since posting this, I've come to terms with accepting that it's part of life ¯\_(ツ)_/¯ P.S. Using a second human-friendly ID to end-users is an alternative adopted by some projects. However, most projects don't bother, and also, most good IDs you might want to share with people would make UUID unnecessary anyways (in practice).
- konschubert 3y ago
  Just use the first N characters of the UUID if you want a shorter version of the UUID. And if you want to be really human friendly, use one of those silly name generators.
- diplocorp 3y ago
  > it's this lengthy (2^32) UUIDs are 128 bits, so it's 2^128 rather than 2^32
- Nullabillity 3y ago
  There is no fundamental tech advance that caa make the birthday paradox irrelevant or obsolete, and there never will be.
  - rwiggum 3y ago
    all of cryptocurrency relies on sha256 the fact that the space is huge, and just ignores collisions. seems to work fine.
    - Nullabillity 3y ago
      SHA-256 has, well, 256 bits of entropy. What I took issue with was the claim that UUID's 128 bits (well, almost, anyway) are overkill with modern advances in randomness.
tombert 3y ago
I don't know that I really agree with the logic here; certainly for some cases having a pretty URL matters (e.g. URL shorteners or even something like YouTube), but if your goal is to minimize URL size, and if you're centrally synchronizing anyway to avoid conflicts, why not just use integers or something? If you use a GMP-style bigint, I can virtually guarantee it'll be faster than any random string concat glue you come up with, and you're never going to run out of integers. I like using GUID specifically because there's usually a built-in implementation everywhere, and because it's so stupidly huge, the likelihood of a conflict is statistically zero, meaning I don't need to bother synchronizing against a server. Yes, it's technically possible I could save a few bytes of bandwidth or something by using some kind of base encoding, and maybe in some kind of embedded case that might matter, but for a vast majority of cases, GUID is absolutely fine, and since it's used everywhere, it's also very thoroughly tested by multi-billion-dollar corporations meaning that I don't have to worry much about any issues. So no, you don't need GUID, you don't need a lot of stuff in the software world. I could poll bytes from /dev/urandom and probably get something that works well enough, I could turn off garbage collection and just pre-allocate all my memory before hand, I could avoid an OS entirely and write straight to the metal, I could do a lot of things, but I don't because I value my time . GUID solves a specific problem pretty well. EDIT: The reason I'm specifying this is because I think the title would be better if it was something like "You (probably) don't need UUID".
0xbadcafebee 3y ago
This is such a HN article.
Glyptodon 3y ago
Is there any existing standardized middle ground between UUIDs, random bits of length n, and serial IDs? I totally buy that UUIDs are unpleasant in URLs in many cases. I can also imagine that in many cases UUIDs for small objects might be a huge % of on the wire payload size. But besides human friendly slugs, which usually have an ID mapping behind the scenes in my experience, it seems like there might be more work than value for startups in many cases.
olalonde 3y ago
I wonder how Stripe generates their IDs. They look quite nice.
- techn00 3y ago
  They use a type safe extension of UUIDv7 Here is a go implementation and explanation: https://github.com/jetpack-io/typeid https://github.com/jetpack-io/typeid
jherskovic 3y ago
It's amusing that the UUIDs are considered 'ugly' while the Amazon and YouTube IDs are not. Beauty is in the eye of the beholder. I don't find UUIDs pretty, at all, but the others are even uglier to my taste. Yes, UUIDs are overkill for most applications. But CPUs and hard drives are, relatively speaking, cheap. Using an existing, battle-tested unique ID library implementation has advantages. The 8 bytes per record you're saving over bigserial is, for most use cases, negligible. 1,000,000 rows? You'll save 8 MB by switching away from UUIDs. Most databases won't be that large. Use a UUID if you want; pretend you'll have Really Big Data some day if it makes you happy. Render it using a special function if the hyphens are too ugly.
pdmccormick 3y ago
Pivoting to the related subject of compact textual representations of _typed_ data, check out CESR[0]. Through some careful choices of code prefixes and payload lengths that won't require Base64 padding, it provides a novel encoding scheme for JSON plus cryptographic signatures (all while enabling true concatenative composability). It is at the heart of the KERI[1][2][3], a decentralized identity management scheme, and ACDC[4], a mechanism for verified credentials. [0] https://trustoverip.github.io/tswg-cesr-specification/draft-ssmith-cesr.html https://trustoverip.github.io/tswg-cesr-specification/draft-... [1] https://keri.one/keri-resources/ https://keri.one/keri-resources/ [2] https://trustoverip.github.io/tswg-keri-specification/draft-ssmith-keri.html https://trustoverip.github.io/tswg-keri-specification/draft-... [3] https://github.com/WebOfTrust/keripy/ https://github.com/WebOfTrust/keripy/ [3] https://trustoverip.github.io/tswg-acdc-specification/draft-ssmith-acdc.html https://trustoverip.github.io/tswg-acdc-specification/draft-...
ekojs 3y ago
IMO, a good middleground is using schemes like TypeID[0], ulid[1], or KSUID[2] that provides a more compact and readable (base32) representation and provides better database locality (K-sortable). [0] https://github.com/jetpack-io/typeid https://github.com/jetpack-io/typeid [1] https://github.com/ulid/spec https://github.com/ulid/spec [2] https://github.com/segmentio/ksuid https://github.com/segmentio/ksuid
- safetytrick 3y ago
  ulids are beautiful little ids without many of the drawbacks and risks of other id systems. They are sufficiently unique for most work (anywhere that you don't need to actually uniquely engineer your ids) and their built in ordering and slick encoding are just icing on the cake.
- hot_gril 3y ago
  As noted on an earlier HN submission about typeid, K-sortable only matters to distributed DBs, and things get tricky there. Spanner for example says to use deliberately unsorted primary keys. I really think when you're picking PKs, you should simply use whatever the DBMS recommends for performance. It's not the PK's job to be typed, sequenced, human-readable, or anything like that; that can be handled by other cols and logging rules. Its job is to be fast in those joins you'll constantly be making against it.
  - sgarland 3y ago
    Sortable primarily keys matters very much for any DB clustering on them, e.g. MySQL/InnoDB and SQL Server. It also matters, albeit to a much lesser extent, to others. Postgres stores tuples in a heap, but the PK is still a B+tree (ish), so an INSERT or UPDATE heavy workload will suffer somewhat.
    - hot_gril 3y ago
      Yeah, you're right. But for those, the typical PK recommendation of bigserial is already time-ordered and mostly packed.
tanepiper 3y ago
Everyone wanted UUIDs and we had IRIs all along...
fuzzythinker 3y ago
I see ulid and nanoid being recommended here. I like cuid2 based on input from its README [0] and discussion [1] [0] https://github.com/paralleldrive/cuid2 https://github.com/paralleldrive/cuid2 [1] https://github.com/paralleldrive/cuid2/issues/7 https://github.com/paralleldrive/cuid2/issues/7
- kiitos 3y ago
  cuid [0] seems to be a random letter, concatenated with a sha3 hash of: - the system time - a process-global singleton counter (incrementing with each ID) - a "salt" derived from some number of bytes taken from an entropy source - a "fingerprint" of some number of bytes taken from the same entropy source this is an awful lot of work to be doing for each ID, to what end? well issue 7 [1] says that > Cuid2 is good when you need something secure, extremely collision resistant, and your system is distributed or decentralized (e.g. you want to be able to create records with ids on the client side), or you are building software that may need to scale horizontally). but I'm not sure how cuid as implemented provides a better answer in any of these dimensions to a plain old UUID maybe I'm just being cynical, I dunno [0] https://github.com/paralleldrive/cuid2/blob/fb07094487ba5ad038a51ef67bcb9c5f525c173b/src/index.js#L97 https://github.com/paralleldrive/cuid2/blob/fb07094487ba5ad0... [1] https://github.com/paralleldrive/cuid2/issues/7 https://github.com/paralleldrive/cuid2/issues/7
konschubert 3y ago
Just use UUID. If you need something user friendly, use the first 6 or 8 characters. If you need to worry about index performance due to enormous index sizes, you probably know it.
rekabis 3y ago
For the web, sequential ID attacks represent a significant problem in terms of a Resource Enumeration Attack. It gets worse when auth isn’t even in play yet (the user isn’t logged in, or doesn’t even have an account yet, or this info is meant to be available outside of auth but to only one specific user), but you need to display only the current item, and not any others. Having a sequential ID of any kind allows the user to trivially hack their way into any other item they wish by just incrementing or decrementing that ID. UUIDs represent a trivially easy way of implementing a non-incrementing ID with a ridiculously unwieldy address space that makes it supremely unrealistic for the vast majority of users to mess with. They’re just going to give up long before the first successful hit.
tommiegannert 3y ago
I'm building an object database. In different places, it uses four different categories of IDs, depending on requirements: 1. Compact sequential is used, obviously, where order matters. It has the drawback of requiring a coordination with a singleton. (This can be sharded/vectorized, of course.) Aside from that, it also leaks the number of objects/transactions, just by looking for the highest number available. Can be varint-encoded very nicely. 2. Compact non-sequential is used where I need a small identifier, but not leak the number of objects. Since it's compact, I must still guarantee uniqueness as in (1). This is currently implemented using a block cipher on top of the compact sequential ID generator. The drawback of this is that the domain may still be in guessable territory, depending on how much is generated. A 64-bit integer filled with 4B only requires 4B guesses to hit a collision. I don't use this much. The key can never be rotated: a key number would eat up precious bits. 3. Sparse random, used where non-guessability is important, aside from not leaking rates/counts. Take a Google Docs sharable link as an example. I doubt YouTube cares about this. This is where something like a 128-bit number like UUID or ULID shines. The space is large enough that uniqueness is assumed, given a decent PRNG. Sure, I try to use the nicest one at any given point (e.g. using a compact sequential instead of non-sequential during debugging.) But fact is that sparse random just tick more boxes. 4) Sparse, human readable. For "vouchers". They are bearer tokens that give requests more powers, e.g. to create an account or act as admin. These should be reasonably human readable, so they can be spoken. They obviously need to be sparse and hard to guess, which requires a trade-off in length. I present them in three ways: base32, english words and QR-code. Pick one; they all do the same. For copy-pasting, base32 might be best (or base58, by all means.) For shouting to a colleague, the sequence of english words might be better. I'd add other languages as needed: it's just a fixed list of words. The nice thing is it can encode the sequence in base-500 or base-1000 without being obnoxious. (The Matrix protocol and others use emoji lists, but it's the same idea. [1]) Finally, if you have a phone in your pocket or camera on your computer, perhaps the QR-code is the easiest way to use the voucher code. [1] Actually, IIRC, Matrix only uses 64 emojis, which feels a bit wasteful.
irq-1 3y ago
> This solution uses the human-readable base58 encoding scheme. How readable is base58 in Arabic or Chinese? I've concluded that only the standard numbers 0123456789, algebra symbols +-/*= and phone symbols #* are universal enough for a global encoding. Any other insights are welcomed!
olalonde 3y ago
In case it helps anyone, I just went down the rabbit hole of calculating the amount of random bits required to avoid collisions. The traditional Birthday Paradox formula gives us a probability but it's not very intuitive to understand what the probability means in terms we are used to as developers, so I tried a different approach: expected time period for a collision to happen. https://colab.research.google.com/drive/1ec4n7Ex9bnkl_c45EUlIPCNtsLKrwaB2?usp=sharing https://colab.research.google.com/drive/1ec4n7Ex9bnkl_c45EUl...
x-complexity 3y ago
No comments on this page about Snowflake IDs. https://en.wikipedia.org/wiki/Snowflake_ID https://en.wikipedia.org/wiki/Snowflake_ID They fit within 64 bits, allow for more than enough processes to handle 10k+ transactions per second, give enough of a timestamp headroom for decades into the future, and where ID generation can be made isolated to each process. They don't work well for anything related to archival work, but you might as well use a regular ID for that anyway, unless you're also actively scraping terabytes of data off of the Internet every second, in which case UUIDv5's good enough for your extreme edge case. ...But at that point you might as well just roll your own 128-bit version of a Snowflake ID.
- M2Ys4U 3y ago
  >...But at that point you might as well just roll your own 128-bit version of a Snowflake ID. Or UUIDv7, which is designed to solve the same problem
RamblingCTO 3y ago
We need them tho. Especially UUIDv4 and UUIDv5. We have a distributed system and we manage permission identities with that. Each item that's supposed to be restricted has either of these ids. UUIDv4 for global things like workspaces and users, UUIDv5 for workspace-specific stuff (the workspace UUIDv4 is given as a namespace for the v5) and it's quite beautiful and prevents clashes. So it really depends on your usage. We still have long ids that are exposed to the client tho.
CiaranMcNulty 3y ago
> If you’re talking about a web system composed of microservice architecture all running on the same datacenter, perhaps sharing the same database We have different ideas what 'microservice architecture' means, I guess. One of the key points of UUIDs is to be able to generate (probably) non-conflicting values without coordination
consoomer 3y ago
I have created a custom unique identifier scheme before. This post doesn't address some of the challenges you come across. 1. When dealing with user friendly IDs, it's often important to make sure there are no ambiguous characters. This is a UX requirement for anyone that may need to read an ID and type it in at any point. For example, this means removing certain confusing characters like "0oIi1l5sS", etc. You end up with a much smaller set of characters. In my case, young children were required to type in teacher codes. Needless to say, I had to limit a lot of possible confusing characters. 2. You will have collisions. How do you handle them (ie, retry N times until you get a valid one)? What happens if you can't get a valid one? This happened in a product of mine. We had ~8 character codes that were human readable and type-able and we... ran out of codes. 3. How can you embed more information into the code such as versioning? It's often useful to prefix a code with some info in the event you need to modify it later (such as expand the character set or length)?
- nick_ 3y ago
  Number two can be addressed with something like this: https://preshing.com/20121224/how-to-generate-a-sequence-of-unique-random-integers/ https://preshing.com/20121224/how-to-generate-a-sequence-of-... You can shrink the range by using a smaller prime number and throwing on out-of-range input. And then you can offset the output to make sure the base-N encoded value is always D digits long.

129 points by popcalc 3y ago

When it comes to random identifiers, the advantage of uuid4 is mainly that it's a standard thing that everyone understands well, and finding a lib that generates one securely won't require any thought. It's never the ideal solution, but it's often good enough.

ddoolin 3y ago
Agreed - not to mention the specification of new ones with different features keeps evolving.
- hot_gril 3y ago
  Yeah I'm not a fan of the non-random UUID specs causing confusion. Seems like they mostly get used accidentally by someone who actually needed uuid4. "uuid5 is newer and better, right?"
koromak 3y ago
I would love to use short UUIDs, but every part of my stack can get a long one with a single, trusted import. I agree. If there was a shorter standard, I would use that for most cases.

iaaan 3y ago

The crux of this argument seems to be that UUIDs are too long? Which I disagree with. I can't memorize them, no, and it would be cumbersome to try to say one aloud, but these aren't situations I've ever found myself in. Does it make the URL in the URL bar longer? Yeah, but does that matter?

hot_gril 3y ago
It matters aesthetically. A huge site like Amazon might care about that, otherwise it's not a high priority.
- eternityforest 3y ago
  It somewhat matters aesthetically, but mostly people are going to ignore long random strings. UUIDs are at least visually kind of white noise, companies have no problem+with+links?that=look&like=this, so why do they care about UUIDs?
  - skybrian 3y ago
    Some of us will strip off the crud to get the base URL, to avoid being tracked.
  - bandergirl 3y ago
    apple.com/iphone You can't beat that. Or would you rather receive texts that include the following? https://www.amazon.com/dp/B09V3HZ8B5/?pd_rd_w=DAycy&content-id=amzn1.sym.04e31e32-5e01-4048-8b59-a557891d595f:amzn1.sym.04e31e32-5e01-4048-8b59-a557891d595f&pf_rd_p=04e31e32-5e01-4048-8b59-a557891d595f&pf_rd_r=M1KXM468KNZJV189JBE7&pd_rd_wg=3O1uC&pd_rd_r=e51c1edc-213f-4c6d-b081-e23afa6bf4ab&ref_=vn_s_iwp&qid=1694453553 > companies have no problem+with+links?that=look&like=this Let's not match the lowest common denominator, shall we?
    - 5e92cb50239222b 3y ago
      You only really need this part: https://amazon.com/dp/B09V3HZ8B5 https://amazon.com/dp/B09V3HZ8B5
      - bandergirl 3y ago
        Yes but that's irrelevant, I used a real URL that you will receive in your texts. It's real life, it happens and it sucks. Let's not normalize it.
        sieabahlpark 3y ago
        [dead]
        3y ago
richbell 3y ago
> Does it make the URL in the URL bar longer? Yeah, but does that matter? Yes. That's one of the author's arguments. ) A simple ID like 3c6n63N is more than enough to represent any product while keeping it readable and making communication easier. A UUID alternative like a73ba12d-1d8b-2516-3aee-4b15e563a835 is just wasteful from an user’s perspective.
- appletrotter 3y ago
  Maybe it’s one if the author’s arguments, but that doesn’t mean the author is right.
  - richbell 3y ago
    Sure — it's just a weird question to pose when the author specifically provides their opinion on it. It'd be different to say "I don't think it matters".
    - Petersipoi 3y ago
      You're just nitpicking. "Does it matter" is a very reasonable response to someone expressing their opinion.
      - richbell 3y ago
        It's not nitpicking. If I say "x matters and here's why" and you respond "yeah but does x matter?", you're either not paying attention or being rude and dismissive.
- NBJack 3y ago
  There is the rub. Readable for what purpose? Aesthetics? And why? It is meaningless in their example. And what do they mean with communication? Reading it aloud? If the URL is too long to stick in a tweet or message, there are plenty of URL shorteners out there.
  - 3y ago
NBJack 3y ago
Agreed. Decorative URL slugs have been successfully used by everything from blogs to Amazon itself for over a decade. If anything, that should be the focus here. Short URLs aren't much of a help, especially when short is relative on something like a mobile device.
Alupis 3y ago
> Does it make the URL in the URL bar longer? Yeah, but does that matter? No, it doesn't. If you're using a central system to generate id's (UUID's or other), then you are indeed missing a lot of the benefits of UUID's and might as well use some other ID scheme. The benefits of UUID are many systems can generate them, and your database can eventually accept them, all with a very high guarantee of uniqueness/no-collisions. Not to mention the other nice bits like non-sequential and non-enumerability. Most modern databases have UUID/Binary types/functions to allow for more efficient storage/handling anyway.
- emodendroket 3y ago
  That’s an excellent point; that can end up being a bottleneck across many systems.
  - refulgentis 3y ago
    What can?
    - marginalia_nu 3y ago
      Unique ID generation.
    - emodendroket 3y ago
      A service responsible for handing out UIDs.
- crooked-v 3y ago
  For a simple practical example, it's awfully useful for a lot of systems to be able to generate UUIDs in an app while offline (think document creation, etc) and reconcile things later by shooting the blob of content off to the server, while knowing that you only need to handle the dance of getting IDs and updating stuff locally again in your error handling case.
croes 3y ago
>Does it make the URL in the URL bar longer? Yeah, but does that matter? Nowadays many links are much longer because of referrer information and tracking parameters. So the UUID doesn't add much.
- dspillett 3y ago
  They might if several sets of tracking information end up there, because people are blocking the cookies they might otherwise be in for instance, and each set of tracking info has an identifier or few. The length of UUIDs will multiply up quite quickly there. Not that people will care. Once a URL is longer than a few words plus a code most users won't even look, even if they bother that far. You'll be in danger of starting to hit some browser/server/proxy limits though. Not that I mind that sort of information being lost, I'm happy for the incessant tracking of everyone's everything these days needs to go to hell.
- hot_gril 3y ago
  I noticed a recent change on YouTube where the "share" button now includes extra tracking stuff, making it longer than the actual URL in the address bar. Which is even funnier considering that they bought the youtu.be domain just to make shared URLs shorter. Address bar: https://www.youtube.com/watch?v=SHWhKRKlG98 https://www.youtube.com/watch?v=SHWhKRKlG98 Share button: https://youtu.be/SHWhKRKlG98?si=gwQHHze3UnjTobJg https://youtu.be/SHWhKRKlG98?si=gwQHHze3UnjTobJg
i_am_jl 3y ago
>Yeah, but does that matter? I'm really reaching here, but an ID that omits special characters is easier to extract from a url, since it can be double-clicked in a URL bar to select it, whereas a GUID with hyphens forces the user to select the beginning and the end of the string. Realistically I think this matters to developers more than users (I hope your application doesn't force users to interact with GUIDs in the address bar) but that's the only "waste" I can identify. EDIT: Someone elsewhere noted that they can base64 encode their GUIDs to achieve this and I'm seething that I'd never thought to do that
- noisem4ker 3y ago
  To select an UUID, double click the first segment and drag the selection till the last segment before releasing. No need to aim for the exact beginning and end characters. (Maybe that's what you meant, amd yes, it's still clunkier than simply double clicking, but not much)
- dheera 3y ago
  > easier to extract from a url, since it can be double-clicked Maybe the double-click logic needs to be less stupid then. Or have it progressive, e.g. double-click = alphanumeric, triple-click = stop at symbols, quadruple-click = stop at slashes only, quintiple-click = the whole URL. Use the device microphone to listen for swear words and adjust algorithm with machine learning accordingly
- tombert 3y ago
  As you said in your edit, you can of course base64 the GUID, but also don't most GUID libraries have an option to just omit the hyphens? I know .NET allows you to do that, and even correctly parse GUIDs that don't have any hyphens. I usually did that in .NET simply because I thought it was prettier without the hyphens.
- tzot 3y ago
  > I'm really reaching here, but an ID that omits special characters is easier to extract from a url, since it can be double-clicked in a URL bar to select it, whereas a GUID with hyphens forces the user to select the beginning and the end of the string. The is-hyphen-part-of-a-word issue depends on the text control (browser, UI widget library, …) / terminal you use, which is not universal. In any case, the hyphens/dashes are not required part of the representation, and one can use the “dashless” representation (exactly 32 hex digits) in URLs and in-code without any issues whatsoever, as far as I know; for example, Postgres accepts either “dashful” or “dashless” UUIDs the same.
thaurelia 3y ago
> Yeah, but does that matter? It might, e.g. with embedded systems. I can neither confirm nor deny that we once BOFed our own devices because we assumed that 1200 bytes was a big enough buffer to hold a URL.
AdamH12113 3y ago
> Does it make the URL in the URL bar longer? Yeah, but does that matter? I appreciate shorter URLs any time I copy and paste them, which always involves looking at them and sometimes involves scrolling to the end to remove tracking- and search-related fluff. 128 bits is an absurd amount for a unique ID within a single system. Even 48 bits is very, very large -- enough to provide a unique ID (MAC address) to every Ethernet device made for something like 100 years. The purpose of a UUID is to provide a negligible probability of collisions between randomly-generated IDs in a global space forever. Almost no applications require that, so most of the time using a UUID is just needlessly taking up space.
- siva7 3y ago
  That’s not the only major purpose for a UUID and even if it was, it still would be flat out wrong to claim that almost no application requires that.
  - AdamH12113 3y ago
    Maybe I'm missing something, then? I'm having trouble thinking of applications where it seems like a good idea to generate a permanent ID number completely randomly at all, much less applications that need to let the whole world do so. I can see how Windows device drivers have that requirement, but the point of the article is that even a massive site like YouTube (which doesn't want videos to be randomly discoverable via URL manipulation) does not need anywhere close to 128 bits. What are these applications you speak of?
  - paulddraper 3y ago
    The purpose of a universally unique identifier was indeed to be universally unique.
- hot_gril 3y ago
  Basically, you want UUID's 122 bits (128 minus 6 wasted on variant + version) if you can't afford collisions. If you have some DB enforcing unique IDs at creation time, you can manage with a lot fewer bits if you want shorter URLs or something.
paulddraper 3y ago
> these aren't situations I've ever found myself in You've never needed to give an order number, member ID, account ID, invoice number, etc?

emodendroket 3y ago

> A simple ID like 3c6n63N is more than enough to represent any product while keeping it readable and making communication easier. A UUID alternative like a73ba12d-1d8b-2516-3aee-4b15e563a835 is just wasteful from a user’s perspective. I would challenge the premise we appear to be starting from, that the average end user cares to be dealing with any random string of numbers and digits. GUIDs work well, they’re implemented everywhere, and you won’t find out long after you go into production you made some mistake that is going to make it so you have to migrate away from them.

czechdeveloper 3y ago
Plus I could easily represent UUID in base64, such as JlEt5BSYe0enwB7nxl5V6g, which makes it shorter if I need that.
- masklinn 3y ago
  Maybe use a smaller alphabet though, base64’s can be quite hard to read / spell out / reproduce as it includes confusing pairs. While somewhat less dense, base58 or rfc 4648 base32 mitigate these issues.
  - Olreich 3y ago
    And for UUIDs, Base58 and Base64 both require 22 characters, so no density loss! Even Base32 only takes you up to 26 characters.
  - ianburrell 3y ago
    Base32 or Base36 have the advantage of using single case which is easier to read out. And no symbols.
    - stevesimmons 3y ago
      The official Base32 doesn't work for encoding UUID7, which is time-ordered, because its symbols for 0-31 are not in ascending ASCII order. My own preferred format for UUID7 is one I call "id25". It's really Base35 because alphabets of 35 and 36 characters both need 25 characters to represent the 128 bits in a UUID. So I can start with Base36 and take out one of the next most ambiguous character pairs. The result looks something like '0pydgw5pifvapk5zyhmpso5tx'. The two other advantages of using this id25 format rather than UUID7 are: - The id25 format is quite distinct from UUID4. So if you have UUID7s being generated distributedly (rather than centrally) and the UUID7 time-ordering feature is important, it's nice to have a format that a trivial "if '-' in uuid_string..." check will spot. - Because there are no hyphens or symbols, the whole id25 uuid can be selected in web page with a double click. Whereas a uuid will need a mouse movement to get all 5 parts.

ddoolin 3y ago

Sure, you don't need them, but why re-invent the wheel? At least UUIDs are not some one-off function in a helpers file that may need to be amended many times before you get it right, but are backed by a specification, are time-sortable, etc.

jaywalk 3y ago
Not all UUIDs are time-sortable.

mattbeck 3y ago

I mean, don't use UUID for user-facing things sure, but that doesn't mean it's not useful in of itself. That's a terrible way to use it is all.

ars 3y ago

Use sequential IDs, but run them through a maximal linear feedback shift register. They will be in a deterministic order, but will appear semi-random to the end user. For things like product IDs or user IDs, etc you don't actually need them to be random. But perhaps you don't want them to simply start counting sequentially.

no_wizard 3y ago

Isn't auto generating UUIDs "fast enough" in most situations that its negligible? In particular databases (where UUIDs taking space was a big concern) they have largely switched to a packed binary format that makes the size of UUIDs over time a non issue for all practical purposes.

hot_gril 3y ago
UUIDs are packed in DBs. But they tend to get misused as PKs. In common relational DBs, your PK for every table should probably be bigserial.
- siva7 3y ago
  No, that shouldn’t be the criteria (common relational db) when deciding to use bigserial v UUID but the overall architecture and requirements of your system.
  - hot_gril 3y ago
    Whatever your overall requirements are, if you've already decided on a relational DB to solve them, you probably want bigserial PKs in said DB. The DBMS is optimized for that (as I've noticed in Postgres), and you don't gain anything by using UUIDs instead there. UUID in a secondary col or elsewhere in your system, that's fine. If you have to look up by UUID from outside and have no foreign keys, UUID PKs can be faster. But that's the classic use case for a non-relational DB.

jackblemming 3y ago

Doesn’t UUID have mechanisms to prevent collisions even if you start two processes simultaneously with the same initial seed? There are human readable tweaks you can make to them that preserve all of the good properties without limiting the pool much though.

WirelessGigabit 3y ago
v4 is. v1 relied on the MAC address and the time. https://devblogs.microsoft.com/oldnewthing/20040211-00/?p=40663 https://devblogs.microsoft.com/oldnewthing/20040211-00/?p=40...

swader999 3y ago

There are so many nuances and edge cases not mentioned in the article. The author needs to do several projects in real life before making global pronouncements like this.

monero-xmr 3y ago
The lack of humbleness and respect for devs who have came before you and solved all these problems is coming through strong. But I myself was that way 15 years ago. It takes a lot of hard lessons to realize everything new is old and we truly stand on the shoulders of 7000 giants all standing on each others’ shoulders.
- jholman 3y ago
  That's how I feel about UUIDs themselves, and the people who advocate in favour of them. Interesting duality.
  - afgrant 3y ago
    It’s just a 128-bit value that’s hexadecimal encoded. What can be controversial about that?
- RockRobotRock 3y ago
  It's good to get a fresh perspective sometimes
- henvic 3y ago
  Did you even read beyond the title? To say that because of this choice of title, I lack humbleness or respect for peers who came before me is despicable. My title is too presumptive. Yet, the reality for almost everyone who doesn't interface with other systems that already require UUID is that. The brief article gives some points – backed by trustworthy sources – and shows some alternatives without further ado. I didn't spend days writing it. I could cover many more scenarios but didn't invest my time in it. So what?

dgb23 3y ago

An issue that is not solved by either UUIDv4 or the proposed solution (random base58 strings) is indexing performance. Both of those solutions typically make it hard for a DB if you write new entries, assuming you have an index on the ID. In addition it might be more calming to actually be sure that a particular ID is not in use without doing a round-trip. Is it practical to pre-allocate empty entries and reserve a set of them?

afgrant 3y ago
Are there modern databases that can’t readily index on a 128-bit value?
- dvlsg 3y ago
  I don't think it's a matter of not being able to index on it, but that mostly random UUIDs (like v4) can lead to some interesting index fragmentation you have to stay on top of somehow.
- konschubert 3y ago
  If you use sequential ids, the IDs most in use are the higher ones. So your index is “hot at the top” and you can keep that part in memory. This is only a problem with extremely big indexes.
- safetytrick 3y ago
  Random ids require random inserts and that is bad for performance. It isn't a deal-breaker in most situations but it is a real cost that you do pay.
- dgb23 3y ago
  You already got multiple answers explaining the performance issue. Now in many, typical applications you'd have to scale up quite significantly before this becomes a problem. But if your application requires you to store small, new entries very quickly, you'll start to notice this even with moderate scale. Disk persistence is often the bottleneck already, and this might make it worse.
- 3y ago

mccrory 3y ago

Best part is watching the video and seeing the guy dance at the end over doing it in a single take.

srameshc 3y ago

Read this as I was about to use uuid for id :/

spion 3y ago
I would use it anyway. This article misses some subtleties that make uuids really great, as well as some real points against uuids (impact of v4 uuids on index access and uuid v7 as a solution to that)

nwah1 3y ago

I recently found ULID. I like its simplicity. It is sequential. It has a very low probability of collision. And it is of a more reasonable length. Because it encodes the time, theoretically you could use it to grab the CreatedDate of a record without a need for another field. https://github.com/ulid/spec https://github.com/ulid/spec

hot_gril 3y ago
Sequential isn't always simple. If your system is distributed in the slightest way (even just multiple threads), it's a bit of a luxury to find sequential guarantees.
- kiitos 3y ago
  ULIDs can get you rough ordering guarantees even in distributed systems. Precision, of course, requires synchronization, which is usually a non-starter.

lifechoseme123 3y ago

You don't need to tell me I don't need UUID ...because I am going to keep using uuid

kolanos 3y ago

You don't "need" them, but they're almost universally understood and supported which makes them super convenient and a good starting point for many use cases. The point in which the "wasteful" nature of a UUUID really bites you is so far down the road that it is almost not even worth thinking about until you get there. Chances are you won't be dealing in petabytes at all and the 64 bit savings you are fretting over doesn't really matter.

kiitos 3y ago

A UUID is 128 bits, or 16 bytes. This code suggests IDs which are 11 bytes. That's not really comparable. A UUID can also be encoded in any form, it doesn't need to be represented as the dashed string notation which is common, you can just as easily use the base58 alphabet suggested in the post. But the code in the post doesn't encode to base58 correctly. You need to map 58 bits of the input, sequentially, to one character in the alphabet as output. You can't just mod each byte of the input by the alphabet length and use the corresponding alphabet element.

cbarrick 3y ago
The base58 code in the post is wrong, but your version isn't right either. Since the alphabet is 58 characters, each character contains log2(58) bits. Since that's not an integer, the encoding process is a bit more complex than just mapping bits to characters in a table. https://digitalbazaar.github.io/base58-spec/#encode https://digitalbazaar.github.io/base58-spec/#encode
- henvic 3y ago
  Hi. I'm the author of the post here. I agree lines 12-14 doesn't do a proper encoding, but that's not what I was after. Would you still say it's still wrong if you consider that my function is not really encoding the string per se, but using rand.Read to generate entropy for what I want to be the final output (random string with the base58 alphabet)?
  - kiitos 3y ago
    Your IDs are []byte of len=11. Those bytes can be represented in many ways. You can represent them as hex strings via encoding/hex.EncodeToString(id), or base64 strings via encoding/base64.StdEncoding.EncodeToString(id), or base32 strings via encoding/base32.StdEncoding.EncodeToString(id), or etc. Looks like the most used base58 package is https://pkg.go.dev/github.com/btcsuite/btcutil/base58 https://pkg.go.dev/github.com/btcsuite/btcutil/base58, but looking at the implementation [0] I'm not impressed, there's definitely a much better approach. [0] https://github.com/btcsuite/btcutil/blob/v1.0.2/base58/base58.go#L49 https://github.com/btcsuite/btcutil/blob/v1.0.2/base58/base5... But how you encode 11 bytes of data is kind of orthogonal to the important thing, which is that you have 11 bytes of data. Those bytes should be always be stored in memory (in your application, or in your DB, or wherever) as the actual 11 bytes of the ID, and not as a base58 or base64 or JSON or whatever other kind of string that can be decoded to the actual 11 bytes they represent. Likewise, a UUID shouldn't be stored as a string like "64d3f2e0-a4dc-48d3-98ad-7f09eb3b082f", that's a specific encoding of the actual 16 UUID bytes, you should store, process, etc. those bytes directly.

erlendellingsen 3y ago

I usually go for Nano Id for new projects https://github.com/ai/nanoid https://github.com/ai/nanoid

eagleinparadise 3y ago
I generate uuidv7 as PK and have a publicId "slug" with a 13 character nanoId to display in the URL not sure if this is duplicative but down the road i figure I can navigate whichever direction i need to go in
phlogisticfugu 3y ago
+1 for nanoid, which matches what I used to implement myself as a lib function in new projects

BiteCode_dev 3y ago

If your users needs to communicate an ID orally to you, you already failed in numerous ways.

alwaysbeconsing 3y ago

> Version 4 is completely randomly generated (hence, it has more entropy) and is what most web systems seem to use. It has 16^32 = 2^128 bits that guarantee uniqueness and has an insignificant risk of collision. Small correction here but it is not 128 random bits entirely: 6 bits are reserved for the version marker.

tcoff91 3y ago

UUIDs are great for when you have a distributed system where connectivity between the nodes is intermittent, and every node needs to be able to create new entities without ever having ID collisions. For instance, UUIDs allow mobile apps to create entities while offline.

rdtsc 3y ago

> This solution uses the human-readable base58 encoding scheme. We could encode the 128 bit UUID integer in base58 as well if needed. Its textual representation won't conform to the standard but we'd get the same number of bits. Which would be 2^122 or 2^121 bits of uniqueness not 2^128 if it's a proper UUID. 11base58 character certainly doesn't have the 2^122 bits. So we could decided separately if we could either reduce the number of bits needed and/or use a different encoding. > If you click and buy any of these from Amazon after visiting the links above, I might get a commission from their Affiliate program. :-)

nodesocket 3y ago

I’m not sure if this is accurate any longer, but it was bad practice to use UUID’s as PK’s in PostgreSQL for example due to indexing and performance. Is this still true? It’s really nice in PostgreSQL to just do: id UUID DEFAULT gen_random_uuid()

xcdzvyn 3y ago

> As Tom Scott shows in his video, 11 base58-encoded characters are enough for YouTube to serve content even when considering that private videos should be undiscoverable. Nitpick: Google isn't concerned about the discoverability of private videos; those can only be viewed when granted access. You're thinking of unlisted videos.

henvic 3y ago
Fixed! Thanks!

stayfrosty420 3y ago

uuids are ubiquitous and easy and implemented in many languages in the standard library - therefore a dev can get near-guaranteed uniqueness without thinking too hard about it. Until another standard similar to what the article is suggesting becomes widely implemented in standard libraries then uuid isn't going anywhere, although in principle I agree with many of the arguments presented.

pphysch 3y ago

You Don't Need UUID, But You Probably Want It: - Don't invent your own ID datatype (especially 11 byte one), this is almost guaranteed to cause dangerous bugs, because each integration will have to carefully implement/hack it. - Use UUID (preferably the new v7). 128-bit UUID is implemented, for you, pretty much everywhere. - Serial integers still work too, but you should choose them consciously to fit the data model. - Implement "natural keys" if you want pretty/memorable/Cool URLs. Never use non-standard PKs to store custom semantic data, because inevitably you will get garbage PKs that need to be fixed, and migrating a PK value is extremely risky.

afgrant 3y ago

It’s a sales pitch for base58 that I’m not buying at all. According to the code segment, they include numeral “1” but exclude lowercase “l”? 0-9 and a-f are unambiguous and widely understood. They’re more “human readable”.

saulpw 3y ago

I completely agree. UUIDs are almost always evidence of overengineering, especially when user-facing. 64-bits is "enough" of an address space for almost any purpose, even global. And while it seems like speaking an ID over the phone or having to scan it manually is something you'd never have to do, in practice it happens all the time. Cut-and-paste is not always an option on all platforms.

afgrant 3y ago
128 bits versus 64 bits is a deal breaker?
- saulpw 3y ago
  Yes. Even 64 bits is too many, really: at 6 bits per human-recognizable token, it's more than 10 tokens, which is outside the range of what almost any human can keep in their working memory. You can't even hold it in your head long enough to type it in a different window. 128 bits is completely beyond that; when confronted with 128-bits like a UUID, people just give up. Seriously, try actually typing in a UUID sometime. Even with the grouping it's incredibly difficult. Between 64-bits and 128-bits you discover that you absolutely need an automated way to transmit the data, and then how do you get it to that airgapped computer?
  - WA 3y ago
    The more important requirement is that the id should not be guessable and not easily be brute-forced (example: YouTube private video listings). Now, what's the minimum required entropy to make it unguessable but also make it short enough to have people tell the id over the phone or something?

insanitybit 3y ago

I think there's a better "You Don't Need UUID", which is that you probably aren't going to generate so many integers per second that you need to do so in a way that's perfectly collision free. For example, you can update a postgres integer 1000s of times per second, sequentially, and 100s of 1000s of times per second if you allow for gaps/ unordered sequencing. The reason to choose a UUID is because it doesn't require a db. Totally fair. But you probably already have a db lying around if you're generating UUIDs to begin with? Or because you need an unguessable token, which a UUID is great for. But if you have a postgres db around somewhere consider just having a counters table and using that. It'll be fast enough for almost anyone, it'll return smaller tokens, and it'll return tokens that are sequentially allocated (in range. These are nice properties to have!

SoftTalker 3y ago
Database "sequence" types that are used to create integer IDs often no not make any guarantees about being gap-free. This can be a problem for people who don't realize this, and use them for something that is legally required to be gap-free, such as invoice numbers in some jurisdictions. Unguessability is a nice layer to have in your security strategy, especially if you're passing IDs to and from a web page, but if that's all you rely on you're not doing enough.
- insanitybit 3y ago
  > Database "sequence" types that are used to create integer IDs often no not make any guarantees about being gap-free. Yep, the docs from Postgres are very clear that `serial` has gaps. But you can also just create a counter table and manually manage the sequence if you have strict serial requirements. Of course, if you have serial requirements a UUID would already not be viable.

siva7 3y ago

> A UUID alternative like a73ba12d-1d8b-2516-3aee-4b15e563a835 is just wasteful from an user’s perspective. The argumentation in this article is pretty poor from my experience. A UUID isn’t meant to be handled by the non-technical end user. The end user usually doesn’t and shouldn’t care about the URL. I can assure there are bigger architectural problems in your design if your user has to care about accessible internal ids.

olalonde 3y ago
Does this help see what's wrong with UUIDs in URLs? A GitHub issue URL in a parallel universe: https://github.com/a73ba12d-1d8b-2516-3aee-4b15e563a835/e16957bf-7d7f-41f1-97f4-98c93a6c3540/issues/4e4e9133-bcdb-45b9-8dfe-9e951846e3c6
- siva7 3y ago
  I see bad url design for other reasons but nothing wrong with UUIDs, so no it doesn’t help me see the argument.
  - olalonde 3y ago
    What do you mean by bad url design? Because it's using UUIDs? That was my point.
    - JaredCliff 3y ago
      Not OP but it's obvious to me that by bad url design, they mean that you could have used usernames and issue numbers in the URL but you chose to go with random IDs where there was no need for it.
- j-pb 3y ago
  A fairer comparison would be that in that universe the url would be: https://github.com/4e4e9133-bcdb-45b9-8dfe-9e951846e3c6 or even urn:uuid:4e4e9133-bcdb-45b9-8dfe-9e951846e3c6 with the right infrastructure.

wnolens 3y ago

My experience: If you're storing data in a database keyed by a GUID, you get a nicer random distribution in an index, avoiding hot partitions when your workload is seemingly random.

Exuma 3y ago

YoU dOnT nEeD _____ AKA I will make broad sweeping stupid headlines

Waterluvian 3y ago

I think is an “engineer thinks harder about UX” kind of problem. UUIDs are fine for what they’re good at. Try to avoid making them user-facing because, yeah, they’re ugly. I also wouldn’t hand 128 bit integers to the user if I could avoid it. Don’t forget: UUIDs are not dash-separated strings. They’re integers. You can render them differently if those dashes are sucky.

Ameo 3y ago

I like UUIDs aesthetically. They're also an instant queue that something is an "entity" and familiar to people for that purpose. Databases and lots of other things also have native support for them which makes them even more appealing to use. Of course, there are plenty of reasons to avoid them like the article discusses. I'd almost certainly not criticize someone for choosing smaller IDs as long as security isn't an issue.

henvic 3y ago

Author here. I posted this because I've witnessed many systems in companies I've worked for where our end-users needed UUID to communicate with it (technical support, customer ID, etc.) in a way that makes communication harder. We could've used another shorter ID scheme, which would be fine. The good thing about UUID is that it's omnipresent. From what I've heard, it's this lengthy (2^32) because it was hard to guarantee uniqueness when it was conceived in the telecom industry. The length is overkill, and per se, that's fine, but the fact that it dampers communication is awful. That all said, since posting this, I've come to terms with accepting that it's part of life ¯\_(ツ)_/¯ P.S. Using a second human-friendly ID to end-users is an alternative adopted by some projects. However, most projects don't bother, and also, most good IDs you might want to share with people would make UUID unnecessary anyways (in practice).

konschubert 3y ago
Just use the first N characters of the UUID if you want a shorter version of the UUID. And if you want to be really human friendly, use one of those silly name generators.
diplocorp 3y ago
> it's this lengthy (2^32) UUIDs are 128 bits, so it's 2^128 rather than 2^32
Nullabillity 3y ago
There is no fundamental tech advance that caa make the birthday paradox irrelevant or obsolete, and there never will be.
- rwiggum 3y ago
  all of cryptocurrency relies on sha256 the fact that the space is huge, and just ignores collisions. seems to work fine.
  - Nullabillity 3y ago
    SHA-256 has, well, 256 bits of entropy. What I took issue with was the claim that UUID's 128 bits (well, almost, anyway) are overkill with modern advances in randomness.

tombert 3y ago

I don't know that I really agree with the logic here; certainly for some cases having a pretty URL matters (e.g. URL shorteners or even something like YouTube), but if your goal is to minimize URL size, and if you're centrally synchronizing anyway to avoid conflicts, why not just use integers or something? If you use a GMP-style bigint, I can virtually guarantee it'll be faster than any random string concat glue you come up with, and you're never going to run out of integers. I like using GUID specifically because there's usually a built-in implementation everywhere, and because it's so stupidly huge, the likelihood of a conflict is statistically zero, meaning I don't need to bother synchronizing against a server. Yes, it's technically possible I could save a few bytes of bandwidth or something by using some kind of base encoding, and maybe in some kind of embedded case that might matter, but for a vast majority of cases, GUID is absolutely fine, and since it's used everywhere, it's also very thoroughly tested by multi-billion-dollar corporations meaning that I don't have to worry much about any issues. So no, you don't need GUID, you don't need a lot of stuff in the software world. I could poll bytes from /dev/urandom and probably get something that works well enough, I could turn off garbage collection and just pre-allocate all my memory before hand, I could avoid an OS entirely and write straight to the metal, I could do a lot of things, but I don't because I value my time . GUID solves a specific problem pretty well. EDIT: The reason I'm specifying this is because I think the title would be better if it was something like "You (probably) don't need UUID".

0xbadcafebee 3y ago

This is such a HN article.

Glyptodon 3y ago

Is there any existing standardized middle ground between UUIDs, random bits of length n, and serial IDs? I totally buy that UUIDs are unpleasant in URLs in many cases. I can also imagine that in many cases UUIDs for small objects might be a huge % of on the wire payload size. But besides human friendly slugs, which usually have an ID mapping behind the scenes in my experience, it seems like there might be more work than value for startups in many cases.

olalonde 3y ago

I wonder how Stripe generates their IDs. They look quite nice.

techn00 3y ago
They use a type safe extension of UUIDv7 Here is a go implementation and explanation: https://github.com/jetpack-io/typeid https://github.com/jetpack-io/typeid

jherskovic 3y ago

It's amusing that the UUIDs are considered 'ugly' while the Amazon and YouTube IDs are not. Beauty is in the eye of the beholder. I don't find UUIDs pretty, at all, but the others are even uglier to my taste. Yes, UUIDs are overkill for most applications. But CPUs and hard drives are, relatively speaking, cheap. Using an existing, battle-tested unique ID library implementation has advantages. The 8 bytes per record you're saving over bigserial is, for most use cases, negligible. 1,000,000 rows? You'll save 8 MB by switching away from UUIDs. Most databases won't be that large. Use a UUID if you want; pretend you'll have Really Big Data some day if it makes you happy. Render it using a special function if the hyphens are too ugly.

pdmccormick 3y ago

Pivoting to the related subject of compact textual representations of _typed_ data, check out CESR[0]. Through some careful choices of code prefixes and payload lengths that won't require Base64 padding, it provides a novel encoding scheme for JSON plus cryptographic signatures (all while enabling true concatenative composability). It is at the heart of the KERI[1][2][3], a decentralized identity management scheme, and ACDC[4], a mechanism for verified credentials. [0] https://trustoverip.github.io/tswg-cesr-specification/draft-ssmith-cesr.html https://trustoverip.github.io/tswg-cesr-specification/draft-... [1] https://keri.one/keri-resources/ https://keri.one/keri-resources/ [2] https://trustoverip.github.io/tswg-keri-specification/draft-ssmith-keri.html https://trustoverip.github.io/tswg-keri-specification/draft-... [3] https://github.com/WebOfTrust/keripy/ https://github.com/WebOfTrust/keripy/ [3] https://trustoverip.github.io/tswg-acdc-specification/draft-ssmith-acdc.html https://trustoverip.github.io/tswg-acdc-specification/draft-...

ekojs 3y ago

IMO, a good middleground is using schemes like TypeID[0], ulid[1], or KSUID[2] that provides a more compact and readable (base32) representation and provides better database locality (K-sortable). [0] https://github.com/jetpack-io/typeid https://github.com/jetpack-io/typeid [1] https://github.com/ulid/spec https://github.com/ulid/spec [2] https://github.com/segmentio/ksuid https://github.com/segmentio/ksuid

safetytrick 3y ago
ulids are beautiful little ids without many of the drawbacks and risks of other id systems. They are sufficiently unique for most work (anywhere that you don't need to actually uniquely engineer your ids) and their built in ordering and slick encoding are just icing on the cake.
hot_gril 3y ago
As noted on an earlier HN submission about typeid, K-sortable only matters to distributed DBs, and things get tricky there. Spanner for example says to use deliberately unsorted primary keys. I really think when you're picking PKs, you should simply use whatever the DBMS recommends for performance. It's not the PK's job to be typed, sequenced, human-readable, or anything like that; that can be handled by other cols and logging rules. Its job is to be fast in those joins you'll constantly be making against it.
- sgarland 3y ago
  Sortable primarily keys matters very much for any DB clustering on them, e.g. MySQL/InnoDB and SQL Server. It also matters, albeit to a much lesser extent, to others. Postgres stores tuples in a heap, but the PK is still a B+tree (ish), so an INSERT or UPDATE heavy workload will suffer somewhat.
  - hot_gril 3y ago
    Yeah, you're right. But for those, the typical PK recommendation of bigserial is already time-ordered and mostly packed.

tanepiper 3y ago

Everyone wanted UUIDs and we had IRIs all along...

fuzzythinker 3y ago

I see ulid and nanoid being recommended here. I like cuid2 based on input from its README [0] and discussion [1] [0] https://github.com/paralleldrive/cuid2 https://github.com/paralleldrive/cuid2 [1] https://github.com/paralleldrive/cuid2/issues/7 https://github.com/paralleldrive/cuid2/issues/7

kiitos 3y ago
cuid [0] seems to be a random letter, concatenated with a sha3 hash of: - the system time - a process-global singleton counter (incrementing with each ID) - a "salt" derived from some number of bytes taken from an entropy source - a "fingerprint" of some number of bytes taken from the same entropy source this is an awful lot of work to be doing for each ID, to what end? well issue 7 [1] says that > Cuid2 is good when you need something secure, extremely collision resistant, and your system is distributed or decentralized (e.g. you want to be able to create records with ids on the client side), or you are building software that may need to scale horizontally). but I'm not sure how cuid as implemented provides a better answer in any of these dimensions to a plain old UUID maybe I'm just being cynical, I dunno [0] https://github.com/paralleldrive/cuid2/blob/fb07094487ba5ad038a51ef67bcb9c5f525c173b/src/index.js#L97 https://github.com/paralleldrive/cuid2/blob/fb07094487ba5ad0... [1] https://github.com/paralleldrive/cuid2/issues/7 https://github.com/paralleldrive/cuid2/issues/7

konschubert 3y ago

Just use UUID. If you need something user friendly, use the first 6 or 8 characters. If you need to worry about index performance due to enormous index sizes, you probably know it.

rekabis 3y ago

For the web, sequential ID attacks represent a significant problem in terms of a Resource Enumeration Attack. It gets worse when auth isn’t even in play yet (the user isn’t logged in, or doesn’t even have an account yet, or this info is meant to be available outside of auth but to only one specific user), but you need to display only the current item, and not any others. Having a sequential ID of any kind allows the user to trivially hack their way into any other item they wish by just incrementing or decrementing that ID. UUIDs represent a trivially easy way of implementing a non-incrementing ID with a ridiculously unwieldy address space that makes it supremely unrealistic for the vast majority of users to mess with. They’re just going to give up long before the first successful hit.

tommiegannert 3y ago

I'm building an object database. In different places, it uses four different categories of IDs, depending on requirements: 1. Compact sequential is used, obviously, where order matters. It has the drawback of requiring a coordination with a singleton. (This can be sharded/vectorized, of course.) Aside from that, it also leaks the number of objects/transactions, just by looking for the highest number available. Can be varint-encoded very nicely. 2. Compact non-sequential is used where I need a small identifier, but not leak the number of objects. Since it's compact, I must still guarantee uniqueness as in (1). This is currently implemented using a block cipher on top of the compact sequential ID generator. The drawback of this is that the domain may still be in guessable territory, depending on how much is generated. A 64-bit integer filled with 4B only requires 4B guesses to hit a collision. I don't use this much. The key can never be rotated: a key number would eat up precious bits. 3. Sparse random, used where non-guessability is important, aside from not leaking rates/counts. Take a Google Docs sharable link as an example. I doubt YouTube cares about this. This is where something like a 128-bit number like UUID or ULID shines. The space is large enough that uniqueness is assumed, given a decent PRNG. Sure, I try to use the nicest one at any given point (e.g. using a compact sequential instead of non-sequential during debugging.) But fact is that sparse random just tick more boxes. 4) Sparse, human readable. For "vouchers". They are bearer tokens that give requests more powers, e.g. to create an account or act as admin. These should be reasonably human readable, so they can be spoken. They obviously need to be sparse and hard to guess, which requires a trade-off in length. I present them in three ways: base32, english words and QR-code. Pick one; they all do the same. For copy-pasting, base32 might be best (or base58, by all means.) For shouting to a colleague, the sequence of english words might be better. I'd add other languages as needed: it's just a fixed list of words. The nice thing is it can encode the sequence in base-500 or base-1000 without being obnoxious. (The Matrix protocol and others use emoji lists, but it's the same idea. [1]) Finally, if you have a phone in your pocket or camera on your computer, perhaps the QR-code is the easiest way to use the voucher code. [1] Actually, IIRC, Matrix only uses 64 emojis, which feels a bit wasteful.

irq-1 3y ago

> This solution uses the human-readable base58 encoding scheme. How readable is base58 in Arabic or Chinese? I've concluded that only the standard numbers 0123456789, algebra symbols +-/*= and phone symbols #* are universal enough for a global encoding. Any other insights are welcomed!

olalonde 3y ago

In case it helps anyone, I just went down the rabbit hole of calculating the amount of random bits required to avoid collisions. The traditional Birthday Paradox formula gives us a probability but it's not very intuitive to understand what the probability means in terms we are used to as developers, so I tried a different approach: expected time period for a collision to happen. https://colab.research.google.com/drive/1ec4n7Ex9bnkl_c45EUlIPCNtsLKrwaB2?usp=sharing https://colab.research.google.com/drive/1ec4n7Ex9bnkl_c45EUl...

x-complexity 3y ago

No comments on this page about Snowflake IDs. https://en.wikipedia.org/wiki/Snowflake_ID https://en.wikipedia.org/wiki/Snowflake_ID They fit within 64 bits, allow for more than enough processes to handle 10k+ transactions per second, give enough of a timestamp headroom for decades into the future, and where ID generation can be made isolated to each process. They don't work well for anything related to archival work, but you might as well use a regular ID for that anyway, unless you're also actively scraping terabytes of data off of the Internet every second, in which case UUIDv5's good enough for your extreme edge case. ...But at that point you might as well just roll your own 128-bit version of a Snowflake ID.

M2Ys4U 3y ago
>...But at that point you might as well just roll your own 128-bit version of a Snowflake ID. Or UUIDv7, which is designed to solve the same problem

RamblingCTO 3y ago

We need them tho. Especially UUIDv4 and UUIDv5. We have a distributed system and we manage permission identities with that. Each item that's supposed to be restricted has either of these ids. UUIDv4 for global things like workspaces and users, UUIDv5 for workspace-specific stuff (the workspace UUIDv4 is given as a namespace for the v5) and it's quite beautiful and prevents clashes. So it really depends on your usage. We still have long ids that are exposed to the client tho.

CiaranMcNulty 3y ago

> If you’re talking about a web system composed of microservice architecture all running on the same datacenter, perhaps sharing the same database We have different ideas what 'microservice architecture' means, I guess. One of the key points of UUIDs is to be able to generate (probably) non-conflicting values without coordination

consoomer 3y ago

I have created a custom unique identifier scheme before. This post doesn't address some of the challenges you come across. 1. When dealing with user friendly IDs, it's often important to make sure there are no ambiguous characters. This is a UX requirement for anyone that may need to read an ID and type it in at any point. For example, this means removing certain confusing characters like "0oIi1l5sS", etc. You end up with a much smaller set of characters. In my case, young children were required to type in teacher codes. Needless to say, I had to limit a lot of possible confusing characters. 2. You will have collisions. How do you handle them (ie, retry N times until you get a valid one)? What happens if you can't get a valid one? This happened in a product of mine. We had ~8 character codes that were human readable and type-able and we... ran out of codes. 3. How can you embed more information into the code such as versioning? It's often useful to prefix a code with some info in the event you need to modify it later (such as expand the character set or length)?

nick_ 3y ago
Number two can be addressed with something like this: https://preshing.com/20121224/how-to-generate-a-sequence-of-unique-random-integers/ https://preshing.com/20121224/how-to-generate-a-sequence-of-... You can shrink the range by using a smaller prime number and throwing on out-of-range input. And then you can offset the output to make sure the base-N encoded value is always D digits long.