7 ms·

This seems like useless optimization that has a high probability of biting you in the ass later on. Using the standard UUID generation facilities in your OS of

This seems like useless optimization that has a high probability of biting you in the ass later on. Using the standard UUID generation facilities in your OS of choice there's zero chance you get something wrong and screw yourself. UUIDs are great because we can pretty much guarantee global uniqueness. Acquire a company, decide to integrate with someone, need to merge a database, etc? No problem, zero chance of record collisions no matter what happens in the future. (It also means zero chance of accidentally interpreting record #58274 as type A when you meant type C). Furthermore, a 1 in 1 million chance of collision is far too frequent for my liking, but even if it were acceptable what happens when your service/product becomes far more popular than you imagined and you blow through your initial estimates?

bkirwi 12y ago
This is absolutely right. I suspect some people do this because Twitter went out of their way to get 64-bit ids[0] -- but Twitter did this in large part because they made the mistake of baking that bit-length into the protocol back when they were tiny, and since they don't control all clients, it's very difficult to change. For comparison, here's an eager.io URL: https://eager.io/app/ZYBle8qUhKFJ https://eager.io/app/ZYBle8qUhKFJ And here's an equivalent with UUID in base-64: https://eager.io/app/b8tRS7h4TJ2Vt43Dp85v2A https://eager.io/app/b8tRS7h4TJ2Vt43Dp85v2A It's a rare application for which those differences matter. (NB: I don't know anything about eager.io... it might be important for them!) [0] https://blog.twitter.com/2010/announcing-snowflake https://blog.twitter.com/2010/announcing-snowflake
- zackbloom 12y ago
  It becomes a stylistic choice, but I can say we weren't happy with the experience the longer ids gave users. It made most of the url a random hash of characters, not anything significant. The URL is as much a part of the UX as anything else.
  - al2o3cr 12y ago
    "The URL is as much a part of the UX as anything else." Given what we've seen from the browser vendors lately (hiding bits of the address bar, etc), I'd say there's a lot of momentum to disagree with that statement.
    - xahrepap 12y ago
      I think it depends on who your customer is / what your product is. When I'm developing against an api, I end up writing API calls in my terminal to test things. I prefer cleaner URLs for APIs.
  - lmm 12y ago
    Is the difference between bkirwi's two examples really significant? The eager.io URL is still a mash of random characters. (FWIW I like seeing UUIDs in my URLs, because it tells me that the company on the other end has its shit together. That might be a bit programmer-specific though)
- curun1r 12y ago
  There's also impact at the data layer. Larger IDs make indexes get a lot bigger. Also, any article on UUIDs that mentions locality without even mentioning using type-1 UUIDs should be taken with a grain of salt. We looked at using UUIDs last year for a project in MySQL since we had good use cases for handling ID generation outside of the data tier. We originally prototyped using type-4 UUIDs, but found the locality problem made that a non-starter (updating indexes seemed to get exponentially slower when tables went over 10m rows). But switching to type-1 UUIDs made that problem go away. Still, we eventually chose not to use UUIDs since 128 bits made our indexes too large. At the DB level, we represented them as BINARY(16), but even with the pain that caused, our indexes were still too large. Perhaps it's just a MySQL thing (I didn't make that choice) but, from our testing, I'd be very wary of using 128-bit IDs and would probably choose something like Snowflake or other centralized ID generation service.
  - ademarre 12y ago
    Sometimes you don't need to index all 16 bytes, and can get by with a prefix index.
Sharlin 12y ago
Note that the 1 in 1 million figure is for 100 million IDs generated, that is, after 100 million generated IDs there's 1 in a million chance that two of them are identical.
- imron 12y ago
  It's all fun and games until you discover that one in a million is next Tuesday: http://blogs.msdn.com/b/larryosterman/archive/2004/03/30/104165.aspx http://blogs.msdn.com/b/larryosterman/archive/2004/03/30/104...
  - acveilleux 12y ago
    I've watched as the application we develop at work has switched from ancient single and dual core Linux boxes to (now) dual-dodeca-core boxes and all sort of "completely impossible" races have surfaced. All progressively more "impossible" as the number of cores and their speed has kept on increasing.
zackbloom 12y ago
I appreciate where you're coming from, but as it says in the article, UUIDs are really unnecessarily long for no good reason. The traditional hex encoding is also much longer than it needs to be. In any case, the article exists to help you decide how long you need to make your ids for you to be comfortable. If you decide you aren't comfortable with a 1 in a million chance, the math is there for you to figure out what does work for you.
- jacques_chester 12y ago
  > UUIDs are really unnecessarily long for no good reason. Their purpose is to be universally unique, presumably indefinitely. The practical choice is to align on a power of 2. When UUIDs were invented, they were based on a MAC address plus a timestamp. MAC addresses are 48 bits, leaving 16 bits. Of these, between 1 and 3 are used up giving the variant code and another four bits are used giving the version code. If they'd used 64 bits, there'd be between 8 and 11 bits left for a unique timestamp. Which is OK, but not a very strong guarantee for a "unique" scheme. As it happens the length has made it possible to create multiple versions of UUIDs that are generated in different ways. UUID has been successful at separating the concerns of the structure of the ID from how it is generated. Most alternative schemes are effectively implementation-defined.
  - zackbloom 12y ago
    This is all true, and I'm not here to say the original implementers did anything wrong. As you say though, UUIDs are a general solution, which means they're not going to be optimal for every problem.
    - JoeAltmaier 12y ago
      UUIDs are optimal for every problem, in that they are unique and fairly fast to generate. There can be no better solution than that, except maybe to save a few bits. And we're only talking about 16 bytes here; not much to complain about. They're cheap at the price.
mnarayan01 12y ago
> No problem, zero chance of record collisions no matter what happens in the future. Well until 5236-03-31 21:21:00 UTC (assuming my calculations are correct) using v1 UUIDs and assuming unique MAC addresses and no race conditions.
- ed0 12y ago
  Only until 5236?? We definitely need a better solution.
- wahern 12y ago
  Assuming unique MAC addresses is a very poor assumption. At one company I worked at we once received a huge shipment of ethernet cards (several thousand) that all had the same MAC address. We didn't realize it until customers who had purchased multiple pieces of equipment began to call in. I've heard several similar anecdotes from other engineers at other companies. Yeah, very bad assumption.
  - thethimble 12y ago
    A lot of platforms (iOS) also spoof MAC addresses too
    - yohanatan 12y ago
      The spoofing happens when talking to other devices (not themselves); i.e., there is some layer of the device that knows its true MAC address and the UUID generation code should use that layer.
  - ghuntley 12y ago
    Can confirm that this is a bad idea as manfuarcturing defects do happen. Took a shipment of brand new NetApp toasters this year and they all had the same WWN/WWPN's! Needless to say NetApp where quite embarrassed and we are glad that we noticed it in pre-prod otherwise it could have been quite bad.
yohanatan 12y ago
Yea, if you remove the dashes from regular UUIDs and base64 encode them they're not much longer than his custom ones.
kephra 12y ago
The problem of both locality and length also only exist in modern databases. Good old Ingres QUEL offered the choice of hash and btree indexes, and the the ability to compress indexes. Index compression is of course much easier with sequential indexes, but it would at least optimize the static host part of the uuid away.

bkirwi 12y ago

This is absolutely right. I suspect some people do this because Twitter went out of their way to get 64-bit ids[0] -- but Twitter did this in large part because they made the mistake of baking that bit-length into the protocol back when they were tiny, and since they don't control all clients, it's very difficult to change. For comparison, here's an eager.io URL: https://eager.io/app/ZYBle8qUhKFJ https://eager.io/app/ZYBle8qUhKFJ And here's an equivalent with UUID in base-64: https://eager.io/app/b8tRS7h4TJ2Vt43Dp85v2A https://eager.io/app/b8tRS7h4TJ2Vt43Dp85v2A It's a rare application for which those differences matter. (NB: I don't know anything about eager.io... it might be important for them!) [0] https://blog.twitter.com/2010/announcing-snowflake https://blog.twitter.com/2010/announcing-snowflake

zackbloom 12y ago
It becomes a stylistic choice, but I can say we weren't happy with the experience the longer ids gave users. It made most of the url a random hash of characters, not anything significant. The URL is as much a part of the UX as anything else.
- al2o3cr 12y ago
  "The URL is as much a part of the UX as anything else." Given what we've seen from the browser vendors lately (hiding bits of the address bar, etc), I'd say there's a lot of momentum to disagree with that statement.
  - xahrepap 12y ago
    I think it depends on who your customer is / what your product is. When I'm developing against an api, I end up writing API calls in my terminal to test things. I prefer cleaner URLs for APIs.
- lmm 12y ago
  Is the difference between bkirwi's two examples really significant? The eager.io URL is still a mash of random characters. (FWIW I like seeing UUIDs in my URLs, because it tells me that the company on the other end has its shit together. That might be a bit programmer-specific though)
curun1r 12y ago
There's also impact at the data layer. Larger IDs make indexes get a lot bigger. Also, any article on UUIDs that mentions locality without even mentioning using type-1 UUIDs should be taken with a grain of salt. We looked at using UUIDs last year for a project in MySQL since we had good use cases for handling ID generation outside of the data tier. We originally prototyped using type-4 UUIDs, but found the locality problem made that a non-starter (updating indexes seemed to get exponentially slower when tables went over 10m rows). But switching to type-1 UUIDs made that problem go away. Still, we eventually chose not to use UUIDs since 128 bits made our indexes too large. At the DB level, we represented them as BINARY(16), but even with the pain that caused, our indexes were still too large. Perhaps it's just a MySQL thing (I didn't make that choice) but, from our testing, I'd be very wary of using 128-bit IDs and would probably choose something like Snowflake or other centralized ID generation service.
- ademarre 12y ago
  Sometimes you don't need to index all 16 bytes, and can get by with a prefix index.

Sharlin 12y ago

Note that the 1 in 1 million figure is for 100 million IDs generated, that is, after 100 million generated IDs there's 1 in a million chance that two of them are identical.

imron 12y ago
It's all fun and games until you discover that one in a million is next Tuesday: http://blogs.msdn.com/b/larryosterman/archive/2004/03/30/104165.aspx http://blogs.msdn.com/b/larryosterman/archive/2004/03/30/104...
- acveilleux 12y ago
  I've watched as the application we develop at work has switched from ancient single and dual core Linux boxes to (now) dual-dodeca-core boxes and all sort of "completely impossible" races have surfaced. All progressively more "impossible" as the number of cores and their speed has kept on increasing.

zackbloom 12y ago

I appreciate where you're coming from, but as it says in the article, UUIDs are really unnecessarily long for no good reason. The traditional hex encoding is also much longer than it needs to be. In any case, the article exists to help you decide how long you need to make your ids for you to be comfortable. If you decide you aren't comfortable with a 1 in a million chance, the math is there for you to figure out what does work for you.

jacques_chester 12y ago
> UUIDs are really unnecessarily long for no good reason. Their purpose is to be universally unique, presumably indefinitely. The practical choice is to align on a power of 2. When UUIDs were invented, they were based on a MAC address plus a timestamp. MAC addresses are 48 bits, leaving 16 bits. Of these, between 1 and 3 are used up giving the variant code and another four bits are used giving the version code. If they'd used 64 bits, there'd be between 8 and 11 bits left for a unique timestamp. Which is OK, but not a very strong guarantee for a "unique" scheme. As it happens the length has made it possible to create multiple versions of UUIDs that are generated in different ways. UUID has been successful at separating the concerns of the structure of the ID from how it is generated. Most alternative schemes are effectively implementation-defined.
- zackbloom 12y ago
  This is all true, and I'm not here to say the original implementers did anything wrong. As you say though, UUIDs are a general solution, which means they're not going to be optimal for every problem.
  - JoeAltmaier 12y ago
    UUIDs are optimal for every problem, in that they are unique and fairly fast to generate. There can be no better solution than that, except maybe to save a few bits. And we're only talking about 16 bytes here; not much to complain about. They're cheap at the price.

mnarayan01 12y ago

> No problem, zero chance of record collisions no matter what happens in the future. Well until 5236-03-31 21:21:00 UTC (assuming my calculations are correct) using v1 UUIDs and assuming unique MAC addresses and no race conditions.

ed0 12y ago
Only until 5236?? We definitely need a better solution.
wahern 12y ago
Assuming unique MAC addresses is a very poor assumption. At one company I worked at we once received a huge shipment of ethernet cards (several thousand) that all had the same MAC address. We didn't realize it until customers who had purchased multiple pieces of equipment began to call in. I've heard several similar anecdotes from other engineers at other companies. Yeah, very bad assumption.
- thethimble 12y ago
  A lot of platforms (iOS) also spoof MAC addresses too
  - yohanatan 12y ago
    The spoofing happens when talking to other devices (not themselves); i.e., there is some layer of the device that knows its true MAC address and the UUID generation code should use that layer.
- ghuntley 12y ago
  Can confirm that this is a bad idea as manfuarcturing defects do happen. Took a shipment of brand new NetApp toasters this year and they all had the same WWN/WWPN's! Needless to say NetApp where quite embarrassed and we are glad that we noticed it in pre-prod otherwise it could have been quite bad.

yohanatan 12y ago

Yea, if you remove the dashes from regular UUIDs and base64 encode them they're not much longer than his custom ones.

kephra 12y ago

The problem of both locality and length also only exist in modern databases. Good old Ingres QUEL offered the choice of hash and btree indexes, and the the ability to compress indexes. Index compression is of course much easier with sequential indexes, but it would at least optimize the static host part of the uuid away.