8 ms·

I really like the ideas behind ksuid (near the end of the article). However, two quotes: > Those concerned with UUID collision in a properly-configured system

I really like the ideas behind ksuid (near the end of the article). However, two quotes: > Those concerned with UUID collision in a properly-configured system would find their time better spent pondering far more probable events like solar flares, thermonuclear war, and alien invasion on their systems. And then further down: > A “custom” epoch is used that ensures >100 years of useful life. Wait, so the last 128 bits of a KSUID won't get me in trouble before the sun explodes, but the first 32 bits (the timestamp) will cause trouble well before my grandkids die? I really wonder why they didn't reserve some more bits for the timestamp, if necessary at the cost of some less randomness. Could've made this stuff last for millenia at no extra collision risk, in practice.

OzzyB 9y ago
That's because now matter how many times we're told to simply trust uuids we will always want to add a timestamp just in case :) Maybe we should just admit that no amount of poetic hyperbole like "until an alien invasion" will quell our innate fear of having uuid that will collide.
- reitanqild 9y ago
  I have heard people laugh at the risk of UUID collisions and thought we were safe but heard two years ago that a consulting company I worked with had seen real issues with UUID collisions on a production system.
  - treebeard901 9y ago
    I doubt many people would argue with the math behind modern uuid generation functions. Those of us that have been around for a while know the problem can come from the implementation and the storage for the UUID. For example: Bugs in code that may generate a unique ID once and then use it multiple places. Imaged OS installations that may cause a UUID to be duplicated Spoofing another ID for malicious reasons Accidental database update statement People tend to give UUIDs magical properties but it needs to be treated similar to an unique integer.
  - sroussey 9y ago
    If you are using uuid generation that uses MAC address and time stamp, then you can have problems with multiple processes generating uuid for the same system. There are other uuid generation algorithms though.
fenwick67 9y ago
The 100 years thing is a peeve of mine too. Especially if they want to see this widely adopted. That is how things like IPv4 address exhaustion and Y2K problem happen. It's 2017, add another byte to your timestamps.
stock_toaster 9y ago
Yeah. Not sure why they didn't just go straight for 64bit timestamps (maybe 10^-8 second granularity ~= 5.8k years) and 64bits of random data. I also wonder if base58 would have been a bit nicer. base62 is of course slightly more compact, but base58 is nice that it reduces visual character ambiguity.
- tytso 9y ago
  The original UUID's (from Apollo, as used in OSF DCE, e2fsprogs, Microsoft, etc.) used a 64-bit timestamp, with 100ns granularity. It uses as the start of its Epoch the beginning of the Gregorian Calendar (00:00:00.00, 15 October 1582), so it's good up to 3400 A.D. before it hits the 2038 problem. :-) Note that UUID's, like IPv6 addresses, are sufficiently long that if users are needing to interact with them directly, You're Doing Something Wrong. So the whole base-62 versus using hyphens versus base58 discussion misses the point, in my view. Computers will generally be exchanging them in 128-bit binary format, and they should only really be dumped out for debugging reasons.
  - stock_toaster 9y ago
    > Note that UUID's, like IPv6 addresses, are sufficiently > long that if users are needing to interact with the > directly, You're Doing Something Wrong One would think so, but you would be surprised how many times someone has typed out to me (or copy/pasted) a UUID for debugging purposes, or how often I have had to eyeball UUIDs in production logs.
    - kijin 9y ago
      We eyeball, compare and copy/paste git commit IDs all the time, and they're longer than UUIDs. I don't see any problem with it as long as there's good tooling to help us interact with these IDs (e.g. git allows us to use the first few characters of a commit ID if there's no collision).
  - rbranson 9y ago
    [Author here: Hey there Theodore -- first of all thank you so much for that other comment with first-hand knowledge about Apollo => UUID. This is the kind of priceless nugget that makes HN such an interesting place.] From a systems perspective I can definitely see why one could reach the conclusion that if users are interacting with them directly, they're doing something wrong. What we've seen is that IDs are routinely copied between different systems for debugging purposes. An example at least for Segment it is common to have IDs pasted into support tickets, and the search engine that indexes these will tokenize on the "-" in a UUID. The difference between finding the solution in minutes vs hours could be that two systems can be correlated by searching using the UUID. Leaving these out is a minor tweak that pays dividends over time. For anything human-exposed (and in JSON or URLs) we use the 27-byte base62 encoding. In the database we store KSUIDs in their 20-byte binary (base256?) encoding.
grawlinson 9y ago
Yeah, which is why I'm toying with using ulid identifiers, which won't face timestamp issues until around 10900 AD. Source here: https://github.com/alizain/ulid/ https://github.com/alizain/ulid/
- Sevein 9y ago
  Related: https://github.com/oklog/ulid https://github.com/oklog/ulid
  - eriknstr 9y ago
    The repo in the comment parent to yours has a list that includes the one you mentioned and several others in various languages :) https://github.com/alizain/ulid#implementations-in-other-languages https://github.com/alizain/ulid#implementations-in-other-lan...
graphememes 9y ago
I'm banking on ULID and surprised that they didn't adopt it, it's a much more approachable solution and battle tested.
philip1209 9y ago
Are UUIDs secure for creating a unique user session identifier?
- __s 9y ago
  No, UUIDs aren't meant to be cryptographically unique. Session identifiers should be generated by your friendly neighbourhood crypto rng man
  - MichaelGG 9y ago
    If you're not specifically requesting a time based ID, any proper uuid lib should just return 16 bytes of randomness from the system rng.
    - johncolanduoni 9y ago
      Not quite, most generate v4 uuids which have a few fixed bits.
    - nucleardog 9y ago
      Getting pedantic, but a proper uuid lib should be returning 122 bits of randomness, and six bits of fixed information that specify the uuid version and generation method ("random").
      - MichaelGG 9y ago
        For what purpose? Where's the real world reason and interop? Just because an RFC says to set some bits? Is there any code you're likely to hit in your app that actually cares?
        vog 9y ago
        I guess the purpose is to be able to put different UUID generation schemes into separate kind-of "namespaces". That way, a UUIDv4 (random) will never collide with a UUIDv1 (timestamp+mac+...), which again will never collide with UUIDv5 (SHA-1). So if one of the UUID generation schemes is flawed, these are easily filtered out, and moreover, will never interfere with any other UUID version/scheme.

OzzyB 9y ago

That's because now matter how many times we're told to simply trust uuids we will always want to add a timestamp just in case :) Maybe we should just admit that no amount of poetic hyperbole like "until an alien invasion" will quell our innate fear of having uuid that will collide.

reitanqild 9y ago
I have heard people laugh at the risk of UUID collisions and thought we were safe but heard two years ago that a consulting company I worked with had seen real issues with UUID collisions on a production system.
- treebeard901 9y ago
  I doubt many people would argue with the math behind modern uuid generation functions. Those of us that have been around for a while know the problem can come from the implementation and the storage for the UUID. For example: Bugs in code that may generate a unique ID once and then use it multiple places. Imaged OS installations that may cause a UUID to be duplicated Spoofing another ID for malicious reasons Accidental database update statement People tend to give UUIDs magical properties but it needs to be treated similar to an unique integer.
- sroussey 9y ago
  If you are using uuid generation that uses MAC address and time stamp, then you can have problems with multiple processes generating uuid for the same system. There are other uuid generation algorithms though.

fenwick67 9y ago

The 100 years thing is a peeve of mine too. Especially if they want to see this widely adopted. That is how things like IPv4 address exhaustion and Y2K problem happen. It's 2017, add another byte to your timestamps.

stock_toaster 9y ago

Yeah. Not sure why they didn't just go straight for 64bit timestamps (maybe 10^-8 second granularity ~= 5.8k years) and 64bits of random data. I also wonder if base58 would have been a bit nicer. base62 is of course slightly more compact, but base58 is nice that it reduces visual character ambiguity.

tytso 9y ago
The original UUID's (from Apollo, as used in OSF DCE, e2fsprogs, Microsoft, etc.) used a 64-bit timestamp, with 100ns granularity. It uses as the start of its Epoch the beginning of the Gregorian Calendar (00:00:00.00, 15 October 1582), so it's good up to 3400 A.D. before it hits the 2038 problem. :-) Note that UUID's, like IPv6 addresses, are sufficiently long that if users are needing to interact with them directly, You're Doing Something Wrong. So the whole base-62 versus using hyphens versus base58 discussion misses the point, in my view. Computers will generally be exchanging them in 128-bit binary format, and they should only really be dumped out for debugging reasons.
- stock_toaster 9y ago
  > Note that UUID's, like IPv6 addresses, are sufficiently > long that if users are needing to interact with the > directly, You're Doing Something Wrong One would think so, but you would be surprised how many times someone has typed out to me (or copy/pasted) a UUID for debugging purposes, or how often I have had to eyeball UUIDs in production logs.
  - kijin 9y ago
    We eyeball, compare and copy/paste git commit IDs all the time, and they're longer than UUIDs. I don't see any problem with it as long as there's good tooling to help us interact with these IDs (e.g. git allows us to use the first few characters of a commit ID if there's no collision).
- rbranson 9y ago
  [Author here: Hey there Theodore -- first of all thank you so much for that other comment with first-hand knowledge about Apollo => UUID. This is the kind of priceless nugget that makes HN such an interesting place.] From a systems perspective I can definitely see why one could reach the conclusion that if users are interacting with them directly, they're doing something wrong. What we've seen is that IDs are routinely copied between different systems for debugging purposes. An example at least for Segment it is common to have IDs pasted into support tickets, and the search engine that indexes these will tokenize on the "-" in a UUID. The difference between finding the solution in minutes vs hours could be that two systems can be correlated by searching using the UUID. Leaving these out is a minor tweak that pays dividends over time. For anything human-exposed (and in JSON or URLs) we use the 27-byte base62 encoding. In the database we store KSUIDs in their 20-byte binary (base256?) encoding.

grawlinson 9y ago

Yeah, which is why I'm toying with using ulid identifiers, which won't face timestamp issues until around 10900 AD. Source here: https://github.com/alizain/ulid/ https://github.com/alizain/ulid/

Sevein 9y ago
Related: https://github.com/oklog/ulid https://github.com/oklog/ulid
- eriknstr 9y ago
  The repo in the comment parent to yours has a list that includes the one you mentioned and several others in various languages :) https://github.com/alizain/ulid#implementations-in-other-languages https://github.com/alizain/ulid#implementations-in-other-lan...

graphememes 9y ago

I'm banking on ULID and surprised that they didn't adopt it, it's a much more approachable solution and battle tested.

philip1209 9y ago

Are UUIDs secure for creating a unique user session identifier?

__s 9y ago
No, UUIDs aren't meant to be cryptographically unique. Session identifiers should be generated by your friendly neighbourhood crypto rng man
- MichaelGG 9y ago
  If you're not specifically requesting a time based ID, any proper uuid lib should just return 16 bytes of randomness from the system rng.
  - johncolanduoni 9y ago
    Not quite, most generate v4 uuids which have a few fixed bits.
  - nucleardog 9y ago
    Getting pedantic, but a proper uuid lib should be returning 122 bits of randomness, and six bits of fixed information that specify the uuid version and generation method ("random").
    - MichaelGG 9y ago
      For what purpose? Where's the real world reason and interop? Just because an RFC says to set some bits? Is there any code you're likely to hit in your app that actually cares?
      - vog 9y ago
        I guess the purpose is to be able to put different UUID generation schemes into separate kind-of "namespaces". That way, a UUIDv4 (random) will never collide with a UUIDv1 (timestamp+mac+...), which again will never collide with UUIDv5 (SHA-1). So if one of the UUID generation schemes is flawed, these are easily filtered out, and moreover, will never interfere with any other UUID version/scheme.