6 ms·

How does this compare to Twitter's Snowflake? https://blog.twitter.com/engineering/en_us/a/2010/announcing-snowflake.html https://blog.twitter.com/engineering/e

How does this compare to Twitter's Snowflake? https://blog.twitter.com/engineering/en_us/a/2010/announcing-snowflake.html https://blog.twitter.com/engineering/en_us/a/2010/announcing...

exyi 8y ago
From the obvious things: ULID relies more on randomness while Twitter's Snowflake relies on some assigned worker IDs
BugsJustFindMe 8y ago
There's a trade-off between assigning IDs up front or randomly generating IDs inside a large space. Randomly generating IDs can be done without a central arbiter, but doesn't provide any real guarantee against collisions. People punt on that problem by making their random IDs larger and hoping for the best.
- erik_seaberg 8y ago
  The usual argument: the probability of a collision only needs to be as low as the probability of a single-bit error anywhere on the path. That's the best you could possibly do.
  - giornogiovanna 8y ago
    How do you usually estimate the probability of a single-bit error?
    - erik_seaberg 8y ago
      Most errors are detected by checksums or hashes, so measure that across all your hardware (client requests, network hops, server RAM) and estimate how often your checksum should have collided and let an error slip by. Granted, it's pretty rare to work on a big enough system to have solid data on this.
- senderista 8y ago
  More precisely: to avoid collisions with high probability, random IDs need to be twice the size of sequential IDs for a fixed number of IDs generated, thanks to the birthday problem. You don’t need to “hope for the best”; the probabilities in question can be precisely quantified: https://en.wikipedia.org/wiki/Birthday_problem https://en.wikipedia.org/wiki/Birthday_problem.
  - dragonwriter 8y ago
    You still need to hope for the best because probabilities that aren't zero or one aren't a guarantee of anything except for the asymptotic limit of frequency as the number of trials approaches infinity. If your system isn't resilient to collisions (and distributed ID schemes are usually the chosen to enable a system which isn't), you are gambling. Perhaps with very good odds, but gambling.
tveita 8y ago
They solve roughly the same problem. Snowflake is able to create 64-bit unique IDs at the cost of needing a central coordinating service, this format uses 128 bits for IDs which lets them be created independently as long as you have the correct time.

How does this compare to Twitter's Snowflake? https://blog.twitter.com/engineering/en_us/a/2010/announcing-snowflake.html https://blog.twitter.com/engineering/en_us/a/2010/announcing...

exyi 8y ago

From the obvious things: ULID relies more on randomness while Twitter's Snowflake relies on some assigned worker IDs

BugsJustFindMe 8y ago

There's a trade-off between assigning IDs up front or randomly generating IDs inside a large space. Randomly generating IDs can be done without a central arbiter, but doesn't provide any real guarantee against collisions. People punt on that problem by making their random IDs larger and hoping for the best.

erik_seaberg 8y ago
The usual argument: the probability of a collision only needs to be as low as the probability of a single-bit error anywhere on the path. That's the best you could possibly do.
- giornogiovanna 8y ago
  How do you usually estimate the probability of a single-bit error?
  - erik_seaberg 8y ago
    Most errors are detected by checksums or hashes, so measure that across all your hardware (client requests, network hops, server RAM) and estimate how often your checksum should have collided and let an error slip by. Granted, it's pretty rare to work on a big enough system to have solid data on this.
senderista 8y ago
More precisely: to avoid collisions with high probability, random IDs need to be twice the size of sequential IDs for a fixed number of IDs generated, thanks to the birthday problem. You don’t need to “hope for the best”; the probabilities in question can be precisely quantified: https://en.wikipedia.org/wiki/Birthday_problem https://en.wikipedia.org/wiki/Birthday_problem.
- dragonwriter 8y ago
  You still need to hope for the best because probabilities that aren't zero or one aren't a guarantee of anything except for the asymptotic limit of frequency as the number of trials approaches infinity. If your system isn't resilient to collisions (and distributed ID schemes are usually the chosen to enable a system which isn't), you are gambling. Perhaps with very good odds, but gambling.

tveita 8y ago

They solve roughly the same problem. Snowflake is able to create 64-bit unique IDs at the cost of needing a central coordinating service, this format uses 128 bits for IDs which lets them be created independently as long as you have the correct time.