3 ms·

I'm a fan of Cuid2[1] for this reason. They are compact, don't leak information, and make a good case why k-sortable IDs are unnecessary, or even harmful for p

by imiric 3y ago

I'm a fan of Cuid2[1] for this reason. They are compact, don't leak information, and make a good case why k-sortable IDs are unnecessary, or even harmful for performance. I'm using sequential integers and created_at/updated_at timestamps for internal use, and Cuid2 IDs externally. [1]: https://github.com/paralleldrive/cuid2 https://github.com/paralleldrive/cuid2

BiteCode_dev 3y ago
What benefice over uuid4 ?
- pfix 3y ago
  There's a comparison in the README of the project: https://github.com/paralleldrive/cuid2#the-contenders https://github.com/paralleldrive/cuid2#the-contenders Some of the arguments mentioned are explained elsewhere in the README, others are assumed. One argument standing out for me is the lack of collision-resistance for UUIDv4 which is surprising for me and I didn't spot any sources for that argument. Another argument is the entropy source where they go about that Math.random is not reliable as a single entropy source but glimpsing at the source code, they sprinkle the CUID with Math.random data. I am no expert in ID security, so I am not qualified to speak about the validity of their arguments, only that there's insufficient information to validate without prior knowledge about the problem domain.
  - forty 3y ago
    crypto.randomUUID should generate UUIDv4 with a cryptographically secure RNG (ie not math.random) Collision of UUIDV4 (which are 122 bits of entropy) are unlikely enough that it should fit most definitions of the word "impossible". The argument listed in this library README feels like total bullshit to me, I'd avoid using it for this reason alone.
- matharmin 3y ago
  Reading their docs: No real benefits, just misconceptions. 1. Collision resistance / "weak" PRGNs used to generate UUIDv4. Firstly, these are properties of the implementation, not the spec. Secondly, the source for calling the browser `Crypto.getRandomValues()` insecure is an issue that has been fixed back in 2016. I would not trust the developers of this implementation to do a better job than current browsers. 2. "Not URL or name friendly": Fair, but not very strong argument. 3. "Horizontally scalable" and "offline-capable": No argument given for why UUIDv4 does not meet these requirements, apart from point 1 above. 4. "Too fast": No argument given for why having a slower algorithm to generate random ids is more secure. Both UUIDv4 and Cuid2 use a similar number of random bits (122-124). When using a secure PRNG, both are equally difficult to guess, the SHA3 hashing doesn't add anything. You don't have to try and guess the "input" of the Cuid2 - you can just try to guess the "output" and skip the SHA3 hashing. It would be impossible to actually guess a generated ID, but UUIDv4 is just as impossible. Also no argument given for why UUIDv7 is fine but UUIDv4 is not. I've used UUIDv4 for genering unique IDs for over 10 years now. I have run into collisions, when I hand-rolled my own implementation for J2ME with major bugs many years ago - ended up with around 20 bits of entropy instead of 122. That's not a reason to not use UUIDv4, just a reason to not implement it yourself unless you really know what you're doing.
  - AtlasBarfed 3y ago
    "A couple of years ago we decided to standardize the use of sequential integer IDs as primary keys, due to the significant performance issues of non-time-ordered UUIDs." From the article. I'd like a lot more exposition on that, since it goes against some of what used to motivate UUID use in the first place. Sequential ordering across distributed nodes isn't a fun thing to do, and even if you navigate the coding, the network agreement makes it really slow. Do they mean "sequential enough" but some locality of the node that generated it? And I guess that prng can sometimes have some performance bottlenecks, but compared to doing locks on a single incremented integer? Yeah, I don't really get this, lots of usual "better" "faster" etc without actual numbers to back it up or detailed algorithmic discussions. As you basically said, wake me up if it's good enough to get into the standard vetted libraries of UUID generation.
  - wolletd 3y ago
tveita 3y ago
> But not too fast: If you can hash too quickly you can launch parallel attacks to find duplicates or break entropy-hiding. For unique ids, the fastest runner loses the security race. > Cuid2 has been audited by security experts and artificial intelligence, and is considered safe to use for use-cases like secret sharing links. I'm getting some snake oil vibes from this... There absolutely shouldn't be anything like a random ID that is 'too fast' to compute. You might need a rate limit to stay within your collision bounds, but CPU usage is a poor way to do it. And there is currently no publicly available "artificial intelligence" that would be useful in a security audit, unless you want to call fuzzers "AI".
- deleted 3y ago
  [deleted]
ndriscoll 3y ago
Nearly everything in this README about security or performance is wrong. I'd be very wary of using this.
sgarland 3y ago
The comments on performance are utterly incorrect, modulo discussions on hotspots, but you shouldn’t be sharding randomly anyway. If you get to the point where you _need_ to shard for anything other than geolocality, doing so randomly will rapidly reveal your hotspots. > One reason for using sequential keys is to avoid id fragmentation, which can require a large amount of disk space for databases with billions of records. Disk is cheap but not free at higher tiers. But more importantly, record fragmentation means more pages (unless you take the time to do a full table lock and rewrite it, and who’s doing that?) which means more index bloat. I assure you, that adds up once you’re into the billions of records level. > the ids will be generated in a sequential order, causing the tree to become unbalanced, which will lead to frequent rebalancing. Given the width of B+trees used in DBs, I doubt they generally need to go more than one or at most two levels up. I’ll take the ability to rapidly follow the leaf nodes and have a good shot at sequential reads in cache from prefetch, thanks.

by imiric 3y ago

BiteCode_dev 3y ago

What benefice over uuid4 ?

pfix 3y ago
There's a comparison in the README of the project: https://github.com/paralleldrive/cuid2#the-contenders https://github.com/paralleldrive/cuid2#the-contenders Some of the arguments mentioned are explained elsewhere in the README, others are assumed. One argument standing out for me is the lack of collision-resistance for UUIDv4 which is surprising for me and I didn't spot any sources for that argument. Another argument is the entropy source where they go about that Math.random is not reliable as a single entropy source but glimpsing at the source code, they sprinkle the CUID with Math.random data. I am no expert in ID security, so I am not qualified to speak about the validity of their arguments, only that there's insufficient information to validate without prior knowledge about the problem domain.
- forty 3y ago
  crypto.randomUUID should generate UUIDv4 with a cryptographically secure RNG (ie not math.random) Collision of UUIDV4 (which are 122 bits of entropy) are unlikely enough that it should fit most definitions of the word "impossible". The argument listed in this library README feels like total bullshit to me, I'd avoid using it for this reason alone.
matharmin 3y ago
Reading their docs: No real benefits, just misconceptions. 1. Collision resistance / "weak" PRGNs used to generate UUIDv4. Firstly, these are properties of the implementation, not the spec. Secondly, the source for calling the browser `Crypto.getRandomValues()` insecure is an issue that has been fixed back in 2016. I would not trust the developers of this implementation to do a better job than current browsers. 2. "Not URL or name friendly": Fair, but not very strong argument. 3. "Horizontally scalable" and "offline-capable": No argument given for why UUIDv4 does not meet these requirements, apart from point 1 above. 4. "Too fast": No argument given for why having a slower algorithm to generate random ids is more secure. Both UUIDv4 and Cuid2 use a similar number of random bits (122-124). When using a secure PRNG, both are equally difficult to guess, the SHA3 hashing doesn't add anything. You don't have to try and guess the "input" of the Cuid2 - you can just try to guess the "output" and skip the SHA3 hashing. It would be impossible to actually guess a generated ID, but UUIDv4 is just as impossible. Also no argument given for why UUIDv7 is fine but UUIDv4 is not. I've used UUIDv4 for genering unique IDs for over 10 years now. I have run into collisions, when I hand-rolled my own implementation for J2ME with major bugs many years ago - ended up with around 20 bits of entropy instead of 122. That's not a reason to not use UUIDv4, just a reason to not implement it yourself unless you really know what you're doing.
- AtlasBarfed 3y ago
  "A couple of years ago we decided to standardize the use of sequential integer IDs as primary keys, due to the significant performance issues of non-time-ordered UUIDs." From the article. I'd like a lot more exposition on that, since it goes against some of what used to motivate UUID use in the first place. Sequential ordering across distributed nodes isn't a fun thing to do, and even if you navigate the coding, the network agreement makes it really slow. Do they mean "sequential enough" but some locality of the node that generated it? And I guess that prng can sometimes have some performance bottlenecks, but compared to doing locks on a single incremented integer? Yeah, I don't really get this, lots of usual "better" "faster" etc without actual numbers to back it up or detailed algorithmic discussions. As you basically said, wake me up if it's good enough to get into the standard vetted libraries of UUID generation.
- wolletd 3y ago

tveita 3y ago

> But not too fast: If you can hash too quickly you can launch parallel attacks to find duplicates or break entropy-hiding. For unique ids, the fastest runner loses the security race. > Cuid2 has been audited by security experts and artificial intelligence, and is considered safe to use for use-cases like secret sharing links. I'm getting some snake oil vibes from this... There absolutely shouldn't be anything like a random ID that is 'too fast' to compute. You might need a rate limit to stay within your collision bounds, but CPU usage is a poor way to do it. And there is currently no publicly available "artificial intelligence" that would be useful in a security audit, unless you want to call fuzzers "AI".

deleted 3y ago
[deleted]

ndriscoll 3y ago

Nearly everything in this README about security or performance is wrong. I'd be very wary of using this.

sgarland 3y ago

The comments on performance are utterly incorrect, modulo discussions on hotspots, but you shouldn’t be sharding randomly anyway. If you get to the point where you _need_ to shard for anything other than geolocality, doing so randomly will rapidly reveal your hotspots. > One reason for using sequential keys is to avoid id fragmentation, which can require a large amount of disk space for databases with billions of records. Disk is cheap but not free at higher tiers. But more importantly, record fragmentation means more pages (unless you take the time to do a full table lock and rewrite it, and who’s doing that?) which means more index bloat. I assure you, that adds up once you’re into the billions of records level. > the ids will be generated in a sequential order, causing the tree to become unbalanced, which will lead to frequent rebalancing. Given the width of B+trees used in DBs, I doubt they generally need to go more than one or at most two levels up. I’ll take the ability to rapidly follow the leaf nodes and have a good shot at sequential reads in cache from prefetch, thanks.