4 ms·

Won't the resulting data look random too? In that case I see little benefit of your approach over an UUID as a PK or using a sequential integer as a PK and addi

by dtech 6y ago

Won't the resulting data look random too? In that case I see little benefit of your approach over an UUID as a PK or using a sequential integer as a PK and adding a UUID field with a btree index.

dasyatidprime 6y ago
You don't store the scrambled key, you store the sequential-looking key and expose the scrambled key externally. That avoids things like easy key-sequence probing/inference from the outside, but means you don't take the hit of an index with random-looking keys. I've done something similar for weakly-secured spendable tokens in the past, packing some orderly lookup state into 64 bits and then exposing the Speck64/32. That application was actually backed by an object-array sort of storage with numeric indices, so I stored something like object ID, timestamp, and sequence number, and then looked up the right property on the object at validation time. (If I needed strongly-in-the-crypto-sense unguessable IDs for security, I wouldn't do this without doing more research/analysis, since a database-wide key feels like it could be a weak point; compare how peppering is used for password storage in conjunction with salting but not as a replacement. Someone else probably already knows that answer.) (Edit to add cross-reference: ah, looks like tmpyt in another comment claims that YouTube used to do this but then switched away because of the “knowing the small key lets you enumerate everything” loophole.)
- dasyatidprime 6y ago
  Second additional thought: if you need strong unguessability, you could probably tack it onto that scheme by adding a second column that isn't indexed, but only used as a bearer-token-like add-on to the ID. So the index lookup only happens on the (unscrambled) first part, but the access is aborted if the second part doesn't match.

4 ms·

Won't the resulting data look random too? In that case I see little benefit of your approach over an UUID as a PK or using a sequential integer as a PK and addi

by dtech 6y ago

Won't the resulting data look random too? In that case I see little benefit of your approach over an UUID as a PK or using a sequential integer as a PK and adding a UUID field with a btree index.

dasyatidprime 6y ago
You don't store the scrambled key, you store the sequential-looking key and expose the scrambled key externally. That avoids things like easy key-sequence probing/inference from the outside, but means you don't take the hit of an index with random-looking keys. I've done something similar for weakly-secured spendable tokens in the past, packing some orderly lookup state into 64 bits and then exposing the Speck64/32. That application was actually backed by an object-array sort of storage with numeric indices, so I stored something like object ID, timestamp, and sequence number, and then looked up the right property on the object at validation time. (If I needed strongly-in-the-crypto-sense unguessable IDs for security, I wouldn't do this without doing more research/analysis, since a database-wide key feels like it could be a weak point; compare how peppering is used for password storage in conjunction with salting but not as a replacement. Someone else probably already knows that answer.) (Edit to add cross-reference: ah, looks like tmpyt in another comment claims that YouTube used to do this but then switched away because of the “knowing the small key lets you enumerate everything” loophole.)
- dasyatidprime 6y ago
  Second additional thought: if you need strong unguessability, you could probably tack it onto that scheme by adding a second column that isn't indexed, but only used as a bearer-token-like add-on to the ID. So the index lookup only happens on the (unscrambled) first part, but the access is aborted if the second part doesn't match.