5 ms·

I wonder how this might compare to just storing regular autoincrementing ints in the database, and converting to/from hashids (https://hashids.org/ https://hash

by mnutt 4y ago

I wonder how this might compare to just storing regular autoincrementing ints in the database, and converting to/from hashids (https://hashids.org/ https://hashids.org/) at the edge. It eliminates the collision concern and stores more compactly at the cost of a tiny amount of encode/decode when processing requests. You’d want to push it down as close to the database layer as possible to avoid inadvertent int ID leaks; I added native hashids support to clickhouse but I’m not sure what other database support might entail.

Vt71fcAqt7 4y ago
Aren't INTs cumbersome in a distributed database? For example If I have two servers running one database, now I have to keep the autoincrement in sync between both before I generate a new one. That's why UUIDs are generally used here if I understand correctly.
- charcircuit 4y ago
  You don't need to keep them in sync. You can create a partition for each server to use.
  - Vt71fcAqt7 4y ago
    Right, but now you've added more complexity. So, true, autoint doesn't fail at multiple servers, but it becomes a hassle. Is there a usecase of having ordered IDs in the first place?
- mnutt 4y ago
  Yes, if you outgrow autoincremented ints then hashids may not be a great fit I think?
charcircuit 4y ago
Those are not cryptographically secure. It would not be hard for someone to figure out how to decode it.
- chrismorgan 4y ago
  Hashids is worse than you think, and should be treated as easily-reversed obfuscation only. It doesn’t encrypt IDs, but instead shuffles the alphabet. When encoding an number, it rotates the alphabet by that number, recording that as the first character of the output, and then converts the number to a string using this rotated alphabet. This is so bad it’s basically negligence. (In its early days, it made claims of security that seem to me bald-faced lies, or staggering and fairly implausible incompetence.) In its default configuration, 44 sequential IDs gives you the key to decode all IDs, and that’s not the only way of breaking it.
  - mnutt 4y ago
    I don’t know what they said before, but they now say: “Do you have a question or comment that involves "security" and "hashids" in the same sentence? Don't use Hashids.” Still, I can see the temptation that these generated strings “look random so people can’t guess them”. Possibly good enough to obfuscate the number of db records from the casual onlooker but not good protection against enumerating accessible records.

by mnutt 4y ago

Vt71fcAqt7 4y ago

Aren't INTs cumbersome in a distributed database? For example If I have two servers running one database, now I have to keep the autoincrement in sync between both before I generate a new one. That's why UUIDs are generally used here if I understand correctly.

charcircuit 4y ago
You don't need to keep them in sync. You can create a partition for each server to use.
- Vt71fcAqt7 4y ago
  Right, but now you've added more complexity. So, true, autoint doesn't fail at multiple servers, but it becomes a hassle. Is there a usecase of having ordered IDs in the first place?
mnutt 4y ago
Yes, if you outgrow autoincremented ints then hashids may not be a great fit I think?

charcircuit 4y ago

Those are not cryptographically secure. It would not be hard for someone to figure out how to decode it.

chrismorgan 4y ago
Hashids is worse than you think, and should be treated as easily-reversed obfuscation only. It doesn’t encrypt IDs, but instead shuffles the alphabet. When encoding an number, it rotates the alphabet by that number, recording that as the first character of the output, and then converts the number to a string using this rotated alphabet. This is so bad it’s basically negligence. (In its early days, it made claims of security that seem to me bald-faced lies, or staggering and fairly implausible incompetence.) In its default configuration, 44 sequential IDs gives you the key to decode all IDs, and that’s not the only way of breaking it.
- mnutt 4y ago
  I don’t know what they said before, but they now say: “Do you have a question or comment that involves "security" and "hashids" in the same sentence? Don't use Hashids.” Still, I can see the temptation that these generated strings “look random so people can’t guess them”. Possibly good enough to obfuscate the number of db records from the casual onlooker but not good protection against enumerating accessible records.