4 ms·

Hashids – Generate short hashes from numbers

53 points by yildizbe 12y ago

zrail 12y ago
Interesting. Effectively it's base62 encoding with a fixed salt. I think saying "encrypt" and "decrypt" is sort of misleading, even though you call the non-cryptographic nature of the hashing in the README files. For my own products I do something a little different. Instead, when I create a record I generate a random number (with Ruby's SecureRandom module) and store the base32 encoding in the database. With the universe set at 1bil, this reliably generates a random 6 character string that I can safely show to the customer. Edit: base62, not base82
- amvp 12y ago
  Base82? You mean base62 - 26(letters) x 2 (other case) +10 (digits)? Or am I missing something?
  - zrail 12y ago
    Nope, you're exactly right. Need more coffee.
- bazzargh 12y ago
  You're storing 6-char base-32 values, so 1073741824 possibilities, but you've got 50-50 chance of collision after 38582 values - that's not a lot, the birthday paradox bites again! Using much larger random values or hashes gives you a better safety margin.
  - zrail 12y ago
    I also have a unique constraint on the value and re-run the generator if there's a collision.
    - d55 12y ago
      That is something that hashid avoids.
- JoshTriplett 12y ago
  > this reliably generates a random 6 character string that I can safely show to the customer Are you sure about that? You might want to consider a smaller alphabet, with vowels and ambiguous letters/numerals (0/O, 1/l/I) omitted. That has two advantages: you won't accidentally generate an identifier containing meaningful words (notably profanity), and the identifiers become easier to unambiguously transcribe and interpret. And you'll still reliably get a 6-character identifier as long as you have at least 32 symbols to use.
  - zrail 12y ago
    That's a fair point. So far there hasn't been any transcribing necessary because I have the customers reply to their receipt emails if they have a problem, but that's definitely something to consider if you have to do phone support.
    - mikeash 12y ago
      It's also useful for anything where they might retype the string. (Even if you tell them to copy/paste, some people will still retype.) Not that this necessarily applies to you, but for others considering similar schemes, avoiding I/1/l and such can be really handy.
mischanix 12y ago
I mocked up something similar [1] a while ago, although operating on fixed-length integers instead of db keys and not designed with any regard for the "decryption" process. I'm curious if it's flawed in any significant way for the purpose of converting sequential ids into apparently-random distinct ids. 1. http://ideone.com/qLQHI5 http://ideone.com/qLQHI5
hcarvalhoalves 12y ago
If you need a short code the user will have to type the legibility is more important than the length, you can optimize for human input like this: https://gist.github.com/hcarvalhoalves/5330d8af36e7163d58c4 https://gist.github.com/hcarvalhoalves/5330d8af36e7163d58c4
- underwater 12y ago
  You still need to remove '5' and 'S'. I'd like a to see people use similar algorithms for mobile input. The two most important characteristics would be to be purely lowercase and to cluster numbers and letters to minimize keyboard switching.
mcao 12y ago
I actually created something really similar, except with more options to fine tune the encoding/decoding process: https://github.com/mikecao/hashkit https://github.com/mikecao/hashkit
dsl 12y ago
> use them as forgotten password hashes Please don't. The author tries to trick you into believing it is secure by including a salt. However the resulting key space of the "hash" gives you the same security as a 5-6 character alphanumeric password. A motivated attacker could enumerate all possibilities in a few hours.
- brey 12y ago
  You could use that argument to say 4-digit bank card PINs are really bad security - and you'd be right, except that they're always locked out after N attempts. Why not the same approach here? if you try to brute force a password reset, you lock out further attempts for a few minutes.
  - krallja 12y ago
    You can then use this lockout for a denial of service attack.
    - brey 12y ago
      oh no ... I am unable to change my (perfectly secure) password for ten minutes because an attacker is attempting to brute force my password reset. I'd regard that as a feature, not a bug. you don't need to lock out the entire account.
ExpiredLink 12y ago
If it is encryption and decryption why is it called 'hash'?
- d55 12y ago
  They explain that on their website.
  - deleted 12y ago
    [deleted]
  - ExpiredLink 12y ago
    Oh, I see: >> A true cryptographic hash cannot be decrypted. However, to keep things simple the word hash is used loosely to refer to the random set of characters that is generated -- like a YouTube hash. What?
underwater 12y ago
If you don't need salts you can just use the inbuilt base64 functions: var numbers = [1, 2, 3, 4, 5]; var encoded = btoa(''+numbers).replace(/=/g, ''); var decoded = atob(encoded).split(',').map(Number); console.log(encoded, decoded);
xhrpost 12y ago
I recently tried using this for a project but ended up switching. I don't really need the encryption ability. More of an issue though is that there is no way to force that the hashes are "short". If you use their example for hashing the default _id in Mongo, you'll end up with a fairly long hash. The only way I could get them short was to switch to an auto-increment starting at 1 and then they will remain short for as long as the number is small. This presented other issues and in the end I found ShortId [1] which made the whole process much simpler. - [1] https://github.com/dylang/shortid https://github.com/dylang/shortid
deleted 12y ago
[deleted]
jbaudanza 12y ago
I've used the Blowfish cipher to obfuscate database IDs into youtube-like URLs. Blowfish has a small 64-bit blocksize which fits nicely into a base64'd URL param.
kvz 12y ago
I have an old blogpost around the same idea that many people contributed versions in different languages to: http://kvz.io/blog/2009/06/10/create-short-ids-with-php-like-youtube-or-tinyurl/ http://kvz.io/blog/2009/06/10/create-short-ids-with-php-like... Hope it's useful
latchkey 12y ago
I've been using this for a while now and I'm pretty happy with it. Java implementation of Crockford's Base32 algorithm for encoding simple numeric values: https://code.google.com/p/unsuck/source/browse/trunk/src/main/java/unsuck/io/CrockfordBase32.java https://code.google.com/p/unsuck/source/browse/trunk/src/mai...
deleted 12y ago
[deleted]
- hueving 12y ago
  >For instance, suppose you issue a series of discount coupons. These could be identified by number: 1, 2, 3, 4, ... But of course anyone can fake the number, so you could disguise it with this encryption scheme. Because it's not a one-way hash, the number is recoverable from the coupon, by someone who knows the secret key. When you process the coupon, you decrypt the code, and invalidate that ID so the code can't be used again. This sounds extremely over-engineered though. You have to have a database of all of the IDs to keep track of which ones are valid, right? If so, you just randomly generate the IDs rather than serializing + encrypting. There is no point in trying to hide information in the ID when it's essentially just a primary key into a database anyway.
oh_sigh 12y ago
FYI the salt has a max size n - All salts longer than n are equivalent to using a salt of substr(salt, 0, n)
cordite 12y ago
My only complaint is that not all the implementations yield the same results.
deleted 12y ago
[deleted]

53 points by yildizbe 12y ago

zrail 12y ago

Interesting. Effectively it's base62 encoding with a fixed salt. I think saying "encrypt" and "decrypt" is sort of misleading, even though you call the non-cryptographic nature of the hashing in the README files. For my own products I do something a little different. Instead, when I create a record I generate a random number (with Ruby's SecureRandom module) and store the base32 encoding in the database. With the universe set at 1bil, this reliably generates a random 6 character string that I can safely show to the customer. Edit: base62, not base82

amvp 12y ago
Base82? You mean base62 - 26(letters) x 2 (other case) +10 (digits)? Or am I missing something?
- zrail 12y ago
  Nope, you're exactly right. Need more coffee.
bazzargh 12y ago
You're storing 6-char base-32 values, so 1073741824 possibilities, but you've got 50-50 chance of collision after 38582 values - that's not a lot, the birthday paradox bites again! Using much larger random values or hashes gives you a better safety margin.
- zrail 12y ago
  I also have a unique constraint on the value and re-run the generator if there's a collision.
  - d55 12y ago
    That is something that hashid avoids.
JoshTriplett 12y ago
> this reliably generates a random 6 character string that I can safely show to the customer Are you sure about that? You might want to consider a smaller alphabet, with vowels and ambiguous letters/numerals (0/O, 1/l/I) omitted. That has two advantages: you won't accidentally generate an identifier containing meaningful words (notably profanity), and the identifiers become easier to unambiguously transcribe and interpret. And you'll still reliably get a 6-character identifier as long as you have at least 32 symbols to use.
- zrail 12y ago
  That's a fair point. So far there hasn't been any transcribing necessary because I have the customers reply to their receipt emails if they have a problem, but that's definitely something to consider if you have to do phone support.
  - mikeash 12y ago
    It's also useful for anything where they might retype the string. (Even if you tell them to copy/paste, some people will still retype.) Not that this necessarily applies to you, but for others considering similar schemes, avoiding I/1/l and such can be really handy.

mischanix 12y ago

I mocked up something similar [1] a while ago, although operating on fixed-length integers instead of db keys and not designed with any regard for the "decryption" process. I'm curious if it's flawed in any significant way for the purpose of converting sequential ids into apparently-random distinct ids. 1. http://ideone.com/qLQHI5 http://ideone.com/qLQHI5

hcarvalhoalves 12y ago

If you need a short code the user will have to type the legibility is more important than the length, you can optimize for human input like this: https://gist.github.com/hcarvalhoalves/5330d8af36e7163d58c4 https://gist.github.com/hcarvalhoalves/5330d8af36e7163d58c4

underwater 12y ago
You still need to remove '5' and 'S'. I'd like a to see people use similar algorithms for mobile input. The two most important characteristics would be to be purely lowercase and to cluster numbers and letters to minimize keyboard switching.

mcao 12y ago

I actually created something really similar, except with more options to fine tune the encoding/decoding process: https://github.com/mikecao/hashkit https://github.com/mikecao/hashkit

dsl 12y ago

> use them as forgotten password hashes Please don't. The author tries to trick you into believing it is secure by including a salt. However the resulting key space of the "hash" gives you the same security as a 5-6 character alphanumeric password. A motivated attacker could enumerate all possibilities in a few hours.

brey 12y ago
You could use that argument to say 4-digit bank card PINs are really bad security - and you'd be right, except that they're always locked out after N attempts. Why not the same approach here? if you try to brute force a password reset, you lock out further attempts for a few minutes.
- krallja 12y ago
  You can then use this lockout for a denial of service attack.
  - brey 12y ago
    oh no ... I am unable to change my (perfectly secure) password for ten minutes because an attacker is attempting to brute force my password reset. I'd regard that as a feature, not a bug. you don't need to lock out the entire account.

ExpiredLink 12y ago

If it is encryption and decryption why is it called 'hash'?

d55 12y ago
They explain that on their website.
- deleted 12y ago
  [deleted]
- ExpiredLink 12y ago
  Oh, I see: >> A true cryptographic hash cannot be decrypted. However, to keep things simple the word hash is used loosely to refer to the random set of characters that is generated -- like a YouTube hash. What?

underwater 12y ago

If you don't need salts you can just use the inbuilt base64 functions: var numbers = [1, 2, 3, 4, 5]; var encoded = btoa(''+numbers).replace(/=/g, ''); var decoded = atob(encoded).split(',').map(Number); console.log(encoded, decoded);

xhrpost 12y ago

I recently tried using this for a project but ended up switching. I don't really need the encryption ability. More of an issue though is that there is no way to force that the hashes are "short". If you use their example for hashing the default _id in Mongo, you'll end up with a fairly long hash. The only way I could get them short was to switch to an auto-increment starting at 1 and then they will remain short for as long as the number is small. This presented other issues and in the end I found ShortId [1] which made the whole process much simpler. - [1] https://github.com/dylang/shortid https://github.com/dylang/shortid

deleted 12y ago

[deleted]

jbaudanza 12y ago

I've used the Blowfish cipher to obfuscate database IDs into youtube-like URLs. Blowfish has a small 64-bit blocksize which fits nicely into a base64'd URL param.

kvz 12y ago

I have an old blogpost around the same idea that many people contributed versions in different languages to: http://kvz.io/blog/2009/06/10/create-short-ids-with-php-like-youtube-or-tinyurl/ http://kvz.io/blog/2009/06/10/create-short-ids-with-php-like... Hope it's useful

latchkey 12y ago

I've been using this for a while now and I'm pretty happy with it. Java implementation of Crockford's Base32 algorithm for encoding simple numeric values: https://code.google.com/p/unsuck/source/browse/trunk/src/main/java/unsuck/io/CrockfordBase32.java https://code.google.com/p/unsuck/source/browse/trunk/src/mai...

deleted 12y ago

[deleted]

hueving 12y ago
>For instance, suppose you issue a series of discount coupons. These could be identified by number: 1, 2, 3, 4, ... But of course anyone can fake the number, so you could disguise it with this encryption scheme. Because it's not a one-way hash, the number is recoverable from the coupon, by someone who knows the secret key. When you process the coupon, you decrypt the code, and invalidate that ID so the code can't be used again. This sounds extremely over-engineered though. You have to have a database of all of the IDs to keep track of which ones are valid, right? If so, you just randomly generate the IDs rather than serializing + encrypting. There is no point in trying to hide information in the ID when it's essentially just a primary key into a database anyway.

oh_sigh 12y ago

FYI the salt has a max size n - All salts longer than n are equivalent to using a salt of substr(salt, 0, n)

cordite 12y ago

My only complaint is that not all the implementations yield the same results.

deleted 12y ago

[deleted]