8 ms·
Nano ID: A tiny, secure URL-friendly unique string ID generator for JavaScript
- ijustdontcare 9y agowhat about uuid, a well researched and established standard?
- namelost 9y agoUUIDs are not secure unless they are version 4 (random) and generated from a secure randomness source.
- wruza 9y agoThe same applies for any unique ids in general.
- dchest 9y agoYes, but the holy UUID standard specifies multiple ways of generating them from sticks and stones, invented before people knew how to properly generate random bytes.
- jimktrains2 9y agoWhat do you mean by secure? Unpredictable?
- veeti 9y ago> Compact. It uses more symbols than UUID (A-Za-z0-9_~) and has the same number of unique options in just 22 symbols instead of 36.
- gvx 9y agoThis is basically UUIDv4, encoded with a url-safe variant of base64 (except UUIDv4 reserves a few bits for saying "this is a UUID, specifically version 4").
- falsedan 9y ago> `random % alphabet` is a popular mistake to make when coding an ID generator. The spread will not be even; there will be a lower chance for some symbols to appear compared to others—so it will reduce the number of tries when brute-forcing. I don't understand: is the default implementation of random in node.js flawed? Surely it should produce a uniform distribution… edit nevermind, found the link to https://gist.github.com/joepie91/7105003c3b26e65efcea63f3db82dfba https://gist.github.com/joepie91/7105003c3b26e65efcea63f3db8... just above that paragraph
- jimktrains2 9y ago0,1,2,3,4 % 3 0 => 0 1 => 1 2 => 2 3 => 0 4 => 1 So, a uniform input will produce more 0s and 1s than it will 2s.
- heavenlyblue 9y agoOnly if the alphabet size is not a multiple of two.
- jimktrains2 9y agoMore generally, only if your input is an integer multiple of your alphabet. However, there are methods of converting uniform distributions that will preserve uniformity. I'll have to find a good reference.
- heavenlyblue 9y agoWell, it's obvious for me. Issue is that the root reply seemed to have taken it as a JS implementation issue (due to math.random) as it's quite obvious why you should not do the modulus (or when you can), if you know the underlying maths. So did I, as a matter of fact. You - on the other hand had given the answer to the wrong confusion.
- AndrewCHM 9y ago
- Kiro 9y agoWhy wouldn't this be a one-liner?
- dchest 9y agoIt would be a pretty long line, containing a function wrapping crypto.randomBytes, window.crypto.getRandomValues, window.msCrypto.getRandomValues selected at runtime, and then a few lines to select characters avoiding modulo bias. Try?
- mort96 9y agoThat sounds like a pretty interesting code golf-ish challenge actually, would attempt if I had more than 3% battery on my laptop atm :(
- dchest 9y agoMy version: https://gist.github.com/dchest/751fd00ee417c947c252 https://gist.github.com/dchest/751fd00ee417c947c252
- dchest 9y ago(replying to myself as I can no longer edit:) My favorite feature in it is .entropy(n) method: when you want some fixed amount of unpredictable bits, but don't want to calculate how long the random string should be in your encoding to contain that many bits.
- bmn__ 9y agoFor years already I've been using UUID v4 (generated from a good random source) then base64url encoded, resulting in a 22 characters/bytes long identifier. The nanoid implementation does not really bring anything new to the table except having fewer SLOC (which I personally don't find important). It is also incompatible with https://tools.ietf.org/html/rfc4648#section-5 https://tools.ietf.org/html/rfc4648#section-5 for no good reason. It is noticable again and again that devs publishing to npm are not up to snuff, I wish I knew why, as this does not happen with other dynamic languages/their code repos.
- dchest 9y agoWait, why would you base64-encode UUID? You can encode 16 random bytes and not have a few bits spoiled by UUID version number. But then, there are people who'd like to use 62-character alphabet, avoiding any special characters. This package allows them do generate such strings. > devs publishing to npm are not up to snuff Indeed, there are tons of packages that use Math.random or take modulo without accounting for bias. This is why I'm glad there are new packages like nanoid appearing, created by people who know what they are doing, and this is why your negativity is unfounded.
- bmn__ 9y ago> why would you base64-encode UUID? You can encode 16 random bytes Because it's a standard data type. It's supported by the database and the data abstraction layers. I can use other implementations for creating, storing and retrieving UUIDs because they backed by the standard and are interoperable. I don't get these advantages with your ad-hoc approach. The loss of 4 bits do not hurt me. > this is why your negativity is unfounded I was criticising the needless deviation from an encoding standard. You talk about how it's good that nanoid picked the correct random algorithm. That just does not fit together.
- dchest 9y agoOK, so you use a non-standard representation of UUIDs because you can convert it to the standard representation for compatibility reasons. Fair enough, it's a good reason to do this if that's what you want. But then you complain that there's a different implementation of generating random identifier strings "not up to snuff", as if we all should settle on old crappy standards and stop creating something new. (One thing I don't understand with many developers is that they insist on doing something one way, thinking that their way is the best one and there should not exist other ways.)
- sigi45 9y agoI would still use uuid v4. Databases support this data type and reduce its storage needs. You can also compact it if it is too long for your use case or you like to save traffic.
- zepolen 9y agoBut that goes against the modern principle of reimplement everything.
- artursapek 9y agoHow are we supposed to learn anything if we never try to do it ourselves?
- rootlocus 9y agoIf you want to learn how UUIDs work, implement the standard.
- artursapek 9y agoThis isn't about UUIDs. I think the project creator in the OP was just learning lower level things like working with randomness and bytes... I don't see the problem. Experimenting is fun. Leave them alone.
- StavrosK 9y agoI wrote a Python version of this six years ago, it's been working beautifully and I use it for IDs in all my APIs: https://github.com/skorokithakis/shortuuid https://github.com/skorokithakis/shortuuid That way, elements are neither enumerable nor guessable.
- bluefox 9y agoIf you need a function that returns integers in some range [0..n) equiprobably, just write that. Why conflate a bunch of concepts and have weird limitations? Oh, and by the way, fuck HN for deleting comments.
- dfreire 9y agoYou can also check https://github.com/dylang/shortid https://github.com/dylang/shortid
- arnioxux 9y agoThis is a perfect example of what NOT to do. They reimplemented a seeded linear congruential generator for their random number generator: https://github.com/dylang/shortid/blob/master/lib/random/random-from-seed.js https://github.com/dylang/shortid/blob/master/lib/random/ran... But since LCGs are easy to solve, after seeing a few values you can solve for the seed and generate all past and future values. But you don't even need to do anything fancy for that. Since their implementation of LCG only has a state size of 233280 different values, you can just brute force it. (also means that their rng could only ever generate 233280 different numbers to begin with) Why the fuck do they have 2.5k stars!? EDIT: They already have an open issue for it but project seems unmaintained: https://github.com/dylang/shortid/issues/70 https://github.com/dylang/shortid/issues/70. Stay far far away.
- dfreire 9y agoYou're absolutely right, and this is huge...
- duke360 9y agotilde (~) doesn't looks a wise choice to me as may have special meaning in certain context, fortunately the alphabet are redefinible. i would also prefer if at least some part fo the nanoid would be machine dependent. anyway, it works :)
- magnat 9y agoString generated using default parameters has ~0.1% chance of containing four letter english profanity.
- ludicast 9y agoI'd consider that a feature not a bug :).
- Confiks 9y agoFor a toy project, I'm currently using a 64-bit integer of which the first bit is a sign bit (I'm using Postgres' bigint), the next 32-bits make a UNIX epoch timestamp in seconds, and the rest of the 31 bits make a random number. In this way, the ID is very easily sortable by time, although it's a bit of a gimmick. The scheme allows for 2 billion possible records per second, although if you consider birthday collisions, it comes down to a lot less (but I could re-insert on duplicates). I could later also use some of the random bits to inscribe extra data into the ID. The bits from the two halves are then interspersed to make sure the first half looks random, and transformed to a 11-character id, for example /^[0-9A-Za-z]{11}$/. I can reject some IDs beforehand if they match a list of bad strings. The result is receiving an id such as "Vxe61yU5Cci" or "WgOUBEO88gb".
- mort96 9y agoWhat happens in 2038? I hope you're never representing those 32 bits of UNIX timestamp as a signed integer anywhere. If you're making sure to always keep it unsigned, what happens in year 2106? EDIT: I'm not criticizing you, and a valid counter argument would be that you make absolutely sure to always store the timestamp unsigned, that it doesn't need to work until 2106 because it's a toy project, or that an overflow wouldn't really matter because you make sure to keep it unique even after an overflow, but I get scared any time storing a UNIX timestamp in 32 bits is mentioned, and am wondering if you've considered it.
- bpicolo 9y ago> but I get scared any time storing a UNIX timestamp in 32 bits is mentioned You're scared because an average app hasn't considered a problem that's 21 years off? Seems like a premature worry
- random023987 9y ago> You're scared because an average app hasn't considered a problem that's 21 years off? Seems like a premature worry 'Those who cannot remember the past are condemned to repeat it.'. --George Santayana
- bernadus_edwin 9y agoSilly question, why js not provide method generate uuid without library? I mean this is 2017. One method without param input, wont hurt anybody
- dchest 9y agoUUID is a misguided standard that should have been killed long ago if not for compatibility reasons, so I'm glad it's not in JS.
- pfooti 9y agoI really like the ulid library. Does what I want, compact representation, and lexicographically sorting ulids in time order is great. The character subset is URL friendly, so this is my go-to when I'm making nonces if I am not worried about super-double-secure prngs. https://github.com/alizain/ulid https://github.com/alizain/ulid
- danielbankhead 9y agoI designed something similar (in terms of URL-friendliness), but for distributed systems: https://github.com/AltusAero/bronze https://github.com/AltusAero/bronze It's designed for use cases where collision-resistance (both time-based [UUID1] and random [UUID4]) is valued more than the size of the id. Can be used as a module (Node.js & browser) and via CLI.
- deleted 9y ago[deleted]
- jmull 9y agoI like that this generates the ID directly by choosing random symbols. But I don't like that it's equivalent to UUID v4 but different. The difference doesn't buy anything significant over the standard. With a pretty small change this could directly generate base64 encoded UUID v4 (without going through a two-step process of generating a UUID and then encoding it: Most characters could be generated randomly from the base64 alphabet. The characters at certain indexes would need to use a restricted set of base64 characters to set certain bits to the values specified by UUID v4.
- maxpert 9y agoWait so `%` is bad but `& 63` is fine? Ain't you doing a mod as well??? It's just that you have power of two (64 in this case) that you are trying to exploit here... anyone who knows basics of computer science can tell you it's same! If it's about reducing code size I can demonstrate smaller code: module.exports = function (size) { return random(size || 22) .map(r => url[r & 63]) .join(''); }; I am always disappointed to see these kind of packages, as mentioned somewhere in comments above > It is noticable again and again that devs publishing to npm are not up to snuff, I wish I knew why, as this does not happen with other dynamic languages/their code repos. And I totally agree!
- deleted 9y ago[deleted]
- dchest 9y agoCheck the code below: var masks = [15, 31, 63, 127, 255] ... var mask = masks.find(function (i) { return i >= alphabet.length - 1 }) ... var byte = bytes[i] & mask if (alphabet[byte]) { id += alphabet[byte] Masking results in a random uniform number that is a power of two and greater or equal than the alphabet length and then indexes that are greater than alphabet length are rejected, which is a common way of avoiding modulo bias. I am always disappointed to see these kind of packages, as mentioned somewhere in comments above Because you didn't understand how it worked and assumed that it's bad?
- daveFNbuck 9y agoI believe the criticism here is about index.js, which is what all the other claims are about. var bytes = random(size) for (var i = 0; i < size; i++) { id += url[bytes[i] & 63] } This part of the code just uses an alphabet size of 64, which makes the approach actually used in the exported function identical to modding the byte by the alphabet size.
- dchest 9y agoYes, with a power-of-two alphabet length there's no bias. The poster seems to be confused about modulo bias claims, which apply to non-power-of-two alphabets, not a particular way of calculating modulus.
- loop0 9y agoHow does it compares to hashids (http://hashids.org/ http://hashids.org/)?
- qilo 9y agoHashids are reversible mappings to/from integers. Nano IDs are random strings.