9 ms·
UUIDs generally do not meet security requirements
- miohtama 11y agoUUID4 random is 122 bit of randomness: https://gist.github.com/miohtama/6e72c2458a7138599dc1 https://gist.github.com/miohtama/6e72c2458a7138599dc1
- masklinn 11y agoThe point here is that UUID4 is only as random as the underlying source of randomness. If the source of randomness is garbage your UUID4 is garbage as well. v8's Math.random() does not even remotely come close to a CSPRNG, and as a post a few days ago noted[0] it has a very, very small cycle length, so you can easily (via brute-force) find where in the cycle it was when it generated a given value, and enumerate all the UUIDs it could generate. [0] https://medium.com/@betable/tifu-by-using-math-random-f1c308c4fd9d#.7p2jjzxba https://medium.com/@betable/tifu-by-using-math-random-f1c308...
- im2w1l 11y agoC's rand() is typically poor as well. Thought it was par for the course for builtin rng to be somewhat shitty, but maybe other languages stack up better?
- masklinn 11y ago> Thought it was par for the course for builtin rng to be somewhat shitty, but maybe other languages stack up better? That seems to be pretty common, although v8's seems especially terrible. In thread linked by PhantomGremlin[0] pcwalton also points out that stronger RNG can be "detrimental" to benchmarks if they're slower, as many benchmarks ultimately bench the speed of the basic RNG (or for an other issue in the same category the speed of the basic hash function), so the incentive for builtin RNGs is to be fast and cheap, not to be good. [0] https://news.ycombinator.com/item?id=10598065 https://news.ycombinator.com/item?id=10598065
- revelation 11y agoC's rand() (rather, your libcs rand()) is perfectly fine for what it's supposed to be: fast random data, with no particular guarantees to cryptographic security, for things like randomized algorithms or the 99% of cases where you're picking what powerup to spawn instead of generating super important key material. To make it into a CSPRNG would be a detriment. The only thing I would change is to have a minimum cycle length.
- adrianN 11y agoNo, C's rand is unsuited for most randomized algorithms. MCMC algorithms won't work with rand unless you're lucky, it's even unsuited for choosing pivots for quicksort. The only application where it might be ok is games.
- duaneb 11y ago> The only application where it might be ok is games. This is silly, there are plenty of ways randomness is used to communicate randomness to the user (think games, song shuffling, visualizations, art, etc. Algorithms using randomness ≠ randomized algorithms.
- masklinn 11y ago> The only application where it might be ok is games. Not even that. Different types of games require different types of RNGs (e.g. in an RPG or strategy game you probably want a stable seeded RNG to preclude RNG save-scumming), the C standard requires almost no guarantee of rand().
- jjuhl 11y agoLuckily we have <random> in modern C++ these days. While they unfortunately left the default as implementation defined you can still manualy pick a good PRNG and random_device should give you good CSPRNG quality entropy in any sane implementation (although I wish they would formally say that it couldn't be based on a PRNG). Also worth checking out: https://channel9.msdn.com/Events/GoingNative/2013/rand-Considered-Harmful https://channel9.msdn.com/Events/GoingNative/2013/rand-Consi...
- deleted 11y ago[deleted]
- PhantomGremlin 11y agoHere's the link to the 219 post HN discussion of that Medium post: https://news.ycombinator.com/item?id=10598065 https://news.ycombinator.com/item?id=10598065
- mAritz 11y agoFor anyone else wondering about their node.js code: node-uuid uses nodes crypto.randomBytes: https://github.com/broofa/node-uuid/blob/master/uuid.js#L59 https://github.com/broofa/node-uuid/blob/master/uuid.js#L59
- elithrar 11y agoThe problem highlighted here is not so much UUIDs, but instead the use of poor PRNGs (i.e. things other than /dev/urandom or getrandom(2) on Linux). All of the issues with the example libraries are around of the use of PRNGs from math libs, not crypto libs. A "secure token generation" library might have the same flaws (and many do!). A correctly generated v4 UUID (S4.4 of the RFC) should be acceptable for use as a secure token, given it has 122 random bits. The trouble with UUIDs, if any, is that only v4 UUIDs are really suitable for use as secure tokens, with v1 through v3 being entirely unsuitable. Personally, outside of DB PKs in Postgres (or similar), I prefer to base64 encode 256-bits from /dev/urandom for tokens, as they're a little shorter (and Go makes it easy enough to do that).
- netheril96 11y agoBoost.UUID suffers from the same problem, at least at the time I checked.
- xnyhps 11y ago> A correctly generated v4 UUID (S4.4 of the RFC) should be acceptable for use as a secure token, given it has 122 random bits. No it's not, from §4.4: > Set all the other bits to randomly (or pseudo-randomly) chosen values. I can use a PRNG with a cycle length of 8 and it would be fully correct according to the RFC, but it would be trivial to brute-force all the values.
- viraptor 11y agoKind of... you're technically right and and practically not. Nobody would describe a PRNG with cycle length of 8 as random. It's just reading numbers off of a list. In the same section of uuid RFC: "See Section 4.5 for a discussion on random numbers.", and then: "Advice on generating cryptographic-quality random numbers can be found in RFC1750 [5].", so it's not like authors were unaware of the implications of trivial PRNG. It's just up to the implementation whether CSPRNG is required (user-visible security tokens), or just a PRNG (internal identifiers).
- Natanael_L 11y ago
- viraptor 11y agoFortunately Python is safe in this case. https://hg.python.org/cpython/file/2.7/Lib/uuid.py#l582 https://hg.python.org/cpython/file/2.7/Lib/uuid.py#l582 - urandom is the source for the UUID4. (same for 3.5: https://hg.python.org/cpython/file/3.5/Lib/uuid.py#l600 https://hg.python.org/cpython/file/3.5/Lib/uuid.py#l600)
- methyl 11y agoRuby is also safe: it uses OpenSSL/urandom for UUID: https://github.com/sj26/ruby-1.9.3-p0/blob/master/lib/securerandom.rb#L59 https://github.com/sj26/ruby-1.9.3-p0/blob/master/lib/secure... https://github.com/sj26/ruby-1.9.3-p0/blob/master/lib/securerandom.rb#L246 https://github.com/sj26/ruby-1.9.3-p0/blob/master/lib/secure...
- pkolaczk 11y agoJava is safe as well: http://docs.oracle.com/javase/7/docs/api/java/util/UUID.html#randomUUID%28%29 http://docs.oracle.com/javase/7/docs/api/java/util/UUID.html...
- netheril96 11y agoThis kind of problem is so prevalent that I think directly creating an 128-bit random ID from a CSPRNG is much safer than betting on the saneness of UUID implementers.
- lmm 11y agoThere's nothing insane about using a faster nonsecure PRNG for a library function that you expect to possibly be used on a hot path, as long as the security properties are clearly documented.
- marcosdumay 11y agoCSPRNGs are not much slower than the alternatives, and really, how often the hot path on data creation falls on the processor, instead of memory access or IO? Nope, looks like a really bad choice for generic libraries.
- masklinn 11y agoWell if you just get 122 random bits from a CSPRNG you get an UUIDv4.
- SamReidHughes 11y agoEven if you're only concerned about making unique identifiers, that don't need to be secret, that merely need to be unique, hiding the UUID generation state is a very good idea if users might have the ability to cause new UUID generators to be created. Otherwise, instead of a 2^(128/2) = 2^64 birthday attack, they can make 2^N generators, find the one that overflows into the next the quickest (i.e. a birthday attack on the 2N msb's), and have 2^(128-2N) work to do. This means you could do a 2^36-sized birthday attack and then have 2^56 work left to do, and that part's not memory-bound. For example, this works if you have generators that seed a 128-bit counter with /dev/urandom, and then increment from there. You can avoid this if you "just" use some CSPRNG.
- StavrosK 11y agoI wrote a fairly popular Python library to easily generate short UUIDs (encoded with base57): https://github.com/stochastic-technologies/shortuuid https://github.com/stochastic-technologies/shortuuid Since it turned out that most uses didn't actually need a UUID and just needed a random string, I figured I'd maximize the randomness per bit and added a .random() call that returns a short string that is taken from os.urandom(). It would probably be good if you asked yourself whether you actually need a UUID or a random string, and used the right tool for the job. Using a UUID when you want a random string and vice versa leads to these kinds of problems.
- RyanZAG 11y agoShouldn't this be 'Javascript UUIDs generally do not meet security requirements' ? As he mentions in his story, the server was actually using Java UUIDs which are cryptographically secure. So 'UUIDs generally do not meet security requirements' is false, as generally here only applies to Javascript. Python, Ruby, Java all have cryptographically safe UUID.
- Lazare 11y ago> Shouldn't this be 'Javascript UUIDs generally do not meet security requirements' ? Nothing wrong with v4 UUIDs generated in Javascript, so long as you don't use Math.random(). Every current browser supports, eg, crypto.getRandomValues(), which is cryptographically secure. If you use a broken PRNG in any other language, you'll get broken v4 UUIDs too. :) (Also, don't make the common mistake of conflating "UUID" with "v4 UUID". A v4 UUID contains 122 bits of (hopefully) random data. A v1 UUID contains 0 bits of randomness.)
- sneak 11y agohttps://twitter.com/mjmalone/status/667429857165488130 https://twitter.com/mjmalone/status/667429857165488130 The PRNG behind Math.random() has been fixed in Chrome very recently.
- Filligree 11y agoFixed? As far as I know, there are no CSPRNGs that are as fast as, say, mersenne twister. So if 'fixed' means 'made it a CSPRNG', then I'd have to say they broke it. crypto.getRandomValues already exists.
- TillE 11y agoHow fast do most users of a PRNG really need it do be? I think it's not unreasonable to pick something like ChaCha20 as a default PRNG (cryptographically secure, huge seed space, much smaller state than MT19937), and let people who need millions of random numbers for simulations use something else.
- mmalone 11y agoThe algorithm that V8 chose to replace the current generator, xorshift128+, passes BigCrush and can produce a random uint64 in just over 1 nanosecond on a modern processor (that's ~7GB/s). Seems like a good choice. Not sure how it compares to something like ChaCha20 on performance, but I'm guessing it compares favorably (though it's unlikely to be your bottleneck either way). Since I wrote the above linked article there's been a bunch of discussion, some of which has been about using crypto to back Math.random(). I'm sort of torn on that front - I feel like a good PRNG is useful for some stuff (like array shuffles), but maybe not? Maybe there are undiscovered (or even discovered, but not well known) vulnerabilities that justify CSPRNG even there, as there are with hash collision vulnerabilities. Anyways, what I learned is that the benchmarks (particularly SunSpider, it seems) are putting pressure on implementors to (over)emphasize Math.random() performance, but nothing is really pressuring them to produce good quality. Sounds like the best thing to do might be to put some quality requirements in the ECMA spec to balance the performance pressures. Check out my recent Twitter likes for more details. I'm @mjmalone.
- castell 11y agoWhen do you use UUIDs and why? I use traditional auto increment integer IDs as primary keys in SQL databases. I use those SQL query results in application code. Debugging seems a lot easier with smaller integer values rather than UUIDs that I saw in several enterprise software and SQL-DBs.
- the_mitsuhiko 11y ago> When do you use UUIDs and why? When you need the client to determine the PK before an operation happens. In particularly necessary when working with distributed environments. There are many different forms of UUID and when you know what you are doing you can build very powerful systems with them that you cannot do with auto incrementing integer keys (even with holes).
- vox_mollis 11y agoIn distributed systems, you can easily deal with this by assigning each client node an ordinal, and the client generates a sequential id that is a multiple of the highest ordinal of the ring added to the client's local ordinal. You can synchronize ordinal values between nodes with something like zookeeper or etcd.
- lostcolony 11y agoIf you're having to synchronize it using a strongly consistent mechanism, couldn't you just use an auto-incrementing integer anyway (not meaning to be snarky here - I legitimately don't know)? The benefit of a UUID is you don't need strong consistency; even on a partition of < n+1 nodes you can still generate IDs, and even if the network is unpartitioned, you can generate an ID without making a roundtrip. I would also debate your use of the word 'easily'; what's easier, setting up zookeeper/etcd, or just generating a UUID? But yes, if your use case is "I want to make sure I have non-conflicting IDs across my cluster", synchronizing them is a possible solution. And the right one, if your requirements stipulate absolute ordering based on the generation of each ID.
- rb12345 11y agoActually, this may be even worse than described. The UUID digits all amount to the hex digit of hi in the 163 place (i.e. the 4th least-significant nybble). However, it also turns out that hi[init]*0xFFFF will give the same digit sequence but with a leading 0. This probably can be used to determine the RNG state from a single UUID, but the calculations for that are beyond me.