10 ms·
This is great for internal distributed systems where having ordered keys is useful, however, it should probably be noted that these probably shouldn't be used a
by jonhohle 3y ago
This is great for internal distributed systems where having ordered keys is useful, however, it should probably be noted that these probably shouldn't be used as public identifiers (even though this will probably be the defacto standard and used publicly without thought).
Having any information, specifically time information, leaking from your systems may or may not have unanticipated security or business implications. (e.g. knowing when session tokens or accounts are created).
- 0xEFF 3y agoEvery access and id token issued by oidc already has an issued at (iat) and expiration (exp) fields.
- rtsil 3y agoDon't create them each time a record is created, create a batch in advance in sufficient number, and do the same every time the previous batch has ran out. UUIDv7 is 128 bits, you can store a large number of them without major penalty.
- Macha 3y agoBut then you need to have the client communicate with the server to identify it's newly created object or complicate your logic to have incomplete objects in a pending state, which is one of the things people were using UUIDs to avoid.
- rtsil 3y agoYou don't store the UUIDs in the database as incomplete records. You can put them in a unused_uuids table and store some of the values in memory to minimize the round-trip. You can even store them in a simple file, and remove each used UUID from that file. When the file is empty, you create a million more of them.
- Macha 3y agoThe incomplete objects refers to when someone clicks "new" in your UI. Until it's saved back to the server, that "new" object has no ID, since you need to communicate with the server somehow to get that UUID in this approach. So now the client creates objects without IDs, so now all your models need to assume IDs are optional, and you can't create your object references on unsaved objects.
- rtsil 3y agoThen we can get a random sample of UUIDs in sufficient number from the batch, send it to the client at the beginning of the session along with the other data, lock that batch until the client releases the session to avoid duplicate use, and have the client use these UUIDs until they run out, at which point it can request a new batch.
- pnpnp 3y agoThey’re also incredibly cheap to create & don’t need knowledge of each other. I mostly see batching of IDs like this when a lock is involved to prevent collisions & maintain performance. With UUIDv7, you are reasonably sure that there won’t be collisions (check your use-case first), and can just generate them wherever on-demand (no locks required). I’d argue batching IDs is actually more complicated than UUIDv7 for most use-cases.
- rtsil 3y agoThis was a solution to the UUIDv7 problem of being time-dependend and therefore may leak information. Create the UUIDv7 in advance by batch of 10,000, use these randomly, and you fix that problem.
- tabtab 3y agoPlus ID's often have to be given over the phone for support. Long or complicated ID's will drive both sides bonkers. #BeenThereDoneThat.
- BillinghamJ 3y ago>Having any information, specifically time information, leaking from your systems may or may not have unanticipated security or business implications. (e.g. knowing when session tokens or accounts are created). I don't think this is really true? These are not serially incrementing, they just indicate the time it happened. If you have an ID that you know exists, having the ability to know _when_ it was created is very rarely meaningful. What could present more of a risk is being able to predict a large part of IDs that will be created. Even then though, you shouldn't depend on your IDs for secrecy - best to ensure the IDs are never used as protection by themselves (ie treat them like they're just a simple autoincrementing number, even if they're not)
- Someone 3y agoOne business implication is that third parties can detect whether your sales increase or decrease from sampling those IDs (a variation on https://en.wikipedia.org/wiki/German_tank_problem https://en.wikipedia.org/wiki/German_tank_problem)
- adamckay 3y agoBut how? You would be correct if the ID were an integer being serially increased. I can sign up to your website today and get an ID X and then sign up again in a week and get ID Y, I can then calculate the number of new users you've had by performing Y-X. If this ID is a timestamp then there's no such information I can get out of it from a small sample. I sign up today and get todays timestamp, then I sign up next week and get next weeks timestamp..?
- WorldMaker 3y agoOne real world security problem is the "elder account" problem: as the age of an account increases the likelihood increases that it uses an insecure old password and/or that the account owner isn't paying as much attention to the account in the present. Depending on what the account represents age may also imply more "value" in the account. (Including just "sentimentality" value in the case of ransom operations, not just financial value.) Being able to tell from an ID alone that an account is at least X years older than some other ID in the system can be a handy way to find "potentially high value/low security" accounts to focus on to social engineer. There are certainly mitigations that can be made and not all things are equally valuable as they age. (Plus many public APIs include created/modified timestamps anyway. The information is often easy to discover even when not embedded in an ID.) I don't find it a strong reason to avoid timestamp-based IDs for the threat models of that many things beyond user accounts and other things susceptible for social engineering, but it is something to keep aware of.
- oconnore 3y agoGiven that a UUID identifier fits in a single cipher block, and the whole point is that these are unique by construction (no IV needed so long as that holds true), it seems like a single round of ECB-mode AES-128 would enable quickly converting between internal/external identifiers. 128 bits -> 128 bits
- dajonker 3y agoThat's an interesting idea, how would you deal with the bits in the UUID that are used for the version? Setting them to random bits may cause issues for clients that try to use the identifier in their own database or application, as mentioned in the article.
- notpushkin 3y agoIs there a way to encrypt 122 bits -> 122 bits? If so – do that and set version to 4. Alternatively, just say it's a random string ID and not an UUID.
- bajsejohannes 3y agoI'm actually working on encrypting database keys like this, and I opted for a string ID with "url safe" base64. It avoids the ambiguity of looking like a UUID when it's not, and I prefer "o3Ru98R3Qw-_x2MdiEEdSQ" in a URL over "a3746ef7-c477-430f-bfc7-631d88411d49". (Not that either one is very beautiful)
- matja 3y agoCurious on the preference there, especially when characters in base64url encoding can look similar/ambiguous in some fonts.
- bajsejohannes 3y agoThe preference is purely about the amount of characters. I hope no one will ever actually read or write these URLs, but you never know...
- andrewmatte 3y agojonhohle, thanks. Do you know of examples of when milliseconds are part of the session tokens or accounts being created has been exploited?
- tgv 3y agoThe German tank production capacity was estimated by serial numbers of captured tanks. There are ways to read all kinds of information by observing energy usage. High resolution time and sequence data undoubtedly reveal more than you’d like.
- CSDude 3y agoMost of our lives as boring SaaS etc. software developer will not be near as exciting as this, but of course you may never know. I parsed the EV chargers APIs where I live (using Frida in Android) and one of the fields returned the daily revenue and profit.
- t0mas88 3y agoCould be quite useful information to competing charger networks or in future M&A discussions. I've seen a project for a trading firm that inferred all kinds of traffic and revenue numbers for companies before their quarterly earnings were made public. It wasn't perfect, but knowing with a certain confidence level whether the numbers were going to be better or worse than estimate was profitable for them.
- kozak 3y agoHere in Ukraine trying to estimate enemy's drone and missile production capacities by serial numbers of their parts is quite a mundane task these days.
- tgv 3y agoSure, for many places it doesn't really matter, but if your URLs or user ids can be seen/scraped by others, you might expose some commercially interesting information to competitors. And the indexing argument isn't really compelling, is it? You lose very little by sticking to fully random UUIDs.
- noduerme 3y agoThinking that harder-to-guess IDs will mitigate attacks is an example of security by obscurity. It's better to think of any IDs in your database as being public knowledge, because they will leak anyway. Assuming that no one can guess another ID leads to shoddy practices. I generally keep IDs sequential and build security around the basic assumption that IDs are not keys, passwords, sessions, or secrets - they're just the public matching identifier for those things. To that end, I think it's neat to be able to improve indexing on UUIDs, but it's not a security solution.
- berkes 3y agoDoesn't that still leak (statistical) information? It may not be technically security, but e.g. knowing your competitor just added N products to their shop, might be a security issue for the business.
- hughesjj 3y agoYou're absolutely right, this is also why you generally encrypt sessionized or "consistent view" pagination tokens for public apis (save for primitives like ddb or Kafka) The end user should know no details about your internal key space.
- noduerme 3y agoIt may. Certainly, for instance, sequential invoice numbers do. If a business decides to take measures to obscure that, no problem. All I'm saying is that obscuring a numbering system for data artifacts shouldn't be considered any sort of security as far as keeping your endpoints from being hacked.
- berkes 3y agoThe point on invoice-numbers brings another issue to mind. We model our domain(s) using DDD, and often "The ID" really is best left a thing with meaning. Customer-id, Bank-account-number, invoice-number, email, etc. At least within the domain, it is. The business (and laws etc) already ensure there can only ever be one invoice with this number. Its terribly counterproductive to have two ID's for something. "Hey, can you have a look at invoice 20230233, because it seems the VAT was applied wrong. Hmm, do you have the UUID for that invoice and DM me that? You know, the long one with the hyphens". I guess there isn't a one-size-fits all solution and that "it depends" very much on what e.g. "public" means.
- imiric 3y agoI'm a fan of Cuid2[1] for this reason. They are compact, don't leak information, and make a good case why k-sortable IDs are unnecessary, or even harmful for performance. I'm using sequential integers and created_at/updated_at timestamps for internal use, and Cuid2 IDs externally. [1]: https://github.com/paralleldrive/cuid2 https://github.com/paralleldrive/cuid2
- BiteCode_dev 3y agoWhat benefice over uuid4 ?
- pfix 3y agoThere's a comparison in the README of the project: https://github.com/paralleldrive/cuid2#the-contenders https://github.com/paralleldrive/cuid2#the-contenders Some of the arguments mentioned are explained elsewhere in the README, others are assumed. One argument standing out for me is the lack of collision-resistance for UUIDv4 which is surprising for me and I didn't spot any sources for that argument. Another argument is the entropy source where they go about that Math.random is not reliable as a single entropy source but glimpsing at the source code, they sprinkle the CUID with Math.random data. I am no expert in ID security, so I am not qualified to speak about the validity of their arguments, only that there's insufficient information to validate without prior knowledge about the problem domain.
- forty 3y agocrypto.randomUUID should generate UUIDv4 with a cryptographically secure RNG (ie not math.random) Collision of UUIDV4 (which are 122 bits of entropy) are unlikely enough that it should fit most definitions of the word "impossible". The argument listed in this library README feels like total bullshit to me, I'd avoid using it for this reason alone.
- matharmin 3y agoReading their docs: No real benefits, just misconceptions. 1. Collision resistance / "weak" PRGNs used to generate UUIDv4. Firstly, these are properties of the implementation, not the spec. Secondly, the source for calling the browser `Crypto.getRandomValues()` insecure is an issue that has been fixed back in 2016. I would not trust the developers of this implementation to do a better job than current browsers. 2. "Not URL or name friendly": Fair, but not very strong argument. 3. "Horizontally scalable" and "offline-capable": No argument given for why UUIDv4 does not meet these requirements, apart from point 1 above. 4. "Too fast": No argument given for why having a slower algorithm to generate random ids is more secure. Both UUIDv4 and Cuid2 use a similar number of random bits (122-124). When using a secure PRNG, both are equally difficult to guess, the SHA3 hashing doesn't add anything. You don't have to try and guess the "input" of the Cuid2 - you can just try to guess the "output" and skip the SHA3 hashing. It would be impossible to actually guess a generated ID, but UUIDv4 is just as impossible. Also no argument given for why UUIDv7 is fine but UUIDv4 is not. I've used UUIDv4 for genering unique IDs for over 10 years now. I have run into collisions, when I hand-rolled my own implementation for J2ME with major bugs many years ago - ended up with around 20 bits of entropy instead of 122. That's not a reason to not use UUIDv4, just a reason to not implement it yourself unless you really know what you're doing.
- woile 3y agoCould you explain a bit more how it would be a risk? Maybe for session tokens is understandable. But why leaking account created info is a problem?
- logicchains 3y agoLeaking a monotonic ID could allow outside observers to estimate e.g. number of accounts created or products sold over certain timeframe. Competitors (or traders, for a public company) could use this like a form of inside information on the company (e.g. sell the stock if the rate was falling).
- gorkish 3y agoThis would really only be possible if you leaked a monotonic sequence; the monotonic clock only would potentially leak only event ordering or absolute time. IMO it's not the job of the identifier itself to prevent information leakage vulnerabilities though; if thee is sensitivity to this, the solution should be explicit, such as employing a secondary key derived from the UUID using a secure KDF or similar.
- fanf2 3y agoUUIDv7 does not leak the allocation rate of UUIDs.
- rvnx 3y agoIt’s as convenient as manipulating IPv6 addresses.
- greatgib 3y agoFyi, the timestamp is already encoded inside uuid4. But for having a good distribution of values, the low bits half of the timestamp is stored before the high bits half. Here uuidv7 will just re-order that. So the content of the uuid in itself does not change.
- jacobgorm 3y agoNot to speak of the increased risk of key collisions. https://en.wikipedia.org/wiki/Birthday_problem https://en.wikipedia.org/wiki/Birthday_problem
- hannasanarion 3y agoIn the same millisecond, in the same database system, having rolled the same 74 bit (~22 digit) number?
- jacobgorm 3y agoIf it is the same database system, you may as well use a SERIAL or similar synchronized datatype. The point of using UUIDs is that any participant in a distributed system should be able to generate them, without having to rely on a centralized authority. If the use is purely local then UUIDs would be pointless. If you have 74 bits of entropy, the birthday paradox says that after 2^37 keys you will hit a 50% risk of a key collision. Whether that is going to be a problem depends on the use case, and on the quality of your RNG.
- hannasanarion 3y agoBut the uuid collision chance only matters if there is even a possibility of colliding writes and lookups. So what if your microblogging platform's tweet uuid happens to collide with my cocktail recipe generator's ingredient uuid? The likelihood that any one device will ever be running our apps at the same time and trying to read data from one into the other is even smaller than the 1-in-380000000000000000000000000000000000000 (uuidv4) probability of having a collision in the first place.
- jacobgorm 3y agoYou're right, but why would either of us be using UUIDs for that purpose, when an auto-incrementing integer would suffice?
- 3y ago
- throwaway167 3y agoJust randomise your clock. /s Persistent IDs are a security and information risk. If that's a concern, don't persist IDs.