12 ms·
This is why you should use ids that combine both a time component and a sequence. Eg UUIDv7 has a milliseconds time component and then a field that increments
by wood_spirit 3y ago
This is why you should use ids that combine both a time component and a sequence.
Eg UUIDv7 has a milliseconds time component and then a field that increments for each event in the same millisecond, and then enough random bits to make collisions between ids generated on different machines astronomically unlikely.
Of course there are only so many bits so you might generate too many events in the same time slice so the sequence overflows, and you might actually get collisions between machines, and you are limiting your event generation speed by forcing your cpu to sync on the increment etc.
But in practice UUIDv7 works great at scale.
- darkclouds 3y ago> This is why you should use ids that combine both a time component and a sequence. Computers should run like clockwork, so in this example of using all the cores, in Windows and likely some other OS's, threads are assigned to cores when they are started and you can have many threads per core, ergo the time component should also have the thread and the core number combined with it with multi core systems. Its possible to write the code in such a way some variables are bounced out of the cpu cache back to memory to avoid any caching issues because I think the cache on some cpu's are per core, and some are per cpu.
- Joker_vD 3y ago> threads are assigned to cores when they are started Do they? I thought the normal behaviour was for cores to pick any available thread to run, so core migration is quite normal. > ergo the time component should also have the thread and the core number combined with it with multi core systems. Sorry, how exactly does it follow from the previous? You seem to have omitted the other half of your syllogism. After all, clockworks do not have thread nor core numbers so I don't quite see how having those in UUIDs will make computers run like clockwork.
- darkclouds 3y ago>> threads are assigned to cores when they are started >Do they? I thought the normal behaviour was for cores to pick any available thread to run, so core migration is quite normal. The CPU and its cores know very little about threads, threads are a figment of the OS. >Sorry, how exactly does it follow from the previous? You seem to have omitted the other half of your syllogism Syllogism would perhaps be a intuitive word to use, why is this? What happened to you?
- Mikhail_Edoshin 3y agoIsn’t it simpler to use sequence keys then?
- Cthulhu_ 3y agoYes, but only on single machines, UUID and co are intended for distributed systems. Although now I wonder if / how UUID v7 can do sequential keys on distributed systems. Mind you, on those systems "close enough" will probably be good enough, and sorting will be done by date instead of incremental ID.
- moffkalast 3y agoYou could also keep a global incremental index for that, assuming there's some authoritative server that has to be polled sequentially to get them. Probably too much overhead when conflicting UUIDs are a few orders of magnitude less likely than the clients crashing from some random bug though.
- EGreg 3y agoSo just prefix the node ID which is assigned when the node joins a swarm
- Timon3 3y agoHow is that easier or better? Now you're letting your infrastructure details bleed into your database.
- eru 3y agoWhy do you need the time component anyway? It's just eating up bits in your UUID without contributing much entropy.
- c0balt 3y agoBecause it allows te encoding of information the id. This makes it, at least in my experience, somewhat sortable.
- madsbuch 3y agoYou need it to make database indices perform better. If you don't need that, but just need a random UUID, UUIDv4 is better.
- mcv 3y agoBut shouldn't that be a separate field then?
- mattashii 3y agoThe logistics of combining fields in indexes and identifiers is relatively complex, while the logistics of indexing a single field is comparatively trivial. This is also why you don't ship timestamps using separate fields for second/minute/hour/day/month/year, but a single ISO-string or UNIX timestamp as representation: it makes querying and interpreting the value more consistent.
- frankreyes 3y agoCreate a new column with an MD5 hash of the other columns. Easy. /s
- rvnx 3y agoThis is how it works, except it's not MD5 but a less-costly hash function
- junon 3y agoDon't even need the "same millisecond" part to save a few cycles, use case depending. An overflowing increment with a counter of any sort plus a few random bits is usually enough. If you're clever enough, neither needs a branch.
- SV_BubbleTime 3y agoUUIDv7 has two at least two such monotonic strategies. You can use a counter, or you can use those bytes to increase the random entropy.
- qwerty456127 3y agoThe problem with UUIDs is they are completely unreadable. Not just you can't understand them, it is prohibitively hard to even distinguish between them visually. This is why identifiers which would not include any bit of information (or noise) beyond the necessary minimum are useful in some cases.
- SV_BubbleTime 3y agoThus UUIDv7.
- hardwaresofton 3y agoAnd they play nicely with sort ordering in popular databases like postgres! They’re not in tree yet but there are a bunch of awesome pg extensions that provide access to uuidv7 (outside of just using it at the application level)
- hinkley 3y agoI’m feeling a bit like an accidental time traveler, because I can recall a conversation at a tech meetup that had to have been at least ten years ago where someone was struggling with unique UUIDs because they were bursting above 1000 UUIDs per millisecond and not happy with the available options. How old is UUID7? I can’t get the internet to tell me.
- tredre3 3y agoUUID7 was first drafted a bit over a year ago: https://datatracker.ietf.org/doc/html/draft-peabody-dispatch-new-uuid-format https://datatracker.ietf.org/doc/html/draft-peabody-dispatch...
- hinkley 3y agoThat's kinda what I figured. So you can see my confusion.
- Shawnecy 3y agoPerhaps one of many on this list[1]? ULID is listed as having 1.21e+24 unique per millisecond. I have no idea though about what may match up with your timeline(s). [1] https://github.com/swyxio/brain/blob/master/R%20-%20Dev%20Notes/uuid%20list.md https://github.com/swyxio/brain/blob/master/R%20-%20Dev%20No...
- yohannesk 3y agoWere they not happy due to possible collisions or something else?
- hinkley 3y agoYeah they had a problem where the amortized ID rate was less than the number of valid UUIDs you could generate in a given time interval, but with a peak rate well above that, so UUIDs weren't quite going to function for naming/numbering them. You gotta be pushing a lot of messages through a system to hit that, but it's possible. And adding more layers of coordination on top of something meant to reduce coordination overhead tends to... make things messy. I've half-heartedly tried to look up his problem every time I've had to introduce UUIDs to something (most recently fixing agents not generating correlation IDs), and I have not figured out which version of UUID he was talking about. I now suspect it was some proprietary or industry-specific quote-unquote UUID spec rather than an industry wide one. I may look through the UUID7 spec at some point to see if they mention prior art. Much more plausible than him being a time traveler.