22 ms·

Nanosecond timestamp collisions are common

316 points by ingve 3y ago

jfax 3y ago
This was realised in the development of imageboard software (futaba.php, later yotsuba.php that runs 4chan) in the naming images, which is a unix timestamp + random 3 digit number because just the timestamp often causes clashes.
JacobSeated 3y ago
Timestamps should probably never be used as a "unique" id.
- 3cats-in-a-coat 3y ago
  The problem is achieving locality in cohesion/meaning, which usually involves locality in time, which provokes using timestamps as part of the id at the very least. But it's a chain of very lazy thinking IMHO and it's like a peek at the house of cards some large systems are built like.
  - eru 3y ago
    Why do you want this kind of locality anyway?
    - traviscj 3y ago
      Database indices.
  - Mawr 3y ago
    Would a Snowflake ID work for this? https://en.wikipedia.org/wiki/Snowflake_ID https://en.wikipedia.org/wiki/Snowflake_ID
vbezhenar 3y ago
If you need unique nanosecond, keep track of the previously generated one and increase it if necessary. Would require global lock or atomic stuff, but should be good enough for practical uses.
- pyrale 3y ago
  One benefit of UUID is that you don't need coordination. In coordinated systems, unique IDs are a non-issue.
  - OJFord 3y ago
    Is there any other benefit? That's the raison d'être - Universally.
    - eru 3y ago
      Depends on what you compare them with.
- buster 3y ago
  If you do that, just take some redis server and increment an integer every time you need a new number?
  - marcus_holmes 3y ago
    or y'know, use the thing that was designed specifically for this case: UUID's
    - hnlmorg 3y ago
      UUIDs are designed to solve a slightly different, albeit related, problem: where you don’t have synchronisation of the system as a whole. The solution the GP is solving is for systems that are synchronised and therefore generating a UUID is additional and unnecessary overhead.
- 3y ago
getmeinrn 3y ago
I was going to post about "use a UUID", but I was surprised to learn that no UUID uses both timestamp + a random component. You can either get fully random with UUID4, or have a time + MAC based UUID with UUID1. Strange, I would have thought there would exist a UUID that uses time + random to minimize collisions like described in the post.
- wood_spirit 3y ago
  the new UUIDv6, 7 and 8 work the way you deacribe.
  - dtech 3y ago
    Only v6 and v7, v8 is "whatever you want".
    - mcv 3y ago
      That's not much of a standard.
      - throwawaymobule 3y ago
        From my reading, it's standardized because it has a version number embedded, so it won't get confused with other uuids. Probably a good idea to have somewhere to silo bespoke implementations. Just gotta hope they end up being unique.
      - dtech 3y ago
        It's not uncommon to provide a way how to do it in standards because it will happen whether you want it or not. See e.g. HTTP headers starting with X- or MIME types in application/vnd.
- nabogh 3y ago
  Can't find much info about it but is this UUID v7?
wood_spirit 3y ago
This is why you should use ids that combine both a time component and a sequence. Eg UUIDv7 has a milliseconds time component and then a field that increments for each event in the same millisecond, and then enough random bits to make collisions between ids generated on different machines astronomically unlikely. Of course there are only so many bits so you might generate too many events in the same time slice so the sequence overflows, and you might actually get collisions between machines, and you are limiting your event generation speed by forcing your cpu to sync on the increment etc. But in practice UUIDv7 works great at scale.
- darkclouds 3y ago
  > This is why you should use ids that combine both a time component and a sequence. Computers should run like clockwork, so in this example of using all the cores, in Windows and likely some other OS's, threads are assigned to cores when they are started and you can have many threads per core, ergo the time component should also have the thread and the core number combined with it with multi core systems. Its possible to write the code in such a way some variables are bounced out of the cpu cache back to memory to avoid any caching issues because I think the cache on some cpu's are per core, and some are per cpu.
  - Joker_vD 3y ago
    > threads are assigned to cores when they are started Do they? I thought the normal behaviour was for cores to pick any available thread to run, so core migration is quite normal. > ergo the time component should also have the thread and the core number combined with it with multi core systems. Sorry, how exactly does it follow from the previous? You seem to have omitted the other half of your syllogism. After all, clockworks do not have thread nor core numbers so I don't quite see how having those in UUIDs will make computers run like clockwork.
    - darkclouds 3y ago
      >> threads are assigned to cores when they are started >Do they? I thought the normal behaviour was for cores to pick any available thread to run, so core migration is quite normal. The CPU and its cores know very little about threads, threads are a figment of the OS. >Sorry, how exactly does it follow from the previous? You seem to have omitted the other half of your syllogism Syllogism would perhaps be a intuitive word to use, why is this? What happened to you?
grose 3y ago
This is why it scares me a bit to use a raw timestamp as a sort key in DynamoDB. I append a (random) unique ID to the timestamp text to avoid it. Better safe than sorry, I figure.
- fer 3y ago
  I have a use case that uses a similar solution, but for a different reason: it doesn't scare me to use timestamp as sort key I since I know my hash keys are unique and I only write every few seconds. But I still add random amount of ms (well under the frequency of writes/updates), because otherwise I'll be hitting hot partition issues on the GSI (indexed by timestamp, as you can guess).
- moduspol 3y ago
  We ran into this, though we were using millisecond timestamps with the Number type. Ended up working around it by adding numbers after the decimal point. DynamoDB apparently supports up to 38 digits of precision, and JavaScript (we're NodeJS based) by default encodes numbers to allow for up to 16 digits (combined, before and after the decimal point). A UNIX epoch millisecond timestamp is 13 digits, so we were able to use 3 just to add uniqueness without having to update anything else in our codebase. We certainly now plan to essentially always use string-based (and generically named) partition and sort keys, which would allow us to more easily do what you're describing.
techNoob123 3y ago
At some point doesn’t this come down to the ISA? A CPU running at 3GHz gets 3 clock cycles per nanosecond. I bet there is a fair amount of optimization in the compiler that leads to back to back assembly calls of reading the clock register. If subsequent time.Now() calls happen within 3 clock cycles of each other, can you really fairly expect unique nanosecond precision…
- rwmj 3y ago
  Linux will (x86_64) use RDTSC and adjust that against a value read from the VDSO, so it can indeed happen very quickly.
- dan-robertson 3y ago
  It takes like 20 cycles to read the cycle count register on modern chips. Even if collisions are somewhat unlikely, it’s still much worse to get several a day than ‘almost never going to happen’.
zokier 3y ago
I guess I'm old, the macOS behaviour is more in line with my expectations. But this got me thinking, how feasible it would be to tie the various clock systems of a computer to some reference clock, like 10 MHz GPSDO? Obviously it wouldn't improve the granularity, but you could ensure that the timestamps are actually accurate. Because otherwise I doubt that random computer clock would be accurate down to 32ns even with NTP.
- mgaunard 3y ago
  Getting a 10MHz PPS signal requires specialized expensive hardware and typically doesn't scale well to cover every server. That sort of thing is best left to extreme applications with FPGAs or ASICs. In particular the 10MHz version; a lot of commodity hardware only supports the 1Hz one. There's still an in-between of PPS and NTP: PTP.
  - mlichvar 3y ago
    A major problem in synchronization of the system clock is PCIe. Hardware can timestamp PPS signal or PTP/NTP packets with accuracy of a few nanoseconds if everything is well compensated, but the PCIe bus between the CPU and the timestamping HW has a latency of hundreds of nanoseconds with potentially large asymmetry, degrading the accuracy significantly. Measuring that error is difficult. A possibility is to run a program periodically making random reads from memory (avoiding the CPU cache) to generate a PPS signal on the memory bus, which can be observed with a scope. There is a lot of noise due to other memory activity, RAM refresh, etc. From the configured memory speed and tRCD+tCL timings the uncertainty of the error can be reduced. This might improve with new hardware. There is a feature called Precision Time Measurement (PTM), which is a hardware implementation of an NTP-like protocol in PCIe, but so far I have seen this working only on some onboard NICs.
LAC-Tech 3y ago
A lot of mention of UUDv7 in this thread which is good. But I also wonder what the collision rate for Ulids are.
- 3cats-in-a-coat 3y ago
  I frankly don't understand how it's good. UUID originally was intended as something you use very sparingly, to name, say, a product SKU maybe, an organization, something like that. Not literally content that collides commonly at the same nanosecond, in the same application, in the same platform/org. At some point we have to question the sanity of using one single flat address space for everything from the tiniest identifier to the... well, "Universe", as if it makes sense. We can have registrars for global ids, and we can nest local ids inside them, we can have a hierarchy, and we can have local compact ids, 32, 64 or 128 bit, which will never collide, and be locally cohesive. So why aren't we doing this? Is it ignorance? Is it because magic is easier than actually figuring out the synchronization and order of things in a system like this (and no, synchronization does NOT imply you need to issue ids strictly linearly). Honestly I'm at a loss of words.
  - creatonez 3y ago
    > I frankly don't understand how it's good. UUID originally was intended as something you use very sparingly, to name, say, a product SKU maybe, an organization, something like that. Not literally content that collides commonly at the same nanosecond, in the same application, in the same platform/org. Wikipedia gives a different history, that it was originally for networked computers - https://en.wikipedia.org/wiki/Universally_unique_identifier#History https://en.wikipedia.org/wiki/Universally_unique_identifier#...
  - RicardoLuis0 3y ago
    > So why aren't we doing this? isn't IPv6 is basically that? just restricted to internet addresses
    - 3cats-in-a-coat 3y ago
p-e-w 3y ago
If you want unique identifiers, use version 4 (random) UUIDs. Problem solved. The probability of a collision is roughly the same as the probability of a fully grown dinosaur spontaneously manifesting in your bedroom due to quantum fluctuations.
- jraph 3y ago
  I will take my chance :-) More seriously, If you can use them, good old increments are probably best. They are fast and cheap. Especially in a database. They can have privacy/security issues (you could guess things by the values of ids of stuff). UUIDs are better in those case or when you deal with a distributed system.
  - p-e-w 3y ago
    Unless the RNG itself is the problem, there's no reason not to. All kinds of civilization-ending catastrophes are vastly more likely than a collision in a space of size 2^122.
  - p-e-w 3y ago
    > They can have privacy/security issues (you could guess things by the values of ids of stuff). Push them through a secure hash function, and that problem is solved too (assuming you can keep the base counter private).
    - cacheyourdreams 3y ago
      If you're going to do that then you might as well just use UUID, since you effectively reintroduce the negative aspects of that (infinitesimally miniscule chance of collisions, computation involved in the calculation, etc.)
      - p-e-w 3y ago
        The difference is that you can still use sequential IDs internally, while exposing hashed IDs to the outside. This protects your database from collisions under all circumstances, while in the absolute worst case, a single user might experience bugs because two external IDs collide.
forrestthewoods 3y ago
> On my system, the minimum increment was 32 ns. Then you aren’t using a nanosecond clock. If you want actual nanosecond precision then you probably want rdtsc.
efitz 3y ago
Related I used to be the program manager owner of the security event log in windows. When things happen simultaneously or very closely in time on multi-core systems, thread scheduling can significantly affect observations of those things. For example, your thread quantum might expire before you get to the syscall to get a time stamp, or before you can pass a buffer to queue your event for subsequent time stamping. In fact, on multiprocessing systems, it was very common to see out of order event log entries on Windows back in the day (2000-oughts). You also could not count on the log timestamp accuracy too precisely; 1s was pretty much the safe lower bound (some components truncated or rounded time stamps IIRC).
godelski 3y ago
Despite the resolution being nanoseconds, what is the actual precision of computer clocks? I can't imagine it is actually nanoseconds. Takes me back to teaching physics labs where I had to hound students to remember that the accuracy of their measuring device is not identical to the smallest number it displays...
- ajb 3y ago
  For devices clocked above 1Ghz it's perfectly possible for the clock to increment every ns, although that doesn't make it accurate to that level, and multi core systems may have clocks that are not synchronised to that level. ARMv8 guarantees that it's clock increments at at least 1Ghz, for intel and earlier ARM it's more complicated
  - zokier 3y ago
    Cycle counting is one thing, but it gets tricky when you have frequency scaling in play. Another problem that even without freq scaling, cpu clocks are not designed to be super accurate/exact, and the true frequency might vary significantly even if the nominal freq is fixed
    - daniel-cussen 3y ago
      It doesn't matter that much they're not super accurate exact, they do in fact count ticks n tocks j dandy. I wonder if they are counted equally, interesting question if i do say so myself about my own question, i know rising edge is steady w the next rising edge, n falling edge w falling edge, but yeah...no i think that's a specification figure of merit, that they be balanced w respect to each other. N Intel® despite it's many failures, long ago forecast n long overdue due to the sheer difficulty of their business model n how long they kept it going--they were saying it was going to end soon in the late seventies already--n you know what, i'm fine w that. They don't make old chips rot like other software companies i could mention, bits rot but glass is timeless. Which brings me back to the point, in my analysis the problem is not the clocks being inaccurate but rather the jitter, which means a single run will not suffice in describing a repeatable exact clock time taken for eg an inner loop, which is what is worth bothering with. The minimum jitter attainable currently is 1 cycle, n then i guess you run the same code repeatedly n take the minimum with more repeatability as a consequence of that low jitter. In the early nineties it was not so, you'd get the same number of clock cycles again n again n again. N then it gets tricky because cycle counts are thresholds set within which, if voltages n frequency are proper, the operation will complete deterministically w a soft error rate no greater than the system as a whole, about one per 30 years of hammering on the chip at sea level. Which is not enough for my tastes, n the jitter is a fucking mess. I much prefer the GA144, least jitter of any platform bar none, sensible because it is fully asynchronous logic, no clock anywhere in sight until you connect it to an oscillator, n even then the vibration doesn't rock the system like that of a grandfather clock synchs w another grandfather clock with whose pendulum's swing the former clock's pendulum swing is aligned. GA144 it's pretty easy to tell average case complexity of a function down to the tens of picoseconds, at which point you have to check that there's no open window letting a draft in. In fact the time trial will tell you such a draft is coming into the house, it happened to me, while not from a parallel universe in spacial dimensions it is by all means from another universe in the time dimension.
quickthrower2 3y ago
Another case of precision but not accuracy.
qwerty456127 3y ago
> it is unsafe to assume that a raw nanosecond timestamp is a unique identifier It is reasonable to assume that a raw nanosecond timestamp is a unique identifier in cases when you can reasonably expect such identifiers are not generated too offten. I.e. this is Ok (and even redundant) for identifying manual interactions a single user would pdosuce. It probably is also Ok for labelling manual interactions multiple users would produce if that's a small team.
- aaron695 3y ago
  [dead]
tsenart 3y ago
Our Go ULID package has millisecond precision + monotonic random bytes for disambiguation while preserving ordering within the same millisecond. https://github.com/oklog/ulid https://github.com/oklog/ulid
donatj 3y ago
Slacks API uses straight nanoseconds for the IDs of their messages which I always found very curious but figured they must have some sort of way on the backend of resolving collisions.
- rocho 3y ago
  A message ID only needs to be unique per workspace, in which case you'd expect very few collisions to begin with, and you could even retry on insert failure with a new timestamp. I don't think that would cause significant performance penalties.
bArray 3y ago
> I was wondering: how often do nanosecond timestamps collide on modern systems? The answer is: very often, like 5% of all samples, when reading the clock on all 4 physical cores at the same time. A few things to consider: 1. This would depend a lot on how regularly you are checking, the more regular, the more collisions. 2. There may also be some weirdness where threads doing similar tasks will synchronise or desynchronise to maximize throughput. 3. Your system may not offer accurate nanosecond time. For this reason some OSes tend not to offer lower than microsecond time, as the clocks simply cannot offer higher usable accuracy. You're also measuring the time to call the function(s), create the timestamp, etc. A simple solution I had years ago was to add random precision to reduce collisions. That breaks most tie situations. If you have 5% ties in nano seconds, with random pico second accuracy your collisions are down to 0.005%. You could also seed the random offset by an ID.
scandum 3y ago
I get the current time in microseconds, and increment by one nanosecond for each call between updates. Problem solved for the next 200 years.
- ourmandave 3y ago
  Next article: Picosecond timestamp collisions are common.
- tourist2d 3y ago
  Problem solved until your boss calls you an idiot for storing fake nanosecond precisions.
datenwolf 3y ago
In other news: Water is wet. The high resolution precision time counters are derived from the system base clock, usually operating at ~33MHz, which translates exactly into that 30ns granularity observed. If you really want robust time derived timestamp identifiers, truncate the high resolution timer to at best 10µs resolution replace the low bits with the hash of them, concatenated with the value of the CPU cycle counter (`RDTSC` on x86).
- byteknight 3y ago
  Wouldn't those operations reduce the accuracy of the time stamp?
  - datenwolf 3y ago
    What is this "accuracy" you're talking about? Between the scheduler hanging over a process's head, ready to yank away its compute time for milliseconds between the syscall to read out the HPTC or the CPU cycle counter, and the process actually writing the timestamp into a variable/in-memory struct, reading the HPTC from the underlying hardware also not being super accurate, and the CPU cycle counter being influence by frequency scaling, on a bog-standard machine the highest precision you can reasonable expect is on the order of 100µs or so.
  - crote 3y ago
    Modern CPUs don't really give you accurate nanosecond-scale time stamps anyways. The CPU will randomly speed up or slow down, execute instructions out of order, and even speculatively execute instructions. Not to mention that it'll have a dozen different clocks - which are not guaranteed to be in sync.
    - blibble 3y ago
      > The CPU will randomly speed up or slow down constant_tsc has been a thing for more than a decade > execute instructions out of order, and even speculatively execute rdtscp serialises > Not to mention that it'll have a dozen different clocks - which are not guaranteed to be in sync. synchronised_tsc has been a think for about 6 years now
rdtsc 3y ago
Erlang/Elixir (BEAM VM) makes this very clear - it's a distinction between monotonic vs strictly monotonic. https://www.erlang.org/doc/apps/erts/time_correction.html#monotonically-increasing https://www.erlang.org/doc/apps/erts/time_correction.html#mo... > In a monotonically increasing sequence of values, all values that have a predecessor are either larger than or equal to its predecessor. These are available via the https://www.erlang.org/doc/man/erlang.html#monotonic_time-0 https://www.erlang.org/doc/man/erlang.html#monotonic_time-0 function. https://www.erlang.org/doc/apps/erts/time_correction.html#strictly-monotonically-increasing https://www.erlang.org/doc/apps/erts/time_correction.html#st... > In a strictly monotonically increasing sequence of values, all values that have a predecessor are larger than its predecessor. Strictly monotonic values imply some synchronization / coordination, with an associated performance impact when there are many concurrent processes. This functionality is available via the https://www.erlang.org/doc/man/erlang.html#unique_integer-1 https://www.erlang.org/doc/man/erlang.html#unique_integer-1 function. With a warning associated with it: > Strictly monotonically increasing values are inherently quite expensive to generate and scales poorly. This is because the values need to be synchronized between CPU cores. That is, do not pass the monotonic modifier unless you really need strictly monotonically increasing values.
- derefr 3y ago
  Relatedly, Erlang's own refs aren't created by a strictly-monotonic global generator; but rather are internally a pair of a regular monotonic identifier, and the PID of the requesting process. In other words, they're akin to UUIDv1s, or to https://en.wikipedia.org/wiki/Snowflake_ID https://en.wikipedia.org/wiki/Snowflake_ID s. You only really need strictly monotonic global identifiers if you need immediately-consistent first/last-write-wins. If you can instead rely on eventually-consistent first/last-write-wins (i.e. if your write events enter an event store/queue that linearizes them by ID, and then all "simultaneous" writes but the one with highest ID priority can be dropped/ignored, either during processing, or on read), then I'd recommend first considering packed (nodeID, seq) pairs. And, if you want global event orderibility, considering the snowflake ID formulation (timestampMajor, nodeID, timestampMinor, seq) specifically.
chaoticmass 3y ago
Reminds me of some Lotus Notes folklore. Apparently they used to use timestamps with a resolution of 1 second as unique IDs for things. When there was a collision, they would just add 1 second. Eventually, you'd end up with items being in the future because there would be so many collisions.
hinkley 3y ago
I’ve met too many people who are surprised by millisecond or microsecond timestamp collisions. My most memorable and least favorite variety of this is when people try to assemble a timestamp from two system calls, one for the most significant digits and a second for the least. If the smallest digits roll over from 99x to 00x after you read the large digits, due to process preemption, you can create a time stamp for an entity that happens before the entity that caused it to exist. This breaks some code spectacularly (I’ve seen infinite loops at least twice). If you haven’t memorized this as a thing to always avoid, you end up with tests that pass 99.5% of the time, and it can take someone with very good pattern matching skills to catch that the same test has been red once a week for a month and a half, which is a very long time for a logic bomb to live in CI/CD code before getting fixed.
- dan-robertson 3y ago
  My most memorable example was some support discussion that went something like “we think you have a race condition, <description>” “It can’t be a race condition as those two events happened in the exact same time”
dnstalk 3y ago
Comment out CLOCK_MONOTONIC_RAW because that's not available on FreeBSD, and it looks OK to me? My understanding is that we should see some timestamps repeating if there's a collision, correct? I can't get any collisions... > ./clock_gettime_demo/clock_gettime_demo clock_getres(CLOCK_REALTIME, ...)=1 ns clock_getres(CLOCK_MONOTONIC, ...)=1 ns CLOCK_REALTIME 30 samples: 1689957434662039455 1689957434662039526 (diff=71) 1689957434662039566 (diff=40) 1689957434662039606 (diff=40) 1689957434662039636 (diff=30) 1689957434662039676 (diff=40) 1689957434662039706 (diff=30) 1689957434662039736 (diff=30) 1689957434662039766 (diff=30) 1689957434662039796 (diff=30) 1689957434662039826 (diff=30) 1689957434662039856 (diff=30) 1689957434662039886 (diff=30) 1689957434662039916 (diff=30) 1689957434662039946 (diff=30) 1689957434662039987 (diff=41) 1689957434662040016 (diff=29) 1689957434662040047 (diff=31) 1689957434662040076 (diff=29) 1689957434662040107 (diff=31) 1689957434662040137 (diff=30) 1689957434662040167 (diff=30) 1689957434662040197 (diff=30) 1689957434662040227 (diff=30) 1689957434662040257 (diff=30) 1689957434662040297 (diff=40) 1689957434662040327 (diff=30) 1689957434662040357 (diff=30) 1689957434662040387 (diff=30) 1689957434662040417 (diff=30) CLOCK_MONOTONIC 30 samples: 2168262770533974 2168262770534023 (diff=49) 2168262770534054 (diff=31) 2168262770534094 (diff=40) 2168262770534124 (diff=30) 2168262770534164 (diff=40) 2168262770534194 (diff=30) 2168262770534224 (diff=30) 2168262770534254 (diff=30) 2168262770534284 (diff=30) 2168262770534314 (diff=30) 2168262770534344 (diff=30) 2168262770534374 (diff=30) 2168262770534404 (diff=30) 2168262770534435 (diff=31) 2168262770534464 (diff=29) 2168262770534495 (diff=31) 2168262770534524 (diff=29) 2168262770534555 (diff=31) 2168262770534585 (diff=30) 2168262770534615 (diff=30) 2168262770534645 (diff=30) 2168262770534685 (diff=40) 2168262770534715 (diff=30) 2168262770534745 (diff=30) 2168262770534775 (diff=30) 2168262770534805 (diff=30) 2168262770534835 (diff=30) 2168262770534865 (diff=30) 2168262770534895 (diff=30)
- vient 3y ago
  Do you run it on 4 cores simultaneously as the author did?
  - dnstalk 3y ago
    I'm running it on real hardware (Ryzen 9 5900X, 12 cores 24 threads) and all I'm doing is executing the program once. My understanding is that the program is using runtime.GOMAXPROCS(0) to ensure it's using all available CPU cores because that falls back to runtime.NumCPU ?
non-e-moose 3y ago
Silly Rabbit - absolutely accurate times are a security problem. CPU designers (even waay back in Alpha @ DEC) intentionally introduced clock jitter, just to prevent total predictability. For x86, I think if you performed 3-4 of them, and then saved the values in to registers, and then upon completion reported those values, you would find that the time deltas are NOT exactly the same.
- bajsejohannes 3y ago
  Do you have any sources for this? My googling skills are failing me. I'm surprised early x86 (which I assume you're including) were aware of security issues with accurate clocks; I certainly wasn't until this millennium :D I would rather guess observed clock jitter would be explained by interrupts or some such. Not saying you're wrong, I'd just like to learn more.
maxekman 3y ago
There is some interesting content regarding this in the Timestamps and Sequencing section in this post about the Go execution tracer overhaul: https://go.googlesource.com/proposal/+/ac09a140c3d26f8bb62cbad8969c8b154f93ead6/design/60773-execution-tracer-overhaul.md https://go.googlesource.com/proposal/+/ac09a140c3d26f8bb62cb...
ChuckMcM 3y ago
Let me tell you about the time the performance guy figured out one of the big cycle eaters was that every CPU in the SMP system was trying to read the time of day from the same place in memory so they "fixed" that by giving every CPU its only copy of the current time ...
nurettin 3y ago
I don't think this is true in practice. You don't just produce timestamp_ns in a busy loop. There is also the cpu, memory or io operation that generated or copied the data bound to the timestamp. And that isn't sub-ns.
dreamcompiler 3y ago
I solve this problem with a time check coupled with an atomic test-and-set of a global variable. If the global is less than the timer, use the timer value and set the global to it. If the global is greater than or equal to the timer, increment it and use that value.
rfmoz 3y ago
Someone knows why OSX clock_gettime() doesn't offer CLOCK_UPTIME and only CLOCK_UPTIME_RAW?

316 points by ingve 3y ago

jfax 3y ago

This was realised in the development of imageboard software (futaba.php, later yotsuba.php that runs 4chan) in the naming images, which is a unix timestamp + random 3 digit number because just the timestamp often causes clashes.

JacobSeated 3y ago

Timestamps should probably never be used as a "unique" id.

3cats-in-a-coat 3y ago
The problem is achieving locality in cohesion/meaning, which usually involves locality in time, which provokes using timestamps as part of the id at the very least. But it's a chain of very lazy thinking IMHO and it's like a peek at the house of cards some large systems are built like.
- eru 3y ago
  Why do you want this kind of locality anyway?
  - traviscj 3y ago
    Database indices.
- Mawr 3y ago
  Would a Snowflake ID work for this? https://en.wikipedia.org/wiki/Snowflake_ID https://en.wikipedia.org/wiki/Snowflake_ID

vbezhenar 3y ago

If you need unique nanosecond, keep track of the previously generated one and increase it if necessary. Would require global lock or atomic stuff, but should be good enough for practical uses.

pyrale 3y ago
One benefit of UUID is that you don't need coordination. In coordinated systems, unique IDs are a non-issue.
- OJFord 3y ago
  Is there any other benefit? That's the raison d'être - Universally.
  - eru 3y ago
    Depends on what you compare them with.
buster 3y ago
If you do that, just take some redis server and increment an integer every time you need a new number?
- marcus_holmes 3y ago
  or y'know, use the thing that was designed specifically for this case: UUID's
  - hnlmorg 3y ago
    UUIDs are designed to solve a slightly different, albeit related, problem: where you don’t have synchronisation of the system as a whole. The solution the GP is solving is for systems that are synchronised and therefore generating a UUID is additional and unnecessary overhead.
3y ago

getmeinrn 3y ago

I was going to post about "use a UUID", but I was surprised to learn that no UUID uses both timestamp + a random component. You can either get fully random with UUID4, or have a time + MAC based UUID with UUID1. Strange, I would have thought there would exist a UUID that uses time + random to minimize collisions like described in the post.

wood_spirit 3y ago
the new UUIDv6, 7 and 8 work the way you deacribe.
- dtech 3y ago
  Only v6 and v7, v8 is "whatever you want".
  - mcv 3y ago
    That's not much of a standard.
    - throwawaymobule 3y ago
      From my reading, it's standardized because it has a version number embedded, so it won't get confused with other uuids. Probably a good idea to have somewhere to silo bespoke implementations. Just gotta hope they end up being unique.
    - dtech 3y ago
      It's not uncommon to provide a way how to do it in standards because it will happen whether you want it or not. See e.g. HTTP headers starting with X- or MIME types in application/vnd.
nabogh 3y ago
Can't find much info about it but is this UUID v7?

wood_spirit 3y ago

This is why you should use ids that combine both a time component and a sequence. Eg UUIDv7 has a milliseconds time component and then a field that increments for each event in the same millisecond, and then enough random bits to make collisions between ids generated on different machines astronomically unlikely. Of course there are only so many bits so you might generate too many events in the same time slice so the sequence overflows, and you might actually get collisions between machines, and you are limiting your event generation speed by forcing your cpu to sync on the increment etc. But in practice UUIDv7 works great at scale.

darkclouds 3y ago
> This is why you should use ids that combine both a time component and a sequence. Computers should run like clockwork, so in this example of using all the cores, in Windows and likely some other OS's, threads are assigned to cores when they are started and you can have many threads per core, ergo the time component should also have the thread and the core number combined with it with multi core systems. Its possible to write the code in such a way some variables are bounced out of the cpu cache back to memory to avoid any caching issues because I think the cache on some cpu's are per core, and some are per cpu.
- Joker_vD 3y ago
  > threads are assigned to cores when they are started Do they? I thought the normal behaviour was for cores to pick any available thread to run, so core migration is quite normal. > ergo the time component should also have the thread and the core number combined with it with multi core systems. Sorry, how exactly does it follow from the previous? You seem to have omitted the other half of your syllogism. After all, clockworks do not have thread nor core numbers so I don't quite see how having those in UUIDs will make computers run like clockwork.
  - darkclouds 3y ago
    >> threads are assigned to cores when they are started >Do they? I thought the normal behaviour was for cores to pick any available thread to run, so core migration is quite normal. The CPU and its cores know very little about threads, threads are a figment of the OS. >Sorry, how exactly does it follow from the previous? You seem to have omitted the other half of your syllogism Syllogism would perhaps be a intuitive word to use, why is this? What happened to you?

grose 3y ago

This is why it scares me a bit to use a raw timestamp as a sort key in DynamoDB. I append a (random) unique ID to the timestamp text to avoid it. Better safe than sorry, I figure.

fer 3y ago
I have a use case that uses a similar solution, but for a different reason: it doesn't scare me to use timestamp as sort key I since I know my hash keys are unique and I only write every few seconds. But I still add random amount of ms (well under the frequency of writes/updates), because otherwise I'll be hitting hot partition issues on the GSI (indexed by timestamp, as you can guess).
moduspol 3y ago
We ran into this, though we were using millisecond timestamps with the Number type. Ended up working around it by adding numbers after the decimal point. DynamoDB apparently supports up to 38 digits of precision, and JavaScript (we're NodeJS based) by default encodes numbers to allow for up to 16 digits (combined, before and after the decimal point). A UNIX epoch millisecond timestamp is 13 digits, so we were able to use 3 just to add uniqueness without having to update anything else in our codebase. We certainly now plan to essentially always use string-based (and generically named) partition and sort keys, which would allow us to more easily do what you're describing.

techNoob123 3y ago

At some point doesn’t this come down to the ISA? A CPU running at 3GHz gets 3 clock cycles per nanosecond. I bet there is a fair amount of optimization in the compiler that leads to back to back assembly calls of reading the clock register. If subsequent time.Now() calls happen within 3 clock cycles of each other, can you really fairly expect unique nanosecond precision…

rwmj 3y ago
Linux will (x86_64) use RDTSC and adjust that against a value read from the VDSO, so it can indeed happen very quickly.
dan-robertson 3y ago
It takes like 20 cycles to read the cycle count register on modern chips. Even if collisions are somewhat unlikely, it’s still much worse to get several a day than ‘almost never going to happen’.

zokier 3y ago

I guess I'm old, the macOS behaviour is more in line with my expectations. But this got me thinking, how feasible it would be to tie the various clock systems of a computer to some reference clock, like 10 MHz GPSDO? Obviously it wouldn't improve the granularity, but you could ensure that the timestamps are actually accurate. Because otherwise I doubt that random computer clock would be accurate down to 32ns even with NTP.

mgaunard 3y ago
Getting a 10MHz PPS signal requires specialized expensive hardware and typically doesn't scale well to cover every server. That sort of thing is best left to extreme applications with FPGAs or ASICs. In particular the 10MHz version; a lot of commodity hardware only supports the 1Hz one. There's still an in-between of PPS and NTP: PTP.
- mlichvar 3y ago
  A major problem in synchronization of the system clock is PCIe. Hardware can timestamp PPS signal or PTP/NTP packets with accuracy of a few nanoseconds if everything is well compensated, but the PCIe bus between the CPU and the timestamping HW has a latency of hundreds of nanoseconds with potentially large asymmetry, degrading the accuracy significantly. Measuring that error is difficult. A possibility is to run a program periodically making random reads from memory (avoiding the CPU cache) to generate a PPS signal on the memory bus, which can be observed with a scope. There is a lot of noise due to other memory activity, RAM refresh, etc. From the configured memory speed and tRCD+tCL timings the uncertainty of the error can be reduced. This might improve with new hardware. There is a feature called Precision Time Measurement (PTM), which is a hardware implementation of an NTP-like protocol in PCIe, but so far I have seen this working only on some onboard NICs.

LAC-Tech 3y ago

A lot of mention of UUDv7 in this thread which is good. But I also wonder what the collision rate for Ulids are.

3cats-in-a-coat 3y ago
I frankly don't understand how it's good. UUID originally was intended as something you use very sparingly, to name, say, a product SKU maybe, an organization, something like that. Not literally content that collides commonly at the same nanosecond, in the same application, in the same platform/org. At some point we have to question the sanity of using one single flat address space for everything from the tiniest identifier to the... well, "Universe", as if it makes sense. We can have registrars for global ids, and we can nest local ids inside them, we can have a hierarchy, and we can have local compact ids, 32, 64 or 128 bit, which will never collide, and be locally cohesive. So why aren't we doing this? Is it ignorance? Is it because magic is easier than actually figuring out the synchronization and order of things in a system like this (and no, synchronization does NOT imply you need to issue ids strictly linearly). Honestly I'm at a loss of words.
- creatonez 3y ago
  > I frankly don't understand how it's good. UUID originally was intended as something you use very sparingly, to name, say, a product SKU maybe, an organization, something like that. Not literally content that collides commonly at the same nanosecond, in the same application, in the same platform/org. Wikipedia gives a different history, that it was originally for networked computers - https://en.wikipedia.org/wiki/Universally_unique_identifier#History https://en.wikipedia.org/wiki/Universally_unique_identifier#...
- RicardoLuis0 3y ago
  > So why aren't we doing this? isn't IPv6 is basically that? just restricted to internet addresses
  - 3cats-in-a-coat 3y ago

p-e-w 3y ago

If you want unique identifiers, use version 4 (random) UUIDs. Problem solved. The probability of a collision is roughly the same as the probability of a fully grown dinosaur spontaneously manifesting in your bedroom due to quantum fluctuations.

jraph 3y ago
I will take my chance :-) More seriously, If you can use them, good old increments are probably best. They are fast and cheap. Especially in a database. They can have privacy/security issues (you could guess things by the values of ids of stuff). UUIDs are better in those case or when you deal with a distributed system.
- p-e-w 3y ago
  Unless the RNG itself is the problem, there's no reason not to. All kinds of civilization-ending catastrophes are vastly more likely than a collision in a space of size 2^122.
- p-e-w 3y ago
  > They can have privacy/security issues (you could guess things by the values of ids of stuff). Push them through a secure hash function, and that problem is solved too (assuming you can keep the base counter private).
  - cacheyourdreams 3y ago
    If you're going to do that then you might as well just use UUID, since you effectively reintroduce the negative aspects of that (infinitesimally miniscule chance of collisions, computation involved in the calculation, etc.)
    - p-e-w 3y ago
      The difference is that you can still use sequential IDs internally, while exposing hashed IDs to the outside. This protects your database from collisions under all circumstances, while in the absolute worst case, a single user might experience bugs because two external IDs collide.

forrestthewoods 3y ago

> On my system, the minimum increment was 32 ns. Then you aren’t using a nanosecond clock. If you want actual nanosecond precision then you probably want rdtsc.

efitz 3y ago

Related I used to be the program manager owner of the security event log in windows. When things happen simultaneously or very closely in time on multi-core systems, thread scheduling can significantly affect observations of those things. For example, your thread quantum might expire before you get to the syscall to get a time stamp, or before you can pass a buffer to queue your event for subsequent time stamping. In fact, on multiprocessing systems, it was very common to see out of order event log entries on Windows back in the day (2000-oughts). You also could not count on the log timestamp accuracy too precisely; 1s was pretty much the safe lower bound (some components truncated or rounded time stamps IIRC).

godelski 3y ago

Despite the resolution being nanoseconds, what is the actual precision of computer clocks? I can't imagine it is actually nanoseconds. Takes me back to teaching physics labs where I had to hound students to remember that the accuracy of their measuring device is not identical to the smallest number it displays...

ajb 3y ago
For devices clocked above 1Ghz it's perfectly possible for the clock to increment every ns, although that doesn't make it accurate to that level, and multi core systems may have clocks that are not synchronised to that level. ARMv8 guarantees that it's clock increments at at least 1Ghz, for intel and earlier ARM it's more complicated
- zokier 3y ago
  Cycle counting is one thing, but it gets tricky when you have frequency scaling in play. Another problem that even without freq scaling, cpu clocks are not designed to be super accurate/exact, and the true frequency might vary significantly even if the nominal freq is fixed
  - daniel-cussen 3y ago
    It doesn't matter that much they're not super accurate exact, they do in fact count ticks n tocks j dandy. I wonder if they are counted equally, interesting question if i do say so myself about my own question, i know rising edge is steady w the next rising edge, n falling edge w falling edge, but yeah...no i think that's a specification figure of merit, that they be balanced w respect to each other. N Intel® despite it's many failures, long ago forecast n long overdue due to the sheer difficulty of their business model n how long they kept it going--they were saying it was going to end soon in the late seventies already--n you know what, i'm fine w that. They don't make old chips rot like other software companies i could mention, bits rot but glass is timeless. Which brings me back to the point, in my analysis the problem is not the clocks being inaccurate but rather the jitter, which means a single run will not suffice in describing a repeatable exact clock time taken for eg an inner loop, which is what is worth bothering with. The minimum jitter attainable currently is 1 cycle, n then i guess you run the same code repeatedly n take the minimum with more repeatability as a consequence of that low jitter. In the early nineties it was not so, you'd get the same number of clock cycles again n again n again. N then it gets tricky because cycle counts are thresholds set within which, if voltages n frequency are proper, the operation will complete deterministically w a soft error rate no greater than the system as a whole, about one per 30 years of hammering on the chip at sea level. Which is not enough for my tastes, n the jitter is a fucking mess. I much prefer the GA144, least jitter of any platform bar none, sensible because it is fully asynchronous logic, no clock anywhere in sight until you connect it to an oscillator, n even then the vibration doesn't rock the system like that of a grandfather clock synchs w another grandfather clock with whose pendulum's swing the former clock's pendulum swing is aligned. GA144 it's pretty easy to tell average case complexity of a function down to the tens of picoseconds, at which point you have to check that there's no open window letting a draft in. In fact the time trial will tell you such a draft is coming into the house, it happened to me, while not from a parallel universe in spacial dimensions it is by all means from another universe in the time dimension.

quickthrower2 3y ago

Another case of precision but not accuracy.

qwerty456127 3y ago

> it is unsafe to assume that a raw nanosecond timestamp is a unique identifier It is reasonable to assume that a raw nanosecond timestamp is a unique identifier in cases when you can reasonably expect such identifiers are not generated too offten. I.e. this is Ok (and even redundant) for identifying manual interactions a single user would pdosuce. It probably is also Ok for labelling manual interactions multiple users would produce if that's a small team.

aaron695 3y ago
[dead]

tsenart 3y ago

Our Go ULID package has millisecond precision + monotonic random bytes for disambiguation while preserving ordering within the same millisecond. https://github.com/oklog/ulid https://github.com/oklog/ulid

donatj 3y ago

Slacks API uses straight nanoseconds for the IDs of their messages which I always found very curious but figured they must have some sort of way on the backend of resolving collisions.

rocho 3y ago
A message ID only needs to be unique per workspace, in which case you'd expect very few collisions to begin with, and you could even retry on insert failure with a new timestamp. I don't think that would cause significant performance penalties.

bArray 3y ago

> I was wondering: how often do nanosecond timestamps collide on modern systems? The answer is: very often, like 5% of all samples, when reading the clock on all 4 physical cores at the same time. A few things to consider: 1. This would depend a lot on how regularly you are checking, the more regular, the more collisions. 2. There may also be some weirdness where threads doing similar tasks will synchronise or desynchronise to maximize throughput. 3. Your system may not offer accurate nanosecond time. For this reason some OSes tend not to offer lower than microsecond time, as the clocks simply cannot offer higher usable accuracy. You're also measuring the time to call the function(s), create the timestamp, etc. A simple solution I had years ago was to add random precision to reduce collisions. That breaks most tie situations. If you have 5% ties in nano seconds, with random pico second accuracy your collisions are down to 0.005%. You could also seed the random offset by an ID.

scandum 3y ago

I get the current time in microseconds, and increment by one nanosecond for each call between updates. Problem solved for the next 200 years.

ourmandave 3y ago
Next article: Picosecond timestamp collisions are common.
tourist2d 3y ago
Problem solved until your boss calls you an idiot for storing fake nanosecond precisions.

datenwolf 3y ago

In other news: Water is wet. The high resolution precision time counters are derived from the system base clock, usually operating at ~33MHz, which translates exactly into that 30ns granularity observed. If you really want robust time derived timestamp identifiers, truncate the high resolution timer to at best 10µs resolution replace the low bits with the hash of them, concatenated with the value of the CPU cycle counter (`RDTSC` on x86).

byteknight 3y ago
Wouldn't those operations reduce the accuracy of the time stamp?
- datenwolf 3y ago
  What is this "accuracy" you're talking about? Between the scheduler hanging over a process's head, ready to yank away its compute time for milliseconds between the syscall to read out the HPTC or the CPU cycle counter, and the process actually writing the timestamp into a variable/in-memory struct, reading the HPTC from the underlying hardware also not being super accurate, and the CPU cycle counter being influence by frequency scaling, on a bog-standard machine the highest precision you can reasonable expect is on the order of 100µs or so.
- crote 3y ago
  Modern CPUs don't really give you accurate nanosecond-scale time stamps anyways. The CPU will randomly speed up or slow down, execute instructions out of order, and even speculatively execute instructions. Not to mention that it'll have a dozen different clocks - which are not guaranteed to be in sync.
  - blibble 3y ago
    > The CPU will randomly speed up or slow down constant_tsc has been a thing for more than a decade > execute instructions out of order, and even speculatively execute rdtscp serialises > Not to mention that it'll have a dozen different clocks - which are not guaranteed to be in sync. synchronised_tsc has been a think for about 6 years now

rdtsc 3y ago

Erlang/Elixir (BEAM VM) makes this very clear - it's a distinction between monotonic vs strictly monotonic. https://www.erlang.org/doc/apps/erts/time_correction.html#monotonically-increasing https://www.erlang.org/doc/apps/erts/time_correction.html#mo... > In a monotonically increasing sequence of values, all values that have a predecessor are either larger than or equal to its predecessor. These are available via the https://www.erlang.org/doc/man/erlang.html#monotonic_time-0 https://www.erlang.org/doc/man/erlang.html#monotonic_time-0 function. https://www.erlang.org/doc/apps/erts/time_correction.html#strictly-monotonically-increasing https://www.erlang.org/doc/apps/erts/time_correction.html#st... > In a strictly monotonically increasing sequence of values, all values that have a predecessor are larger than its predecessor. Strictly monotonic values imply some synchronization / coordination, with an associated performance impact when there are many concurrent processes. This functionality is available via the https://www.erlang.org/doc/man/erlang.html#unique_integer-1 https://www.erlang.org/doc/man/erlang.html#unique_integer-1 function. With a warning associated with it: > Strictly monotonically increasing values are inherently quite expensive to generate and scales poorly. This is because the values need to be synchronized between CPU cores. That is, do not pass the monotonic modifier unless you really need strictly monotonically increasing values.

derefr 3y ago
Relatedly, Erlang's own refs aren't created by a strictly-monotonic global generator; but rather are internally a pair of a regular monotonic identifier, and the PID of the requesting process. In other words, they're akin to UUIDv1s, or to https://en.wikipedia.org/wiki/Snowflake_ID https://en.wikipedia.org/wiki/Snowflake_ID s. You only really need strictly monotonic global identifiers if you need immediately-consistent first/last-write-wins. If you can instead rely on eventually-consistent first/last-write-wins (i.e. if your write events enter an event store/queue that linearizes them by ID, and then all "simultaneous" writes but the one with highest ID priority can be dropped/ignored, either during processing, or on read), then I'd recommend first considering packed (nodeID, seq) pairs. And, if you want global event orderibility, considering the snowflake ID formulation (timestampMajor, nodeID, timestampMinor, seq) specifically.

chaoticmass 3y ago

Reminds me of some Lotus Notes folklore. Apparently they used to use timestamps with a resolution of 1 second as unique IDs for things. When there was a collision, they would just add 1 second. Eventually, you'd end up with items being in the future because there would be so many collisions.

hinkley 3y ago

I’ve met too many people who are surprised by millisecond or microsecond timestamp collisions. My most memorable and least favorite variety of this is when people try to assemble a timestamp from two system calls, one for the most significant digits and a second for the least. If the smallest digits roll over from 99x to 00x after you read the large digits, due to process preemption, you can create a time stamp for an entity that happens before the entity that caused it to exist. This breaks some code spectacularly (I’ve seen infinite loops at least twice). If you haven’t memorized this as a thing to always avoid, you end up with tests that pass 99.5% of the time, and it can take someone with very good pattern matching skills to catch that the same test has been red once a week for a month and a half, which is a very long time for a logic bomb to live in CI/CD code before getting fixed.

dan-robertson 3y ago
My most memorable example was some support discussion that went something like “we think you have a race condition, <description>” “It can’t be a race condition as those two events happened in the exact same time”

dnstalk 3y ago

Comment out CLOCK_MONOTONIC_RAW because that's not available on FreeBSD, and it looks OK to me? My understanding is that we should see some timestamps repeating if there's a collision, correct? I can't get any collisions... > ./clock_gettime_demo/clock_gettime_demo clock_getres(CLOCK_REALTIME, ...)=1 ns clock_getres(CLOCK_MONOTONIC, ...)=1 ns CLOCK_REALTIME 30 samples: 1689957434662039455 1689957434662039526 (diff=71) 1689957434662039566 (diff=40) 1689957434662039606 (diff=40) 1689957434662039636 (diff=30) 1689957434662039676 (diff=40) 1689957434662039706 (diff=30) 1689957434662039736 (diff=30) 1689957434662039766 (diff=30) 1689957434662039796 (diff=30) 1689957434662039826 (diff=30) 1689957434662039856 (diff=30) 1689957434662039886 (diff=30) 1689957434662039916 (diff=30) 1689957434662039946 (diff=30) 1689957434662039987 (diff=41) 1689957434662040016 (diff=29) 1689957434662040047 (diff=31) 1689957434662040076 (diff=29) 1689957434662040107 (diff=31) 1689957434662040137 (diff=30) 1689957434662040167 (diff=30) 1689957434662040197 (diff=30) 1689957434662040227 (diff=30) 1689957434662040257 (diff=30) 1689957434662040297 (diff=40) 1689957434662040327 (diff=30) 1689957434662040357 (diff=30) 1689957434662040387 (diff=30) 1689957434662040417 (diff=30) CLOCK_MONOTONIC 30 samples: 2168262770533974 2168262770534023 (diff=49) 2168262770534054 (diff=31) 2168262770534094 (diff=40) 2168262770534124 (diff=30) 2168262770534164 (diff=40) 2168262770534194 (diff=30) 2168262770534224 (diff=30) 2168262770534254 (diff=30) 2168262770534284 (diff=30) 2168262770534314 (diff=30) 2168262770534344 (diff=30) 2168262770534374 (diff=30) 2168262770534404 (diff=30) 2168262770534435 (diff=31) 2168262770534464 (diff=29) 2168262770534495 (diff=31) 2168262770534524 (diff=29) 2168262770534555 (diff=31) 2168262770534585 (diff=30) 2168262770534615 (diff=30) 2168262770534645 (diff=30) 2168262770534685 (diff=40) 2168262770534715 (diff=30) 2168262770534745 (diff=30) 2168262770534775 (diff=30) 2168262770534805 (diff=30) 2168262770534835 (diff=30) 2168262770534865 (diff=30) 2168262770534895 (diff=30)

vient 3y ago
Do you run it on 4 cores simultaneously as the author did?
- dnstalk 3y ago
  I'm running it on real hardware (Ryzen 9 5900X, 12 cores 24 threads) and all I'm doing is executing the program once. My understanding is that the program is using runtime.GOMAXPROCS(0) to ensure it's using all available CPU cores because that falls back to runtime.NumCPU ?

non-e-moose 3y ago

Silly Rabbit - absolutely accurate times are a security problem. CPU designers (even waay back in Alpha @ DEC) intentionally introduced clock jitter, just to prevent total predictability. For x86, I think if you performed 3-4 of them, and then saved the values in to registers, and then upon completion reported those values, you would find that the time deltas are NOT exactly the same.

bajsejohannes 3y ago
Do you have any sources for this? My googling skills are failing me. I'm surprised early x86 (which I assume you're including) were aware of security issues with accurate clocks; I certainly wasn't until this millennium :D I would rather guess observed clock jitter would be explained by interrupts or some such. Not saying you're wrong, I'd just like to learn more.

maxekman 3y ago

There is some interesting content regarding this in the Timestamps and Sequencing section in this post about the Go execution tracer overhaul: https://go.googlesource.com/proposal/+/ac09a140c3d26f8bb62cbad8969c8b154f93ead6/design/60773-execution-tracer-overhaul.md https://go.googlesource.com/proposal/+/ac09a140c3d26f8bb62cb...

ChuckMcM 3y ago

Let me tell you about the time the performance guy figured out one of the big cycle eaters was that every CPU in the SMP system was trying to read the time of day from the same place in memory so they "fixed" that by giving every CPU its only copy of the current time ...

nurettin 3y ago

I don't think this is true in practice. You don't just produce timestamp_ns in a busy loop. There is also the cpu, memory or io operation that generated or copied the data bound to the timestamp. And that isn't sub-ns.

dreamcompiler 3y ago

I solve this problem with a time check coupled with an atomic test-and-set of a global variable. If the global is less than the timer, use the timer value and set the global to it. If the global is greater than or equal to the timer, increment it and use that value.

rfmoz 3y ago

Someone knows why OSX clock_gettime() doesn't offer CLOCK_UPTIME and only CLOCK_UPTIME_RAW?