49 ms·
Latency Numbers Every Programmer Should Know
- akuji1993 8y agoI thought it's funny to not see the roundtrips increase at all. All the other measurements exploding in size, the more you go back and then just the roundtrips stay the same.
- exDM69 8y agoIt's the speed of light staying constant despite best efforts of physicists and engineers. The distance between California and the Netherlands hasn't changed either.
- scurvy 8y agoIt's actually about 60% the speed of light because there's no vacuum in buried fiber. But yes the rest of your point is accurate.
- umvi 8y agoAlso, presumably light has to travel farther in fiber because it's constantly bouncing off of walls vs. straight line travel?
- e12e 8y agoI was going to dismiss it as insignificant, but if I read this right: https://en.m.wikipedia.org/wiki/Total_internal_reflection https://en.m.wikipedia.org/wiki/Total_internal_reflection an angle higer than ~42 degrees seems to indicate roughly a 50% longer path? That said, I wonder what the average "fractal dimension" of a city to city fiber link is - I suspect wavering path around buildings and along roads dominate the increase in path VS theoretical straight (curved along the earth) path?
- scurvy 8y agoDon't forget fiber splices for accidental cuts. Engineers have to try their best to match the existing refraction when splicing in the new patch. Tools help, but there's a lot of skill and craftsmanship there.
- e12e 8y agoBut a splice should either work, or not work? It is literally welding two glass strands into one?
- Hikikomori 8y agoA splice may or may not work, or something in between. The splice might cause enough attenuation that you go over your optical budget (signal strength and receiver sensitivity) so no link comes up. But it might come up just barely and then you may have some bad frames as a result.
- deleted 8y ago[deleted]
- Hikikomori 8y agoThis is true for multimode, anywhere outside short distances in datacenters you use singlemode where light does not bounce at an angle. A singlemode fiber acts as a waveguide, the signal travels in a single mode, this is mainly why it's good for longer distances, no modal dispersion.
- scurvy 8y agoThat is not true. Single mode fiber definitely comes in at an angle and bounces off the cladding. You still have a cone of acceptance at play. You're confusing multiple terms and concepts here.
- jimmaswell 8y agoThe point seemed more along the lines of "the ping is a function of the speed of light" to me, so not necessarily inaccurate. Although that does leave room for improvement like introducing a vacuum or making the lines more direct.
- lloeki 8y agoGood time to dig out some lore: http://www.ibiblio.org/harris/500milemail.html http://www.ibiblio.org/harris/500milemail.html
- Iem3ohvi 8y agoTechnically neutrino signalling could make it faster, but those extra milliseconds are not worth it, even for the HFT folks... but thanks to them we have shorter fiber routes across the atlantic at least.
- supahfly_remix 8y ago> thanks to them we have shorter fiber routes across the atlantic at least. Can you explain how HFT caused a shorter fiber route across the Atlantic? Is this route open to the non-HFT public? I read Flash Boys and am aware of a custom fiber link between Weehauken, NJ and Chicago, IL, but I thought they are were moving to microwave. I thought there were some HFT links (fiber or microwave) within Europe.
- hermitdev 8y agoMicrowave is great for latency, but the bandwidth sucks. When you need bandwidth & low latency, you need a more direct fiber link.
- Iem3ohvi 8y agoI recall a new transatlantic cable constructed a few years ago being advertised as shaving off a few milliseconds. Trading and cloud providers were mentioned as target customers. I don't know whether they also route public internet traffic.
- daddylonglegs 8y agoThere are microwave links across Europe: https://sniperinmahwah.wordpress.com/2016/01/26/hft-in-the-banana-land/ https://sniperinmahwah.wordpress.com/2016/01/26/hft-in-the-b... There are apparently shortwave links across the Atlantic: https://sniperinmahwah.wordpress.com/2018/05/07/shortwave-trading-part-i-the-west-chicago-tower-mystery/ https://sniperinmahwah.wordpress.com/2018/05/07/shortwave-tr... https://sniperinmahwah.wordpress.com/2018/07/16/shortwave-trading-part-iv-sleuthing-examples-research-tools-techniques-deputies-wanted/ https://sniperinmahwah.wordpress.com/2018/07/16/shortwave-tr... These should have a lower latency than fibre links although the shortwave link will probably have rather a low data rate.
- rocqua 8y agoI mean, it is 4844 nautical miles as the crow flies, but we could make that 4458 nmi (nautical miles) by taking a direct route through the crust. I have to say I am disappointed/surprised by how limited those savings are.
- the8472 8y agoPhysicists and engineers actually are working on increasing the speed of light for fibers. https://www.nature.com/articles/nphoton.2013.45?error=cookies_not_supported&code=aed4a1c4-025f-404a-a995-bbd73a07833d https://www.nature.com/articles/nphoton.2013.45?error=cookie...
- preinheimer 8y agoIf ping times are your jam, we've collected a few between major cities across the globe: https://wondernetwork.com/pings https://wondernetwork.com/pings
- ape4 8y agoCool. I don't think the 3 decimal places are significant.
- Iem3ohvi 8y agoAverages are not particularly interesting. CDFs or violin graphs are since hundreds of requests to load a site can turn your 99th percentile into what the user actually experiences. Edit: I just noticed that clicking on it shows box plots. Good enough!
- preinheimer 8y agoThanks! We've got all of our data going back years in AWS Athena, we're just waiting to have time to do something fun with it.
- krn 8y agoThanks for this tool. It was extremely helpful in trying to find the best locations for data-centers across the world. Who knew, that the presence in New York, London, and Singapore alone is enough to be within 100ms reach for 80% of the world's population.
- barrystaes 8y agoWow, it seems they did a great job in 1990 with the packet roundtrip CA to Netherlands, it still lists ~150ms for 2018. I think its closer to half or less than that, depending on what CA this is refering to. Assuming its not in Tokyo.
- dmayle 8y agoThe trip is dominated by the speed of light (or more accurately, the speed of electricity, which is close to the speed of light, at about 300,000 kilometers per second) The circumference of the earth is about 40,000 kilometers. A round trip from CA to Netherlends and back will have electricity going roughly that distance. Speed of electricity times 150 milliseconds is 44,000 kilometers. That means that the number won't change over time.
- e12e 8y ago> (or more accurately, the speed of electricity, which is close to the speed of light, at about 300,000 kilometers per second) Surely, the dominant part of the path city to city is via fiber, so laser light? That doesn't mean that's were most of the time goes (but ideally it should) - for latency things like light-to-signal, buffers, switching, carrier grade nat, all takes its toll. Would be interesting to see a detailed breakdown of typical data center(... Cell phone, home broadband user) to same, though. Including every "point" in the path from ram, via cache, pci, network card, routers and switches, media converters and up again... For bonus something like USB keyboard touch to glyph on screen over udp or other low latency protocol...
- Hikikomori 8y agoYes, it's more or less all fiber. And in that case the speed is 2/3 of the speed of light. Delay will in most part be because of distance, forwarding delay in routers are a few 10's of microseconds while switches are at the sub-microsecond level. Stuff like this is quick when they have to handle 100Gbit interfaces.
- 8y ago
- dvh 8y agoGrace Hopper explaining nanosecond: https://m.youtube.com/watch?v=JEpsKnWZrJ8 https://m.youtube.com/watch?v=JEpsKnWZrJ8
- pwaivers 8y agoGreat video. I've never seen her speak before. She is very likable!
- gambler 8y agoGreat video. I really like when lecturers find ways to convey abstract concepts without dumbing them down with analogies. She achieved that perfectly by relating time to the speed of light, which is a perfectly sensible conversion. Edit: I've never seen any of her videos before. Now I did. She had an amazing sense of humor.
- bluetomcat 8y agoUnless the programmer is programming at a very low level, the listed events occurring are out of his control. CPU caching is mostly transparent in the ISA, disk seeking is scheduled by the kernel or in storage controllers, file buffers are cached in the kernel, application frameworks also provide layers of caching. For the majority of programmers who want to get their shit done with straightforward code, few dependencies and acceptable performance, this is "interesting to know" but not "should know".
- Iem3ohvi 8y ago> Unless the programmer is programming at a very low level If you write unixy tools that do just one job then you often have to deal with this. For example rsync and tar put files into sequential mode and perform readaheads or writebehind-drop the page cache. And it's not just that kind of tool. At $JOB I did a fairly simple optimization to significantly reduce loadtimes (from NFS) in a render farm by importing a 3rd-party library which provided the necessary libc bindings for readaheads. It's only a dozen lines of code but reduces user-perceived latency from minutes to seconds. The PO was quite happy about not having to pay for hundreds of NVMe SSDs.
- deleted 8y ago[deleted]
- TruffleLabs 8y agoThe visual could help programmers think about the kinds of latencies that have to work through, maybe helping highlight them in an infographic for their teams/partners.
- beached_whale 8y agoYou often can control your data access patterns. Array like access, in order, is often cache friendliest
- deleted 8y ago[deleted]
- dragontamer 8y ago> Unless the programmer is programming at a very low level, the listed events occurring are out of his control > CPU caching is mostly transparent in the ISA Nonsense. Even in a relatively high language like Java, you can use primitive types like int[] to ensure that certain elements are close to each other in memory. As such, you can have good memory access patterns even in a high language like Java or C#. I'm fairly certain this stuff is important when choosing data-structures: Vector vs Linked List for instance. Linked Lists are harder to cache than Vectors, and this chart helps explain why both of these O(1) traversals can have dramatically different performance characteristics. > disk seeking is scheduled by the kernel or in storage controllers, file buffers are cached in the kernel But you can read a file from beginning to end. Even in a very, very high language like SQL, you can often ensure a high-speed sequential table scan if you write your joins properly. And knowledge of sequential scans can assist you in knowing which indexes to setup for your tables. Knowing that you have SSDs vs Disks can be helpful in the architecture of SQL architectures.
- dmayle 8y agoThis is great for server development, but I'd love to see the numbers for mobile phones nowadays. Rather than year, mobile phone by year class...
- ssvss 8y agoIs there a similar chart for bandwidth numbers.
- zawerf 8y agoI think the coolest thing about this visualization is that it is now "broken". I remember when I first saw this, there were still visible red and green squares on the top rows. Today those numbers are so small, those squares are missing completely! (That said whoever owns this page should update the scale of the blocks so it's more useful going forward)
- e12e 8y ago> (That said whoever owns this page should update the scale of the blocks so it's more useful going forward) Fwiw the "fork me on github" banner points to: https://github.com/colin-scott/interactive_latencies https://github.com/colin-scott/interactive_latencies
- e12e 8y agoThank you for linking to your attachment blog post. What's the status of ronomon? It seems https://ronomon.com/ https://ronomon.com/ assume existing user - and gives no more information?
- jorangreef 8y agoThanks! Ronomon is in private beta. My email is in my profile if you'd like to chat more.
- ebikelaw 8y agofunction getDCRTT() { // Assume this doesn't change much? return 500000; // ns } :thinking face emoji: High by 10x in my experience.
- jorangreef 8y agoSince I came across Jeff Dean's tip a few years ago about back-of-the-envelope calculations and latency numbers (http://highscalability.com/blog/2011/1/26/google-pro-tip-use-back-of-the-envelope-calculations-to-choo.html http://highscalability.com/blog/2011/1/26/google-pro-tip-use...), I can't count the number of times that sketching out a design's complexity has saved me from implementing designs that can't possibly work, and eventually resulted in designs that are orders of magnitude faster. In fact, it's not just about designs that work, or designs that are fast, but getting into the practice of estimating complexity in terms of hardware numbers also makes for safer code, especially where validating user data is concerned. Just recently even, it kept me back from what might have been a potential denial of service in https://github.com/ronomon/mime https://github.com/ronomon/mime, and lead to discovering a vulnerability in several popular email parsers (https://snyk.io/blog/how-to-crash-an-email-server-with-a-single-email/ https://snyk.io/blog/how-to-crash-an-email-server-with-a-sin...). I think Martin Thompson summarized it well as "Mechanical Sympathy": https://mechanical-sympathy.blogspot.com/2011/07/why-mechanical-sympathy.html https://mechanical-sympathy.blogspot.com/2011/07/why-mechani...
- mdpopescu 8y agoGreat article, thanks for that (and for evidence #1236576 that I have no clue when it comes to software security). One rather off-topic observation: April 23 to June 25 is somewhat shorter than the 90-day window you mentioned. ("A few days before the 90-day public disclosure deadline...") What was the reason for that? It doesn't appear to be because those who were going to fix it had already done so - they published their fixes after the public disclosure. (I'm just curious, not criticizing or anything.)
- jorangreef 8y agoThanks! Regarding the 90-day window, you are spot on. I never realized that until now. I made a mistake with the month, it should have been July 25 not June 25, so it came out after 60 days, not 90 days as I intended. That's evidence #1236577 that I have no clue at all!
- kristianov 8y ago
- philipodonnell 8y agoIt looks like from 2006-2018 the only things that have improved are disk read speeds and the "commodity network" packet speed. Most of this chart is unchanged over that time. Has progress on CPUs, memory, processing speed really stopped over the last 12 years?
- exDM69 8y agoIt measures latency, not throughput. CPU or memory latency has not improved a lot, but throughput has through increased core count and instruction level parallelism. If you look at throughput per watt, the improvement is greater.
- bostonpete 8y agoI assume the "send 2000 bytes over a commodity network" is measuring throughput rather than latency. If not, I really don't understand what it's describing...
- gpderetta 8y agoIt is litterally the lenght of a 2000*8 bit backet in a serial link, in ns. The number seems way off unless my math is wrong or 200Gb ethernet counts as commodity today.
- mkesper 8y agoThis is a good example for bad display of data: ten lines for example don't get recognized by the mind as having that much greater value than a single line.
- tw04 8y ago3ms for a disk drive isn't something you should plan to. 3ms is assuming a 15k drive with little to no workload. Under load your likely 10ms and under heavy load you may be seeing 20+. SATA drives? 5ms best case, under load with anything but sequential workloads and you might as well just take the afternoon off.
- jsty 8y agoThat's a pretty good point. Is anyone aware of one of these sets of numbers with error bars / ranges?
- gamegoblin 8y agoWith hard drives, the main factors are how random (in terms or random access) the workload is, read/write distribution, IO size, and queue depth. Some assorted facts: Rotational latency -- The most common hard drive models are 7200 RPM. That is 120 rotations per second, or 1 rotation per 8.33 milliseconds. So, all things being equal, the average rotational latency will be half of that (best case: 0, worse case: 8.33 ms). Seek time -- The most common hard drive models have 1 head on an actuator (though multi-actuator models are being developed). The actuator has to physical move the head across the surface of the platters so that it's over the bits you want. The worst case seek time is around 8ms on a typical hard drive, for an average of 4ms. The less random your workload is, that is, the smaller subset of the disk you are touching, the smaller your seek time will be. IO size -- the amount of data you can read in a single hard drive rotation actually depends on where on the platter it is. If it's on the outer diameter of the drive, it might be e.g. 2MB, but on the inside diameter, only 1MB. For the reason you can actually get significantly higher throughput when reading from the outside diameter (typically the lower LBAs). Once you start doing even larger IOs, you're talking about waiting for multiple rotations for the IO to complete, which equals more time. Read/Write distribution -- Hard drives have buffers on them to absorb writes, and it is asynchronously flushed in a very efficient manner (either when the hard drive is not doing anything, or is on the way to do another IO and happens to pass over the right part of the disk). Queue depth -- Hard drives allow you to enqueue multiple concurrent IOs (most commonly from different threads/processes). SATA allows for a maximum of 32 concurrent IOs, whereas NVMe goes all the way up to 65K. Usually with an HDD though, you'll be on SATA. The number of concurrent IOs is referred to as the queue depth. You can improve your average latency by increasing queue depth, at the cost of peak latency. The reason for this is that hard drives will re-order queued IOs to reduce rotational latency and seek time. For example, imagine you issue 4 concurrent reads to LBAs 0, 10, 2, 5. The drive will likely reorder this to 0, 2, 5, 10 so that it doesn't have to spend as much time seeking. This will reduce the average latency, but the at particularly high queue depths, a particularly unlucky IO might get deprioritized for 100+ ms. If you care a lot about peak latency, bound your queue depth to 4 or so.
- jandrese 8y agoI'm surprised how cheap a branch mispredict is. I had thought the relatively long pipelines in modern processors would have made that more painful. It seems weird Intel devotes so much silicon to improving the branch predictor when the penalty is so light.
- mooman219 8y agoSpeculative execution probably plays a large role in that. I think this is more of an average that you can expect to see. Additionally, if eager execution is not at play, if a mispredict is triggered when data is finally fetched from main memory, the 5ns (handful of cycles) might just be the cost of unwinding, meaning the actual cost was 105ns.
- gpderetta 8y agoIt estimates 10 clock cycles penalty, which at 3GHz is ~3ns. I think that it is the correct order of magnitude, but 15-20 clocks is probay more accurate. The penalty is variable though and hard to characterize. For example recovering from the uop cache shaves 3-4 cycles.
- OskarS 8y agoTry it for yourself! Create an array with a million random bytes from 0 to 255, then count (in a for loop) with an "if" statement how many of them are less than 128. This branch will be essentially impossible to predict, and the branch predictor will fail roughly 50% of the time. Then try again, but sort the array before you begin (which will make the branch trivial to predict). Time the difference. I think you'll be surprised.
- AceJohnny2 8y agoWell, I took the bait. Time to go through an unsorted vs sorted array on an iMac17,1 with 4Ghz Core i7 (not sure which CPU exactly).: * 1MB: 6.52ms vs 2.16ms (3x speedup) * 1GB: 5.73s vs 2.25s (2.5x speedup) Tested with this C code, called as "bpredict-time [size] [sort]": #include <stdio.h> #include <stdint.h> #include <stdlib.h> #include <sys/time.h> #define SIZE (1024*1024) int cmp (const void *val1, const void *val2) { return *(uint8_t*)val1-*(uint8_t*)val2; } int main (int argc, char **argv) { uint8_t *array; size_t size = SIZE; struct timeval start, end; if (argc > 1) { size = atoi(argv[1]); } array = malloc(sizeof(*array) * size); if (array == NULL) { return -1;} for (size_t i=0; i<size; i++) { array[i] = random(); } if (argc > 2) { qsort(array, size, sizeof(*array), cmp); } (void)gettimeofday(&start, NULL); int cnt = 0; for (size_t i=0; i<size; i++) { if (array[i] < 128) { cnt++;} } (void)gettimeofday(&end, NULL); printf("N ints < 128: %d\n", cnt); printf("Time: %f\n", (float)(end.tv_sec-start.tv_sec)\ +(float)(end.tv_usec-start.tv_usec)/1000000); free(array); return 0; }
- lucio 8y agoSo there's no more room for improvement at the microchip level. Will software become leaner? Is Moore's law reaching a plateau?
- jiveturkey 8y agoTufte should be required reading if one is publishing something like this. I assume this is the work of an undergrad because it's horrible. EDIT: downvote is absurd. it's a no-question horrible visualization. I don't know which is worse, the poor presentation or the lack of credit to the original author (Dean).
- mmirate 8y ago> the lack of credit to the original author (Dean) The linked GitHub repo's description reads "Jeff Dean's latency numbers plotted over time".
- jiveturkey 8y agoyes, the github. 5% of the target audience might read that. trivial to add the credit on the graphical page, and no excuse not to do so. third fault, which to me should ban this page from the internet: he has dared to put a "2018" text on his page, insinuating that it's new data, or some new insight, as opposed to being 15+ years old. when the original text is more understandable than your visualization, you dun goofed
- jdoliner 8y agoThere are a few things in this diagram that always drive me a little crazy. Why is it that 1,000ns ≈ 1μs while 1,000,000ns = 1ms (almost equals vs. regular equals). It kinda makes sense to use almost equals for some of the real values, but they do it for pure values that are just there for conversions... but not with all of them, only with some. All this being said, this is actually a very useful and well done diagram. But, imo, it would be even better if it used equal signs in a consistent way.
- drewmol 8y agoMy guess is it's a typo in: 10,000ns ~ 10us = [green cube] and it's supposed to read 10,000ns = 10us = [green cube] and this is a key as are black red and blue, all of the speed measures are approximate.
- ecesena 8y agoIs there a cloud version, e.g. what we should expect on AWS?
- valarauca1 8y agoThese numbers are based on values from Google's production stack. Insofar as these numbers are based on Jeff Dean's post: http://highscalability.com/blog/2011/1/26/google-pro-tip-use-back-of-the-envelope-calculations-to-choo.html http://highscalability.com/blog/2011/1/26/google-pro-tip-use... That being said, these numbers are roughly identical to what you can expect on AWS.
- moftz 8y agoIt will never change but another latency to think of with regards to hardware is that electrical signals go at the speed of 6in/ns in circuit boards.
- dj43nq 8y agoThis page needs some work my iPad doesn’t render it properly. The Linux box didn’t like it either.
- Waterluvian 8y agoChanging the scale using colour but keeping the block the same really makes it hard for me to gain any sense of scale beyond what I already know.
- mmirate 8y agoWith SSDs, don't the critical determiners of latency and bandwidth become the data transfer buses, rather than the storage media themselves? (especially if one uses an external USB SATA controller, in the vain hope of having storage that is all three of durable, fast and easily-replaced)
- CalChris 8y agoThese latencies should include register access (nominally 1 clock cycle). Compilers work very hard to keep things in registers and it's nice to know the relative benefit of register vs L1.
- harry8 8y agoSeems like in keeping a thing up to date it hasn't kept pace with the way computing has changed. Multicore, numa, extra cache level in heirarchy etc. Lacking: - L3 cache reference - major fault - tlb miss - syscall overhead - Inter-thread latency 64b - threads on same package - Inter-thread latency 64b- across packages - latency to ethernet tx 64b - latency between 2 machines attached to local switch 64 byte transfer Comments?
- stevage 8y agoIf you're a web developer, only one of these is even slightly relevant.
- skyde 8y agocan someone explain this part to me Send 2000 byte over network = 44ns Roundtrip in same datacenter = 500000 ns Isn't a rountrip simply a send from A to B and then a send from node B to node A?
- detaro 8y agoThe first might be just the time it occupies on the wire (although it'd be a 400G network, that's not quite "commodity" yet IMHO...), vs roundtrip being transfer time, time through switches, time through network stacks, ... to send a packet and reply to it as fast as possible.
- skyde 8y agonot sure what you mean by "on the wire". But even if this is assumed to be the time to transfer a UDP packet from one network card buffer to another network card buffer directly connected by a cable this seem extremely low.
- detaro 8y ago"on the wire" as in what you'd see if you'd connect an oscilloscope to the "wire" (which would be fiberoptics at that speed...) and watched how long it took from packet start to end. That could work, but it could also just be an error on the page.
- Mauricio_ 8y agoWhat about your average call to malloc() / new?