16 ms·
How “latency numbers everybody should know” decreased from 1990–2020
- fomine3 5y agoBandwidth per frame is interesting metric. We need more computer power. (I don't know we should call low latency as power)
- cmrdporcupine 5y agoI don't understand the disk I/O numbers here. As others have pointed out, SSD metrics before ~2008 or so are not meaningful, and IOPS on pre-SSD-era machines an order of magnitude worse, but this time traveler doesn't seem to show that.
- mrfusion 5y agoHow did it decrease?
- chairmanwow1 5y agoYeah this link isn’t that useful without a comparison.
- dekhn 5y agolatencies generally got smaller but spinning rust is still slow and the speed of light didn't change
- tejtm 5y agolots till 2005, then not much since
- deleted 5y ago[deleted]
- csours 5y agoThere's a slider at the top. It took me 2 minutes to find it.
- bryanrasmussen 5y agoIt used to take 3 minutes, that's quite an improvement.
- genewitch 5y ago6×10^10 ns (that's a lot of zeros!)
- the-dude 5y agoThe slider is very bad UX : I missed it too at first. It is not pronounced enough, partly because it is all the way to the right. A former boss would say : make it red.
- wolpoli 5y agoIt's really hard to notice the grey slider when the content are already red, green, blue, and black.
- the-dude 5y agoBlinking Magenta then.
- nhoughto 5y agoOh right I was trying to see where the comparison was, totally missed the slider, thanks!
- greggsy 5y agoI sympathise that the site probably wasn’t designed with mobile in mind, but it’s impossible to go beyond 2015 without hitting the GitHub link.
- ygra 5y agoYou can also drag on the main view instead of the slider.
- _cbsz 5y agoThe source displayed at the bottom if this page clearly shows it's just extrapolating from numbers that are older than 2020.
- gregwebs 5y agoAccording to this all latencies improved dramatically except for SSD random read (disk seek only improved by 10x as well). Reading 1 million bytes sequentially from SSD improved 1000x and is now only 2-3x slower than a random read and for disk reading 1 million bytes is faster than a seek. Conclusion: avoid random IO where performance matters. CPU and RAM latencies stopped improving in 2005 but storage and network kept improving.
- csours 5y agoIt looks like almost everything is blazing fast now. I'm not sure how long the first X takes though - how long does it take to establish a TCP/IP connection? How long does it take an actual program to start reading from disk?
- thundergolfer 5y agoIt looks like that because an inaccurate projection from 2012 has been made by the tool. See https://github.com/colin-scott/interactive_latencies/issues/17 https://github.com/colin-scott/interactive_latencies/issues/....
- bmitc 5y agoToday I learned that I don't know any of these numbers that "every" programmer should know. Where do I turn in my programmer card, Jeff Dean?
- morelisp 5y agoYou could take this instead as an opportunity to learn them, instead of reveling in your ignorance.
- bmitc 5y agoThere is plenty I don't know. It's not me reveling in my ignorance. My point is that programming is an incredibly diverse field and yet people, even people who supposedly should know better, are obsessed with making global laws of programming. I know relative comparisons of speeds that have been useful in my day jobs, but I'd wager that needing to know the details of these numbers, how they've evolved, etc. is a relatively niche area. Regarding learning, I try to constantly learn. This is driven by two things: (1) need, such as one finds in their day job or to complete some side project; (2) interest. If something hits either need or interest or hopefully both, I learn it.
- zachberger 5y agoI don’t think it’s important to know the absolute numbers but rather the relative values and rough orders of magnitude. I can’t tell you how many times I’ve had to explain to developers why their network attaches storage has higher latency than their locally attached NVME SSD
- morelisp 5y agoThe absolute numbers are also important. I can't tell you how many times I've had someone coming from a front-end world tells me 5ms for some trivial task (e.g. sorting a 1000ish element list) is "fast" just because it happened faster than their reaction time.
- bmitc 5y agoThat's reasonable. My comment is a reaction to the unneeded title. Knowing relative speeds of things as well as ways to characterize and debug them is generally important.
- kragen 5y agoDoes anyone have a plot of these on a log-linear scale? Where does the data come from? http://worrydream.com/MagicInk/ http://worrydream.com/MagicInk/
- ChuckMcM 5y agoNot an intuitive thing but the data is fascinating. A couple of notes of people who are confused by it: 1) The 'ns' next to the box is a graph legend not a data label (normally that would be in a box labeled legend to distinguish it from graph data) 2) The weird box and rectangle thing on the top is a slider, I didn't notice that until I was looking at the code and said "what slider?" 3) The only changes from 2005 to present are storage and networking speeds. What item #3 tells you is that any performance gains in the last decade and a half you've experienced have been driven by multi-core, not faster processors. And that means Amdahl's Law is more important than Moore's Law these days.
- ianai 5y ago"And that means Amdahl's Law is more important than Moore's Law these days." idk, sure seems like we could have 1-2 cores (permanently pegged?) at 5 ghz for UI/UX then ($money / $costPerCores) number of cores for showing off/"performance" by now. But the OEMs haven't gone that way.
- ChuckMcM 5y agoWe probably see things differently. As I understand it, this is exactly the use case for "big/little" microarchitectures. Take a number of big fast cores that are running full bore, and a bunch of little cores that can do things for them when they get tasked. So far they've been symmetric but with chiplets they needn't be.
- ianai 5y agoYes, for 'computational' loads. I've read though UI/UX benefits the most from fastest response times. I'm talking about the cores which actually draw the GUI the user sees/uses being optimized for the task at the highest possible rate. Then have a pool of cores for the rest of it.
- moonchild 5y agoUI should be drawn on the GPU. Absent rendering, slow cores are more than sufficient to do layout/etc. interactively.
- deleted 5y ago[deleted]
- jeffbee 5y agoI doubt that same-facility RTT has been fixed at 500µs for 30 years. In EC2 us-east-1 I see < 100µs same-availability-zone RTT on TCP sockets, and those have a lot of very unoptimized software in the loop. function getDCRTT() { // Assume this doesn't change much? return 500000; // ns }
- genewitch 5y agoI show 180-350µs between various machines on my network, all of which have some fiber between them. devices with only a switch and copper between them somehow perform worse, but this is anecdotal because i'm not running something like smokeping! Oh, additionally between VMs i'm getting 180µs, so that looks to be my lower bound, for whatever reason. my main switches are very old, so maybe that's why.
- jeffbee 5y agoAre you measuring that with something like ICMP ping? I think the way to gauge the actual network speed is to look at the all-time minimum RTT on a long-established TCP socket. The Linux kernel maintains this stat for normal TCP connections.
- genewitch 5y agoYes, ICMP ping. I don't have any long established sockets anyhow so i don't know how useful my set of those stats would be.
- gameswithgo 5y agoAn instructive thing here is that a lot of stuff has not improved since ~2004 or so, and working around those things that have not improved (memory latency from ram all the way down to l1 cache really) requires fine control of memory layout and minimizing cache pollution, which is difficult to do with all of our popular garbage collected languages, even harder with languages that don't offer memory layout controls, and jits and interpreters add further difficulty. To get the most out of modern hardware you need to: * minimize memory usage/hopping to fully leverage the CPU caches * control data layout in memory to leverage the good throughput you can get when you access data sequentially * be able to fully utilize multiple cores without too much overhead and with minimal risk of error For programs to run faster on new hardware, you need to be able to do at least some of those things.
- greggsy 5y agoIt’s interesting that L2 cache has basically been steady at 2MB/core since 2004 aswell. It hasn’t changed speed in that time, but is still an order of magnitude faster than memory across that whole timeframe. Does this suggest that the memory speed bottleneck means that there simply hasn’t been a need to increase availability of that faster cache?
- ip26 5y agoEvery level of cache strikes a balance between latency & capacity. Bigger caches have higher latency; it's a fundamental property of caches. What you can conclude is that 0.5MB-2MB and 12-15 cycles of latency has been a steady sweet spot for L2 size for twenty years. Sidebar: it was a property of caches. 3D assembly may upend the local optima.
- gameswithgo 5y agothe bigger the cache the longer it takes to address it, and kinda fundamental physics prevents it being faster
- throwawaylinux 5y agoBigger caches could help but as a rule of thumb cache hit rate increases approximately with the square root of cache size, so it diminishes. Then the bigger you make a cache, the slower it tends to be so at some point you could make your system slower by making your cache bigger and slower.
- CSSer 5y agoIs a commodity network a local network?
- kens 5y agoAmazing performance improvements, except no improvement at all on the packet roundtrip time to Netherlands. Someone should really work on that.
- deleted 5y ago[deleted]
- toqy 5y agoWe really need to go back to 1 supercontinent
- dsr_ 5y agoDirect point to point conduits carrying fiber would reduce latency to a worst case of 21ms, but requires a fiber that doesn't melt at core temps (around 5200C).
- gkfasdfasdf 5y agoWe really need that WiFi-Neutrino spec to get approved
- ptspts 5y agoIs it easy to dig down to the core, apart from the temperature issues?
- dsr_ 5y agoNo. The deepest we've ever sent anything is a little over 12Km; the crust's minimum thickness is about 40Km, and the diameter of the Earth is about 12700Km.
- skunkworker 5y agoMaybe if hollow core fiber is deployed we could see a 50% reduction in latency (from .66c to .99c). Past that physics take over, and unfortunately the speed of light is pretty slow.
- warmwaffles 5y agoNow we just need to either beat the speed of light, or speed light up. (thanks futurama)
- gxs 5y agoIt's cool to think about how as humans we're actually running into situations where we have to consider the speed of light. Like, at some point we're really going to wish it were higher than it is. Amazing, can you imagine a few hundred years ago worrying about how fast light travels?
- jandrese 5y agoThe "commodity network" thing is kind of weird. I'd expect that to make a 10x jump when switches went from Fast Ethernet to Gigabit (mid-late 2000s?) and then nothing. I certainly don't feel like they've been smoothly increasing in speed year after year. I'm also curious about those slow 1990s SSDs.
- dgl 5y agoI was curious too, the network one is a TODO: https://github.com/colin-scott/interactive_latencies/blob/master/interactive_latency.html#L117 https://github.com/colin-scott/interactive_latencies/blob/ma... Although I'd expect more jumps than Fast Ethernet to Gigabit, it's true the consumer space is kind of stuck on Gigabit (with 2.5 GbE just becoming more common now). Datacenters have had at least 10 GbE for many years.
- fouc 5y agoIt seems to be entirely based on NIC bandwidth, rather than the actual time it takes for information to travel between servers via ethernet or optics within a LAN.
- jandrese 5y agoDatacenter speeds were a different part of the graph. I thought about this some more and it might be referring to WiFi speeds. Those have been steadily increasing over time with the transition from 802.11b networks of old to modern WiFi 6. Of course WiFi only became a thing in the very late 90s/early 2000s.
- dustingetz 5y agoHow can network be faster than memory?
- zamadatix 5y agoThe memory number is measuring access time while the network number is measuring average bandwidth. The two values can't be compared even though they are presented using the same unit.
- lamontcg 5y agoHow are people practically taking advantage of the increase in speed of SSDs these days compared to network latencies? It looks like disk caches directly at the edge with hot data would be the fastest way of doing things. I'm more familiar with the 2001-2006 era where redis-like RAM caches for really hot data made a lot of sense, but with spinning rust a disk drives, it made more sense to go over the network to a microservice that was effectively a big sharded RAM cache than to go to disk. Seems like you could push more hot data to the very edge these days and utilize SSDs like a very large RAM cache (and how does that interact with containers)? I guess the cost there might still be prohibitive if you have a lot of edge servers and consolidation would still be a big price win even if you take the latency hit across the network.
- gameswithgo 5y agoI don’t know, I have observed in my workloads, booting, game load, and building programs, that super fast ssds make almost no difference compared to cheap slow ssds. But any ssd is miraculous compared to a spinny drive Presumably video editing or something might get more of a win but I don’t know.
- noizejoy 5y agoWhen I got my first NVMe SSD, I was disappointed that it wasn’t significantly faster than my SATA SSD. But soon I realized that it was Samsung’s Magician software that made the SATA SSD competitive with an NVMe SSD via RAM caching.
- lamontcg 5y agoI'm talking about back in the day when I worked at Amazon with thousands of servers and in the 2001-2006 era we had basically no SSDs and it was all spinny rust (set the slider to 2006 and that is about my mental model). Spinny rust was always shit, so it was all about RAM and network and then the long tail siting on disk. I'm wondering how datacenter SSDs impact architectural decisions with e.g. 'microservices' by having that new latency layer which is cheaper than RAM and faster than network.
- ohazi 5y agoDiminishing returns over the last decade, as expected. It would be interesting to look at the energy consumed by each of these operations across the same time periods.
- _pastel 5y agoThese numbers focus on reads. How does writing speed to cache, main memory, or disk compare? Anyone have some ballparks to help me build intuition?
- favorited 5y agoWrites are much slower than reads, but modern CPUs have a write-buffer for each core so they don't need to wait for the write to propagate back down the caches and into main memory before continuing execution.
- rvr_ 5y ago20 years without meaningful improvements on memory access ?
- gameswithgo 5y agoyep, got any ideas?
- aidenn0 5y agoGood news is that SSDs are only 160x slower for random reads, so maybe we should just beef up L3 or L4 cache and get rid of ram? /s
- not2b 5y agoIt takes at least one clock cycle to do anything, and clock frequency stopped increasing in the 2003-2005 time frame, mainly because of the horrible effects on power with very small feature size.
- jll29 5y agoThe time to open a Web browser seems roughlz constant since 1993.
- dekhn 5y agoyou mean andreesen's constant?
- dweez 5y agoOkay since we're not going to improve the speed of light any time soon, here's my idea for speeding up CA to NL roundtrip: let's straight shot a cable through the center of the earth.
- DonHopkins 5y agoWe could really use some decent Mexican food here in the Netherlands. https://idlewords.com/2007/04/the_alameda_weehawken_burrito_tunnel.htm https://idlewords.com/2007/04/the_alameda_weehawken_burrito_...
- almog 5y agoWhat's more, we can run that cable along a gravity train from CA to NL, saving the costs of digging another tunnel. :)
- Archelaos 5y agoFrom CA you will end up off the coast of Madagaskar, and from the NL somewhere near New Zealand. You do not have to go very deep inside the earth to get straight from CA to NL.
- banana_giraffe 5y agoAssuming my math is right, it'd be a 10% faster trip, but I'd be all for seeing that tunnel!
- Arwill 5y agoRead somewhere that neutrino particles are interesting in this regard, as they travel straight trough the earth.
- mhh__ 5y agoThey would go through the earth at the speed of light but detecting, let alone manipulating neutrinos is a very non-trivial task.
- farzher 5y agothe internet doesn't travel at the speed of light. i think it's around 2/3 the speed of light. could still reduce that ping by about 33%
- sigstoat 5y agothe code appears to just do a smooth extrapolation from some past value. it claims that (magnetic) disk seek is 2ms these days. since when did we get sub-4ms average seek time drives? it also seems to think we're reading 1.115GiB/s off of drives now. transfer rate on the even the largest drives hasn't exceed 300MiB/s or so, last i looked. ("but sigstoat, nvme drives totally are that fast or faster!" yes, and i assume those fall under "SSD" on the page, not "drive".)
- thundergolfer 5y agoYes, this is a huge problem with the data for 2012 onwards. See: https://github.com/colin-scott/interactive_latencies/issues/17 https://github.com/colin-scott/interactive_latencies/issues/....
- bob1029 5y agoLatency is everything. I believe that sometime around 2010 we peaked on the best software solution for high performance, low-latency processing of business items when working with the style of computer architecture we have today. https://lmax-exchange.github.io/disruptor/disruptor.html https://lmax-exchange.github.io/disruptor/disruptor.html I have been building systems using this kind of technique for a few years now and I still fail to wrap my brain around just how fast you can get 1 thread to go if you are able to get out of its way. I caught myself trying to micro-optimize a data import method the other day and made myself to do it the "stupid" way first. Turns out I was definitely wasting my time. Being able to process and put to disk millions of things per second is some kind of superpower.
- zepolen 5y ago> made myself to do it the "stupid" way first. Turns out I was definitely wasting my time I think you put into words something that has been apparent for a while now with all the bloated software out there. Software has become stupid. Not because we're not capable of doing it smart, rather it's a accumulation of small instances of 'I'll do it fast and stupid' like that import, which 10-20 years ago would have been a long enough delay for the developer to do something about it. Then across the multiple dependencies and imports you end up with an overall slow system since every one of those was implemented with a 'stupid but fast enough for this' mindset.
- bb123 5y agoThis site is completely unusable on mobile
- jmyeet 5y agoThis could be a really great visualization and really interesting but I literally nad to look at he comments here to figure out how to read it and use it. For example, you can edit the year text box. I would suggest adding graphs to show how different things have changed over time. it is interesting that the only major improvement over the last 15+ years seems to have been the invention of SSDs.
- conor_f 5y agoIs there an equivalent with energy costs?
- throwaway81523 5y ago"Commodity network" is 400 gbit/sec?! I want that. I mean, it exists, but is still pretty exotic afaik.
- torginus 5y agoThis isn't particularly helpful/insightful. Considering that, for example, waiting for cache is meaningfully expressed in processor cycles, having that piece of data in ns isn't particularly useful. Mutex locking perf also is a bit suspect, since probably nobody had multi-core CPUs pre-2003, meaning mutex locks/unlocks were simple memory writes, no shared cache invalidation involved. The only latency stats that can be meaningfully expressed in ns are the network and disk latencies.
- lucb1e 5y agoIf it was an hour earlier I'd save everyone time with a graph instead of trying to get an idea by eyeballing blocks while playing with a slider, but alas it's 1:09am and I really should sleep. I think a simple multi-line plot/graph made using a simple spreadsheet or whatever would be a lot more useful than having to keep in mind different values and moving time forwards manually, so if anyone else feels encouraged...
- spion 5y agoSSD read speed for 1MB today is the same as RAM read speed for 1MB in 2008. I wonder what kind of implications this has for designing databases... or swap files.
- threatripper 5y agoWe have memory mapped files now which look to the program like ordinary RAM but are backed by disk storage. (You still need to call sync to actually write back to disk.)
- weibing 5y agoI hate all the articles whose title includes a “should”.
- Lamad123 5y agoI like this but I wish this had some kinda log scale so I don'tt have to scroll deep down the page!!
- thundergolfer 5y agoIt's very important to realize that this tool makes inaccurate projections from 2012 data. It is not at all accurate for 2020. See: https://github.com/colin-scott/interactive_latencies/issues/17 https://github.com/colin-scott/interactive_latencies/issues/...
- ZainRiz 5y agoWoah, this code hasn't been updated in 9 years D:
- peterfield 5y agoComplete laggard questions: * Did SSD exist in 1993? * Why using an "approximately equal" sign (as opposed to a real = sign) between e.g. 1000ns and 1 micros ? (I would undersatnd between binary and decimal multiples of bytes...but decimal fractions of seconds don't work this way)
- deanmen 5y ago1024 ns = 1 micro second ;)
- peterfield 5y agoTo my knowledge the second is still a metric system unit, not a "SW engineering geek system unit". Wikipedia agrees https://en.wikipedia.org/wiki/Nanosecond https://en.wikipedia.org/wiki/Nanosecond
- farzher 5y agothe html formatting is whack. also where's the switch from HDD to SSD ?
- eternityforest 5y agoThe really interesting thing is that at one point, cache misses were a thing "every programmer" probably did need to know.
- lofatdairy 5y agoI feel like the boxes are comically unintuitive given that it's nearly a 50/50 split on which boxes just there to convey scale.
- deleted 5y ago[deleted]
- tashbarg 5y agoNeeds desperately a 2012 label in the title!
- knorker 5y agoWould be nice to see it in relative terms.
- yolo69420 5y agoI wish there was a version that normalized the times somehow instead of showing absolute times. Most of these numbers will get smaller simply because CPUs get faster, but it would be super helpful to get an accurate read whether the ratios of e.g. l1 to l2 to l3 to main memory lookup latencies changed significantly.
- deleted 5y ago[deleted]
- yolo69420 5y agoin 2006 cpus made a momentous breakthrough by breaking l1 cache access time barrier of 1.0ns and reduced it to 1ns
- toxik 5y agoPage is absolutely foobar on phone. Safari.
- ahaferburg 5y agoIn case anybody else didn't know, Zippy = https://en.wikipedia.org/wiki/Snappy_(compression) https://en.wikipedia.org/wiki/Snappy_(compression)