7 ms·
Latency Numbers Every Programmer Should Know (2012)
- pieterr 6y agoReminds me of Dan Luu's various latency articles. https://danluu.com/input-lag/ https://danluu.com/input-lag/
- Remnant44 6y agoThis gets posted here frequently - I submitted it a year ago - but imo that's not a problem because it's insightful and useful. Should have a (2012) tag though. I also ran across this version that updates the numbers to 2020 values: https://colin-scott.github.io/personal_website/research/interactive_latency.html https://colin-scott.github.io/personal_website/research/inte... edit: it appears they're just estimating for a given time above, not measuring...
- pvg 6y agoPreviously: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=Latency%20Numbers%20Every%20Programmer%20Should%20Know&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
- dang 6y agoThe threads with comments: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=comments%3E2%20Latency%20Numbers%20Every%20Programmer%20Should%20Know&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- pvg 6y agocomments>2 might be a good default qualifier for the 'past' link.
- Waterluvian 6y agoHaving a solid mental model for “how fast is fast” is, in my opinion, critical to what makes an excellent engineer: knowing when to care about performance up front. And not even big O notation of algorithms or IO latency. But just a general feel for the performance of higher level abstractions. To look at a design that involves some input, data processing, a transfer somewhere, rendering, presentation, or whatever, and to instantly have an intuition on what parts to worry most about.
- kevin_thibedeau 6y agoWhen I get onto an unfamiliar platform I do a test of sequential search vs binary search to see where the crossover point is on number of elements for small arrays. Then I know roughly when not to bother with a better algorithm.
- RhysU 6y agoHow fast is fast can be known, but how important are tradeoffs is a better way to test an engineer.
- vvanders 6y agoI would much prefer people know the scale of these numbers than know big O notation. The number of times I've seen big O used as a hammer because everything looks like a nail can be frustrating. This is doubly true on embedded systems where N > 3000 tends to be the less common case. Most of the time when I do performance profiles it's a 'flat profile' because the majority of the time is lost thrashing the L1/L2 cache.
- meisel 6y agoThe % of programmers that actually need to know any of these numbers is tiny. I've done plenty of optimizations for performance-critical systems and have never known any of these numbers. Certainly, I know general principles like "disk is a lot slower than memory" and "don't block the UI on a network call". But knowing whether an L1 cache read is 0.5ns or 5ns has never been necessary. You can optimize so much "highly optimized" code out there without those numbers. I'm sure there are some people that have to know that stuff, maybe a couple people on an AAA video game title, or computer engineers at Intel. But it's the exception and not the norm.
- pvg 6y agoThis is one of the most popular answers on SO. The effects of these latencies can easily become visible in pretty vanilla programming. https://stackoverflow.com/questions/11227809/why-is-processing-a-sorted-array-faster-than-processing-an-unsorted-array https://stackoverflow.com/questions/11227809/why-is-processi...
- chrisandchris 6y agoWhile I totally agree with the argument that everyone should think sonetimes a bit more about the impact of his code (or the execution of it) the relationship between your mentioned SO article and the gist ist close to zero. There‘s no way from conclude from „memory latency“ to „branch prediction“.
- pvg 6y ago'Branch mis-predict' is the second item in the Latency Numbers thing.
- The_rationalist 6y agoThe actual number is not used on SO and moreover most of those numbers should be obscolete in 2020.
- formalsystem 6y agoNoBugsHare has an amazing chart https://twitter.com/NoBugsHare/status/1022129373292445696?s=20 https://twitter.com/NoBugsHare/status/1022129373292445696?s=...
- quietbritishjim 6y agoMutex lock/unlock 25ns Main memory reference 100ns Honest question: how can this be right? Surely living a mutex requires synchronising across the cores of a CPU, which requires at least as much time – probably quite a bit more – than an uncached access to memory?
- dragontamer 6y agoMutex lock/unlock is commonly over L3 cache on modern systems. It won't hit main-memory.
- chrisseaton 6y ago> Surely living a mutex requires synchronising across the cores of a CPU Right it's done over cache coherence, which doesn't need to reach all the way out to main memory. And that's only if it's contested.
- dragontamer 6y agoA contested mutex lock / unlock is over L3 cache. Uncontested mutex lock/unlock is just a swap instruction followed (or in "unlock": preceded) by a memory barrier. The flush pushes data to L3 cache, where it can be shared between multiple cores. (L1 and L2 cache is local to a core). L3 cache is far closer than main-memory, and roughly 20ns these days. ------- In "actuality", its more of a MESI message, but in the abstract its your L3 caches communicating with each other and synchronizing.
- chrisseaton 6y agoNot sure if you're agreeing with me or contradicting me? Isn't that what I said?
- GeneralMayhem 6y agoFunnily enough, there is another version of the slides these numbers come from in which mutex lock/unlock is listed as the same 100ns as memory access: http://static.googleusercontent.com/media/research.google.com/en/us/people/jeff/stanford-295-talk.pdf http://static.googleusercontent.com/media/research.google.co.... I can't tell which came first, or if one is a typo, or what's going on there. In any case, assuming the mutex in cache, and appears to not currently be locked, the core performing the atomic operation needs to: (1) broadcast an invalidate/exclusive hold on the cache line to the other cores, (2) update its own cache, (3) eventually write back to memory. (1) is faster than main memory access, since CPU cores are closer to each other, physically, than they are to the memory bus, and have hard-wired access specifically for this operation. The slowest part is (3), but with a write-back cache the writer doesn't really pay the latency cost, because it gets amortized into the next cache flush.
- lbacaj 6y agoThis was just posted and already there are two types of comments: 1. Most devs don’t need this, it’s not so helpful to know etc. 2. These are critical numbers to know and in the very least devs should know these numbers. This sort of disagreement is common in our industry, it’s not just this it’s also Big O, algorithms and data structures, and even OS fundamental people disagree on. I’d love to take both groups of devs commenting and give each group a set of programming tasks to complete. Judge them based on correctness, speed of development, speed the tasks run, the quality of the code etc, etc. I think the results would be profound.
- bJGVygG7MQVF8c 6y ago> set of programming tasks You set off in the right direction but unfortunately haven't engaged with the heart of the matter at all. That there is no easily measurable "set of programming tasks" in common between even highly proficient developers of different types is the point. The core comms failure here is that "software engineer" is too broad a term. We're actually discussing superficially similar but essentially different professions that aren't yet acknowledged as such.
- nullsense 6y agoYou'd then also have to give each dev a task representative of the kind of work they nah do on a daily basis. I think you'd find probably people are good at what they spend time doing a lot and tend to brush off things they don't as not important.
- sverhagen 6y agoIs it but because our jobs are so varied? My team implements what I would call "business logic" for a (relative to Google scale) small number of users. We often frown at algorithmic coding exercises, like what we believe Google would ask in interviews, using the argument: we suppose that makes sense if you have to optimize compression of YouTube's video streams, which we don't have to... (Rather, our small, ambitious team emphasizes optimization for maintainability over performance.)
- taneq 6y ago
- deleted 6y ago[deleted]
- stefan_ 6y agoLoading a webpage that is considered "lightning fast" - 1 second Of course the common opinion expressed here is people couldn't care less for those numbers - it's hard to get them to care that the website they are working on can't finish rendering in the time it takes anyone to pour a cup of coffee.
- Sirupsen 6y agoMore numbers with accompanying code: https://github.com/sirupsen/napkin-math https://github.com/sirupsen/napkin-math
- lalwanivikas 6y agoHey man! Love your newsletter but many times I am just too lazy or occupied to properly read through them :/ Since you already are on YouTube, my one suggestion to improve the adoption would be to make videos on them. I know it might be super time consuming but imo it will reach more people.
- wombatmobile 6y agoConsumer latency caused by cookie syncs https://s6.io/consumer-latency-caused-by-cookie-syncs/ https://s6.io/consumer-latency-caused-by-cookie-syncs/ Latency in Digital Advertising: A Guide for Publishers https://blog.ad-juster.com/latency-in-digital-advertising-a-guide-for-publishers/ https://blog.ad-juster.com/latency-in-digital-advertising-a-... Case Study: Ads Increase Page Load Times by 40 Percent https://rigor.com/blog/ads-increase-latency-by-40-percent/ https://rigor.com/blog/ads-increase-latency-by-40-percent/
- twotwotwo 6y agoIn "The Datacenter as a Computer" there was an interesting diagram sketching a datacenter memory hierarchy latency and capacity and bandwidth to access RAM, SSD, and HDD in the same machine, the same rack, and a larger cluster. It's page 72 of https://www.morganclaypool.com/doi/pdf/10.2200/S00874ED3V01Y201809CAC046 https://www.morganclaypool.com/doi/pdf/10.2200/S00874ED3V01Y... and with luck https://books.google.com/books?id=Td51DwAAQBAJ&pg=PA72 https://books.google.com/books?id=Td51DwAAQBAJ&pg=PA72 will get you to the page. The numbers have changed (commodity SSDs have markedly improved!) but the idea still makes sense. Fun to consider that boxes under one top-of-rack switch can hold dozens of TB of RAM and hundreds of TB of Flash with latency measured in µs between them. There's legitimately interesting space in between writing code for a single machine and code for huge, spread-out clusters of thousands of boxes.
- smusamashah 6y agoPer year version https://colin-scott.github.io/personal_website/research/interactive_latency.html https://colin-scott.github.io/personal_website/research/inte...
- nojvek 6y agoI loved the latency numbers posted by one stackoverflow engineers Nick Craver: https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we-do-app-caching/ https://nickcraver.com/blog/2019/08/06/stack-overflow-how-we... L1: 1.3ns L2: 3.92ns (3x slower) L3: 11.11ns (8.5x slower) DDR4 RAM: 100ns (77x slower) NVMe SSD: 120,000ns (92,307x slower) SATA/SAS SSD: 400,000ns (307,692x slower) Rotational HDD: 2–6ms (1,538,461x slower) Microsoft Live Login: 12 redirects and 5s (3,846,153,846x slower, approximately) “How many Microsoft Live Logins before this thing loads?”
- skytreader 6y agoGreat idea! I think this is the way to make this more relevant and easier to consume. Besides, I think the point of the exercise is not to "know" that reading 1MB sequentially from memory is 250K ns but rather that it is worth 50K branch mispredicts. Knowing these numbers won't make you a better engineer, but being able to compare them (and influence your design decisions with it) will. Though every now and then you get some time-critical tasks in which case the actual numbers might matter. But I imagine at that scale so much more factors will influence your execution that ballpark estimates are nowhere close to tight. Sure 1KB over 1Gbps is 10K ns and you can extrapolate from that how long your 1TB data migration will take. Then it's more worthwhile to check Grafana and reason out from the bandwidth trend.
- PaywallBuster 6y agoDo we need an update? Latency between AWS AZs (2-3ms) Latency to NVMe Latency to NAS vs directly attached (Cloud options) Latency to external RAM (RAMCloud?)
- saagarjha 6y agoThis reminds me of one of the few good r/ProgrammerHumor posts from this year: https://old.reddit.com/r/ProgrammerHumor/comments/g4cr9m/my_i3_is_working_hard/ https://old.reddit.com/r/ProgrammerHumor/comments/g4cr9m/my_...
- willemlabu 6y agoThe link rot on that page frustrates me :/
- known 6y agoWith C programming variables can we access L1/L2 cache
- bor100003 6y agoIt's great if you know them, but you can put "Every programmer should know" on everything you use daily and post it. It looks like the clickbait articles "N books every programmer should read" and see the thick books on algorithms.
- lazyant 6y agoHow do you folks think regarding network bandwidth vs disk throughput? I used to think disk was way slower but now if we look at AWS ec2 instances, they are within one order of magnitude of each other: Network: ec2 instances (except tiny ones) 1-2 Gbit/s Baseline, 10 Gbits/s Burst disk: EBS volumes, general purpose, throughput 128 MiB/s - 250 MiB/s or ~ 2 Gbit/s (1,000 MiB/s for io optimized) So the top of the baseline network bandwidth is the same as the top of the disk throughput, or am I wrong?
- brokenmachine 6y agoI found this article from coding horror about the same thing interesting - "The Infinite Space Between Worlds". https://blog.codinghorror.com/the-infinite-space-between-words/ https://blog.codinghorror.com/the-infinite-space-between-wor...