6 ms·
What every programmer should know about memory (2007) [pdf]
- AnimalMuppet 3y ago(2007).
- bediger4000 3y agoDoes the age affect how I, as a programmer, should read it?
- mdaverde 3y agoLiterally, yes. As example, the north bridge isn't as explicit today as the document makes it seem. A lot of functionality that used to be reserved for the north bridge is now being tucked away into the CPUs or motherboards themselves. Still, worth reading.
- burnished 3y agoYes, because the advice is necessarily hardware dependent.
- wmf 3y agoThis is why I think this document was too detailed and should have omitted low-level details. The way you optimize your code hasn't really changed since 2007, but the hardware churn since then triggers people to say "well akshually the northbridge..."
- cbsmith 3y agoYes and no. A lot of the concepts about memory management and how memory functions are still the same or similar. It's a good baseline to start with, and then look at how things have changed since then (they certainly have changed, but most of the paper still applies).
- AnimalMuppet 3y agoPerhaps yes. Multiples cores are now much more a thing than they were in 2007; that at least could be a change in emphasis. OSes may do things differently in terms of how they implement memory mapping. And so on.
- kevstev 3y agoTbh, this paper is a bit much, unless you have a lot of time on your hands for it, and also the necessary understanding of CPU architecture to really grok it. I think there are better blog posts out there that explain the concepts. "Cache locality" "memory cost of context switching" are some terms to put in search engines. Maybe add "thread local storage" in there too. The big takeaway from this paper though, at least when I first read it, and the concept has risen to prominence in the years since- backed up by benchmarks- that Big O isn't really the end of the argument when it comes to performance. The way CPUs work is that its extremely fast to rip through contiguous datasets. Jumping around to random memory addresses is way slower. A corollary to that is that jumping between threads is very slow for the same reasons- loading and unloading random chunks of memory. There are a lot of details around the specifics of why though- different cache levels and their latencies, and what happens when there is a cache miss, etc.. This lead HFT type programs to favor linear arrays to store things, and pinning threads to CPU cores, amongst other things, but that's the gist of it.
- kaycey2022 3y agoWhat would be an up to date alternative to this document?
- PurplePants 3y ago[flagged]
- redhal 3y agoFor a great read with more recent numbers, I recommend you checkout this post https://news.ycombinator.com/item?id=22287993 https://news.ycombinator.com/item?id=22287993
- alberth 3y agoFYI - this doesn't seem to be related to FreeBSD (even though it's hosted there). It just appears to be an external PDF that FreeBSD committer Lawrence Stewart saved to their fileshare, written by someone else.
- cbsmith 3y agoYeah, there's a pretty famous history on this paper (and with Ulrich Drepper).
- quackulus 3y agoPlease do tell
- cbsmith 3y agoHe's been the maintainer of glibc (still a developer), and a variety of other key bits of open source software. He's kind of famously abrasive/obtuse: https://news.ycombinator.com/item?id=2378013 https://news.ycombinator.com/item?id=2378013. The paper was kind of a canonical work, referenced by most everyone (which is no doubt why there's a copy of it on the FreeBSD site).
- everyone 3y agoI dunno, I'm a game programmer and it seems a bit too much for me personally. I just John Carmack it, and always do the simplest thing for me as the dev. If the benchmarks arent fit for purpose¹ then I will look at profiler to see what lowest hanging fruit is to optimise. I'm not gonna guess what is slow, I am not a computer, I only just look at profiler. ¹This almost never happens.
- sbmthakur 3y agoWhat does "John Carmack it" means? Take the simplest and correct approach?
- everyone 3y agoLike what I described, start with the utter simplest/easiest code for you the developer, and only optimise it/make it more complicated, later if it's benchmarked and not up to a standard you are aiming for (of feature quality or performance or whatever). There was a bit about it in that book "Masters of DOOM" I also personally equate simplest/easiest code for you with most maintainable code also.
- dragontamer 3y agoAh yes, John Carmack. The developer who is known for writing simple, easy to understand code. float Q_rsqrt( float number ) { long i; float x2, y; const float threehalfs = 1.5F; x2 = number * 0.5F; y = number; i = * ( long * ) &y; // evil floating point bit level hacking i = 0x5f3759df - ( i >> 1 ); // what the fuck? y = * ( float * ) &i; y = y * ( threehalfs - ( x2 * y * y ) ); // 1st iteration // y = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration, this can be removed return y; }
- mhh__ 3y agoCarmack didn't write that
- 3y ago
- frithsun 3y agoWhat only a small subset of highly specialized programmers should know about memory (2007) [pdf]
- santiagobasulto 3y agoI'm a very high level programmer, turned Data Scientist, turned founder (I successfully exited my prev startup). I still read this paper (a few years ago). I guess if you're "born" a programmer, all these nerdy/curious things are entertaining. One of my "developer" passions is multithreading and concurrency. Even now running my second startup, I, from time to time, sit to write some concurrent puzzle programs. I also like to follow Tobi Lütke. The guy is the founder/CEO of Shopify (with all which that might entail), and he still tweets amazingly interesting low level coder stuff.
- niuzeta 3y ago> sit to write some concurrent puzzle programs. I concur. There's something about concurrency and multithreading that ticks your brain in a weird way. It keeps me stimulated. My favorite is setting up a scenario that would normally be considered as a disasterous concurrency failure. Like intentionally messing up a `volatile` state. You can't ever have those in prod, of course, but seeing them in a controlled, _expected_ way tickles my brain in a very perverse way.
- sbmthakur 3y agoAny sample of your puzzle program that you may be willing to share? I want to perform my own experiments with multi threading but being a Javascript dev, it's not something I deal with quite often. Also great work on DataWars. The onboarding experience and the site layout is pleasant.
- mhh__ 3y agoThe principles here apply at basically every level of the stack.
- lizardking 3y agoYes, every developer needs 114 pages worth of knowledge about memory.
- mrwnmonm 3y agoYeah, I read some of it before, and had no idea why I would need this.
- mmkos 3y agoWhenever articles like this bubbled up, I used to want to delve right in to satisfy some inner need to be deeply knowledgeable about the craft. I don't feel that way anymore now that I'm in my 30s, something flipped. Now I'm much more interested in higher-level abstractions and building complete products that solve some real problems, rather than occupying myself learning about the nitty gritty details of how software works under the hood. I think it's maybe nice to know how we interact with memory in software (memory allocation/deallocation, naybe how manual memory mgmt is done, something about garbage collection?), but that's about it.
- vbh21 3y agoI feel the same wayy now that I'm older. The nitty-gritty of some of the memory management stuff just isn't relevant. The work I'm doing now is at a different scale and I let the language designers worry about this. It is still useful to know some details as they apply when I'm programming, but a lot of this stuff just doesn't command my attention!
- mhh__ 3y agoYes, why not? You don't have to understand all of it but if you can't be bothered to get the gist of this maybe computers aren't for you? Note that I'm implicitly making a distinction between people who write code and people who are explicitly developers.
- lizardking 3y ago
- utopcell 3y agoI've read this doc many years ago, and I remember it was worth the time. If folks are contemplating on what has changed since 2007 that this doc is not covering, I'd say it is CCX awareness for AMD CPUs.
- karmakaze 3y agoI get by with latency numbers every programmer should know[0]. Knowing a bit about how cache sharing works is helpful some times[1]. [0] https://gist.github.com/hellerbarde/2843375?permalink_comment_id=1753093#latency-numbers-every-programmer-should-know https://gist.github.com/hellerbarde/2843375?permalink_commen... [1] https://en.wikipedia.org/wiki/False_sharing https://en.wikipedia.org/wiki/False_sharing
- antonvs 3y agoHardware memory characteristics may have changed in the last 16 years, but the attractions of misleading clickbait titles have not.
- harry8 3y agohttps://stackoverflow.com/questions/8126311/how-much-of-what-every-programmer-should-know-about-memory-is-still-valid https://stackoverflow.com/questions/8126311/how-much-of-what... Discussion of it in slightly more modern context, worth looking at.
- tunnuz 3y agoIs there an updated version of this?
- Am4TIfIsER0ppos 3y agoIn this thread: the reason why software is so shit in 2023
- dragontamer 3y ago1. RAM is a parallel machine of its own. This document describes DDR3 (which is 2 versions out from modern DDR5), but that doesn't really matter. Today's DDR5 is just more parallel. 2. This document very precisely describes the parallelization mechanism of DDR3. Read/writes to DDR3 are parallelized by channel, rank, and bank. DDR4 added bank-group as an additional layer of parallelism (above bank below rank). 3. An individual bank has performance characteristics: an individual bank is much faster when it can stay "open", thus only getting a CAS-latency, rather than a close+precharge+RAS+CAS latency cycle. 4. CPUs try to exploit this parallelism with various mechanisms. Software prefetching, hardware prefetchers, levels of cache, and so forth. 5. NUMA is more relevant today as EPYC systems are innately quad-NUMA. HugePages is also more relevant today as 96MB+ L3 per CCX have appeared on Ryzen/EPYC machines, meaning you need HugePages to access L3 without a TLB hit. TLB isn't really RAM, but its closely related to RAM and could be a bottleneck before you hit a RAM bottleneck, so its still important to learn if you're optimizing against RAM-bottlenecks.
- jeffffff 3y agoone thing that is missing from this document is that there are a limited number of line fill buffers in the L1 cache of each core, which limits the number of concurrent memory accesses that can be in flight well before anything related to channels/ranks/banks begins to matter. https://lemire.me/blog/2018/11/05/measuring-the-memory-level-parallelism-of-a-system-using-a-small-c-program/ https://lemire.me/blog/2018/11/05/measuring-the-memory-level...