14 ms·

What Every Programmer Should Know About Memory (2007) [pdf]

214 points by jxub 8y ago

busfahrer 8y ago
Ulrich Drepper used to be the glibc maintainer, IIRC
- v_lisivka 8y ago
  memcpy != memmove , $#$@#@ https://sourceware.org/bugzilla/show_bug.cgi?id=12518 https://sourceware.org/bugzilla/show_bug.cgi?id=12518
  - okl 8y ago
    https://sourceware.org/bugzilla/show_bug.cgi?id=3266 https://sourceware.org/bugzilla/show_bug.cgi?id=3266 https://sourceware.org/bugzilla/show_bug.cgi?id=12701 https://sourceware.org/bugzilla/show_bug.cgi?id=12701 https://sourceware.org/bugzilla/show_bug.cgi?id=386 https://sourceware.org/bugzilla/show_bug.cgi?id=386 https://sourceware.org/bugzilla/show_bug.cgi?id=10134 https://sourceware.org/bugzilla/show_bug.cgi?id=10134
    - cryptonector 8y ago
      Classics.
    - cls59 8y ago
      Also: https://sourceware.org/bugzilla/show_bug.cgi?id=10354 https://sourceware.org/bugzilla/show_bug.cgi?id=10354
CalChris 8y ago
I wish Ulrich Drepper (thank you, Mr. Drepper) would update this with a section on Row Hammer and also Spectre and Meltdown. Programmer's need to know about memory because of these exploits, more so with the latter two in order avoid creating exploitable gadgets. But then I also think that What Every Computer Scientist Should Know About Floating-Point Arithmetic should be updated to include UNUMs. I don't think that will happen either. Also, thank you Mr. Goldberg.
- 131012 8y ago
  Thanks for bringing that up, I came to the comments section to see if some other details were outdated. Any other insights?
  - CalChris 8y ago
    Well, I think hardware and software prefetch, speculation is a special form of prefetch, might be revisited. Software prefetch was at best a hopeful technology even in 2007 and is widely avoided today. https://yarchive.net/comp/linux/software_prefetching.html https://yarchive.net/comp/linux/software_prefetching.html Something I don't think either Drepper or Hennessy + Patterson's books get across is memory banks from a programmer's perspective. How cache organization affects a program is explained well but how banks affect said same isn't. Construction yes. Visibility, no.
- Retric 8y ago
  Row Hammer, Spectre, and Meltdown are hardware design flaws which don’t nessisarily stay meaningful over time. Further, they are completely irrelevant for many developers. The next Mars rover for example is not going to be running untrusted code.
- nestorD 8y ago
  What Every Computer Scientist Should Know About Floating-Point Arithmetic is great because it makes people aware of problems with the current system. Adding UNUMs would be suggesting the devil you don't know over the devil you know. However, and I can't remember if it is alreaddy in the book, a section on compensated summation / dot product would give strong time tested tools to attack the problem from within the current framework.
louthy 8y ago
What every programmer should know about memory shouldn't be 114 pages long.
- twtw 8y ago
  If you are objecting to "every," then sure, not every programmer needs to know anything about memory. You can program without knowing that there is such a thing. But it's a fun title, and an excellent resource.
  - oarabbus_ 8y ago
    No, he's right. It's more than simply semantics. A better title would be "an extremely overwhelming of programmers don't need to know THIS much about memory"
    - netmonk 8y ago
      This is what people says when they have a dataset of 100 entries(<100mb) on their developpment laptop. How funny they look when production guys show them how their lame piece of code doesn't scale up with billion of entries in the production platform...But they dont care cause they are developpers, (or so called). Until the production guy rewrite their code and have them fired for incompetence (true story).
      - Chlorus 8y ago
        > How funny they look when production guys show them how their lame piece of code doesn't scale up with billion of entries in the production platform...But they dont care cause they are developpers, (or so called) Why did the production guy even need them in the first place? Why were they hired? > (true story). And that production guy's name was Albert Einstein.
    - abraae 8y ago
      Or perhaps some more positive phrasings: - If you know this much about memory, you'll know more than 99% of programmers. - What 99% of programmers don't know about memory.
saberience 8y ago
Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, <insert language here> programmer would need to know about transistor configurations or voltages, pin configurations, etc. Let alone 114 pages of low level hardware diagrams and jargon! This document is for someone working on low level drivers for memory, or working on the hardware side. For any normal software engineer, this information is not helpful for doing your job.
- gautamdivgi 8y ago
  It depends. I find that most programmers not knowing some of this happens to be the bane of most performance and reliability issues. Take Java for example - pretty sure the Java dev needs to know how to optimize their jvm (memory settings, etc.). They would need to know direct memory, etc. and how it relates to the operating system. This nastier your traffic profile the more important the tuning described in that document are. Like all good "books" I don't remember all of it but keep coming back to refer to it. But again this is speaking from my experience as an SRE/SDE (or a perf/reliability engineer).
- dragontamer 8y ago
  I would argue that the overclocker needs to know more about these details than most programmers, yes. (Overclockers actually tweak these values to maximize the performance of their computer). But any high-performance programmer needs to understand the RAS / CAS / PRE cycle, if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. If you are accessing RAM randomly, you better be sure its within L3 cache (or nearer). I've done some experiments, and "streaming" data from beginning to end can be 2x to 3x faster than random access on modern DDR4 RAM. Understanding the RAS / CAS / PRE cycle helps me understand why streaming data to RAM is faster. And understanding that cells are simply capacitors helps me understand why the RAS / CAS / PRE cycle is necessary in DRAM.
  - 8y ago
kevstev 8y ago
There are some salty comments here, but I think the context is important. This paper passed across my desk in early 2008 when I was doing HFT stuff. It might be a bit of a stretch to say that the reason people are taught about cache lines in most CS programs is because of this paper, but at the time this paper was written, this was really specialized knowledge and groundbreaking to most software developers. This would go on to be a popular topic on C++ blogs from Important People (Boost maintainers, STL devs, etc) at least for the next 5 years. Also, if you know Ulrich Drepper at all, either from some of his talks or his mailing list presence, this is just a very fitting title from him. Just pure deadpan, you think its funny, he probably does not, the fact that you think its amusing is just disappointing him like a professor looking out at freshman undergrads wondering how he got stuck teaching this class.
- netmonk 8y ago
  I really do agree with you. HAving done HFT for some years, this paper was crucial when it was running on linux system. Now FPGA took over the field. Different kind of techs.
- dewhelmed 8y ago
  He mentions his reason for the title: > The title of this paper is an homage to David Goldberg’s classic paper “What Every Computer Scientist Should Know About Floating-Point Arithmetic”
mjw1007 8y ago
Interesting that in 2007 he thought FB-DRAM was going to win. That seems to have been about the time it dropped dead.
- Const-me 8y ago
  Right, also about NUMA: > It is expected that, from late 2008 on, every SMP machine will use NUMA. Outside servers, still not happened. They are not that exotic anymore and are no longer exclusive to very expensive servers, e.g. Threadripper 2920X is a $650 CPU, but market penetration is still low.
orzig 8y ago
I was so excited to dive into this, but ended up with the same Takeaway as most other commenters. Aside: As a data scientist, I’ve been surprised how much I’ve needed to learn about the finer points of optimizing GPU utilization for training. It has all been from more experienced coworkers, and I would much appreciate any resources anybody could point me to (free or paid) so that I could round out my knowledge
- dragontamer 8y ago
  Learn enough about GPUs to be able to read the profiler. That should be your #1 goal: learning to use the profiler and performance counters. The profiler not only tells you how fast your code is, but also why your code is fast or slow... at least to the best ability of the hardware performance counters. Is it RAM-bottlenecked? Is it Compute bound? Are your Warps highly utilized? Etc. etc. If you don't know what the profiler is saying, then study some more. https://docs.nvidia.com/nsight-visual-studio-edition/Content/Analysis/Report/CudaExperiments/KernelLevel/PerformanceCounters.htm https://docs.nvidia.com/nsight-visual-studio-edition/Content...
blakehaswell 8y ago
For an accessible talk about the real-world implications of this, I enjoy watching Mike Acton's CppCon talk "Data-Oriented Design and C++": https://www.youtube.com/watch?v=rX0ItVEVjHc https://www.youtube.com/watch?v=rX0ItVEVjHc
- drainyard 8y ago
  I routinely re-watch this talk. It always gets me back in the right mindset.

214 points by jxub 8y ago

busfahrer 8y ago

Ulrich Drepper used to be the glibc maintainer, IIRC

v_lisivka 8y ago
memcpy != memmove , $#$@#@ https://sourceware.org/bugzilla/show_bug.cgi?id=12518 https://sourceware.org/bugzilla/show_bug.cgi?id=12518
- okl 8y ago
  https://sourceware.org/bugzilla/show_bug.cgi?id=3266 https://sourceware.org/bugzilla/show_bug.cgi?id=3266 https://sourceware.org/bugzilla/show_bug.cgi?id=12701 https://sourceware.org/bugzilla/show_bug.cgi?id=12701 https://sourceware.org/bugzilla/show_bug.cgi?id=386 https://sourceware.org/bugzilla/show_bug.cgi?id=386 https://sourceware.org/bugzilla/show_bug.cgi?id=10134 https://sourceware.org/bugzilla/show_bug.cgi?id=10134
  - cryptonector 8y ago
    Classics.
  - cls59 8y ago
    Also: https://sourceware.org/bugzilla/show_bug.cgi?id=10354 https://sourceware.org/bugzilla/show_bug.cgi?id=10354

CalChris 8y ago

I wish Ulrich Drepper (thank you, Mr. Drepper) would update this with a section on Row Hammer and also Spectre and Meltdown. Programmer's need to know about memory because of these exploits, more so with the latter two in order avoid creating exploitable gadgets. But then I also think that What Every Computer Scientist Should Know About Floating-Point Arithmetic should be updated to include UNUMs. I don't think that will happen either. Also, thank you Mr. Goldberg.

131012 8y ago
Thanks for bringing that up, I came to the comments section to see if some other details were outdated. Any other insights?
- CalChris 8y ago
  Well, I think hardware and software prefetch, speculation is a special form of prefetch, might be revisited. Software prefetch was at best a hopeful technology even in 2007 and is widely avoided today. https://yarchive.net/comp/linux/software_prefetching.html https://yarchive.net/comp/linux/software_prefetching.html Something I don't think either Drepper or Hennessy + Patterson's books get across is memory banks from a programmer's perspective. How cache organization affects a program is explained well but how banks affect said same isn't. Construction yes. Visibility, no.
Retric 8y ago
Row Hammer, Spectre, and Meltdown are hardware design flaws which don’t nessisarily stay meaningful over time. Further, they are completely irrelevant for many developers. The next Mars rover for example is not going to be running untrusted code.
nestorD 8y ago
What Every Computer Scientist Should Know About Floating-Point Arithmetic is great because it makes people aware of problems with the current system. Adding UNUMs would be suggesting the devil you don't know over the devil you know. However, and I can't remember if it is alreaddy in the book, a section on compensated summation / dot product would give strong time tested tools to attack the problem from within the current framework.

louthy 8y ago

What every programmer should know about memory shouldn't be 114 pages long.

twtw 8y ago
If you are objecting to "every," then sure, not every programmer needs to know anything about memory. You can program without knowing that there is such a thing. But it's a fun title, and an excellent resource.
- oarabbus_ 8y ago
  No, he's right. It's more than simply semantics. A better title would be "an extremely overwhelming of programmers don't need to know THIS much about memory"
  - netmonk 8y ago
    This is what people says when they have a dataset of 100 entries(<100mb) on their developpment laptop. How funny they look when production guys show them how their lame piece of code doesn't scale up with billion of entries in the production platform...But they dont care cause they are developpers, (or so called). Until the production guy rewrite their code and have them fired for incompetence (true story).
    - Chlorus 8y ago
      > How funny they look when production guys show them how their lame piece of code doesn't scale up with billion of entries in the production platform...But they dont care cause they are developpers, (or so called) Why did the production guy even need them in the first place? Why were they hired? > (true story). And that production guy's name was Albert Einstein.
  - abraae 8y ago
    Or perhaps some more positive phrasings: - If you know this much about memory, you'll know more than 99% of programmers. - What 99% of programmers don't know about memory.

saberience 8y ago

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, <insert language here> programmer would need to know about transistor configurations or voltages, pin configurations, etc. Let alone 114 pages of low level hardware diagrams and jargon! This document is for someone working on low level drivers for memory, or working on the hardware side. For any normal software engineer, this information is not helpful for doing your job.

gautamdivgi 8y ago
It depends. I find that most programmers not knowing some of this happens to be the bane of most performance and reliability issues. Take Java for example - pretty sure the Java dev needs to know how to optimize their jvm (memory settings, etc.). They would need to know direct memory, etc. and how it relates to the operating system. This nastier your traffic profile the more important the tuning described in that document are. Like all good "books" I don't remember all of it but keep coming back to refer to it. But again this is speaking from my experience as an SRE/SDE (or a perf/reliability engineer).
dragontamer 8y ago
I would argue that the overclocker needs to know more about these details than most programmers, yes. (Overclockers actually tweak these values to maximize the performance of their computer). But any high-performance programmer needs to understand the RAS / CAS / PRE cycle, if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. If you are accessing RAM randomly, you better be sure its within L3 cache (or nearer). I've done some experiments, and "streaming" data from beginning to end can be 2x to 3x faster than random access on modern DDR4 RAM. Understanding the RAS / CAS / PRE cycle helps me understand why streaming data to RAM is faster. And understanding that cells are simply capacitors helps me understand why the RAS / CAS / PRE cycle is necessary in DRAM.
- 8y ago

kevstev 8y ago

There are some salty comments here, but I think the context is important. This paper passed across my desk in early 2008 when I was doing HFT stuff. It might be a bit of a stretch to say that the reason people are taught about cache lines in most CS programs is because of this paper, but at the time this paper was written, this was really specialized knowledge and groundbreaking to most software developers. This would go on to be a popular topic on C++ blogs from Important People (Boost maintainers, STL devs, etc) at least for the next 5 years. Also, if you know Ulrich Drepper at all, either from some of his talks or his mailing list presence, this is just a very fitting title from him. Just pure deadpan, you think its funny, he probably does not, the fact that you think its amusing is just disappointing him like a professor looking out at freshman undergrads wondering how he got stuck teaching this class.

netmonk 8y ago
I really do agree with you. HAving done HFT for some years, this paper was crucial when it was running on linux system. Now FPGA took over the field. Different kind of techs.
dewhelmed 8y ago
He mentions his reason for the title: > The title of this paper is an homage to David Goldberg’s classic paper “What Every Computer Scientist Should Know About Floating-Point Arithmetic”

mjw1007 8y ago

Interesting that in 2007 he thought FB-DRAM was going to win. That seems to have been about the time it dropped dead.

Const-me 8y ago
Right, also about NUMA: > It is expected that, from late 2008 on, every SMP machine will use NUMA. Outside servers, still not happened. They are not that exotic anymore and are no longer exclusive to very expensive servers, e.g. Threadripper 2920X is a $650 CPU, but market penetration is still low.

orzig 8y ago

I was so excited to dive into this, but ended up with the same Takeaway as most other commenters. Aside: As a data scientist, I’ve been surprised how much I’ve needed to learn about the finer points of optimizing GPU utilization for training. It has all been from more experienced coworkers, and I would much appreciate any resources anybody could point me to (free or paid) so that I could round out my knowledge

dragontamer 8y ago
Learn enough about GPUs to be able to read the profiler. That should be your #1 goal: learning to use the profiler and performance counters. The profiler not only tells you how fast your code is, but also why your code is fast or slow... at least to the best ability of the hardware performance counters. Is it RAM-bottlenecked? Is it Compute bound? Are your Warps highly utilized? Etc. etc. If you don't know what the profiler is saying, then study some more. https://docs.nvidia.com/nsight-visual-studio-edition/Content/Analysis/Report/CudaExperiments/KernelLevel/PerformanceCounters.htm https://docs.nvidia.com/nsight-visual-studio-edition/Content...

blakehaswell 8y ago

For an accessible talk about the real-world implications of this, I enjoy watching Mike Acton's CppCon talk "Data-Oriented Design and C++": https://www.youtube.com/watch?v=rX0ItVEVjHc https://www.youtube.com/watch?v=rX0ItVEVjHc

drainyard 8y ago
I routinely re-watch this talk. It always gets me back in the right mindset.