9 ms·

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to

Is this title and content supposed to be ironic? I quickly perused the article and I think this link should be renamed "What 99.9% of programmers don't need to know about memory." I've managed to go from Associate to Principal without knowing 99% of what's covered in this document, and I'm struggling to understand why the average Java, C#, Python, Rust, <insert language here> programmer would need to know about transistor configurations or voltages, pin configurations, etc. Let alone 114 pages of low level hardware diagrams and jargon! This document is for someone working on low level drivers for memory, or working on the hardware side. For any normal software engineer, this information is not helpful for doing your job.

gautamdivgi 8y ago
It depends. I find that most programmers not knowing some of this happens to be the bane of most performance and reliability issues. Take Java for example - pretty sure the Java dev needs to know how to optimize their jvm (memory settings, etc.). They would need to know direct memory, etc. and how it relates to the operating system. This nastier your traffic profile the more important the tuning described in that document are. Like all good "books" I don't remember all of it but keep coming back to refer to it. But again this is speaking from my experience as an SRE/SDE (or a perf/reliability engineer).
dragontamer 8y ago
I would argue that the overclocker needs to know more about these details than most programmers, yes. (Overclockers actually tweak these values to maximize the performance of their computer). But any high-performance programmer needs to understand the RAS / CAS / PRE cycle, if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. If you are accessing RAM randomly, you better be sure its within L3 cache (or nearer). I've done some experiments, and "streaming" data from beginning to end can be 2x to 3x faster than random access on modern DDR4 RAM. Understanding the RAS / CAS / PRE cycle helps me understand why streaming data to RAM is faster. And understanding that cells are simply capacitors helps me understand why the RAS / CAS / PRE cycle is necessary in DRAM.
- 0815test 8y ago
  So, RAM is the new tape, cache is the new RAM?
- a1369209993 8y ago
  > if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. I think what they're saying is that 99.9% of programmers don't need to know why, they just need to know that streaming access is 2-3x faster than random even without any memory stalls.
- cortesoft 8y ago
  I am all for knowing things just to know things...but why does a programmer need to know WHY the streaming of data is efficient?
  - dragontamer 8y ago
    Because fundamentally, the DRAM is a major component of the computer, just as CPU cores are a major component of the computer. Now, I'd personally explain things in a far more simple manner than what was described in the PDF. Here are the facts that programmers need to know: 1. DRAM stores data in very tiny capacitors. These tiny capacitors have two properties: they run out of electricity in just 64ms. And second, they run out of electricity after a SINGLE read operation. 2. DRAM has a temporary location called "sense amplifiers" where data is stored during a refresh or a read. These sense amplifiers can hold data permanently. 3. This "temporary read" is called Row-open (or RAS). Reading from an already open row is called a Column-read (CAS). Sending the data back to DRAM proper is called Precharge (PRE). Remember, the sense amplifiers must be clear before they can read from a new row. (The old data in DRAM was destroyed when you read it with the RAS operation) 4. I guess there's a periodic refresh you should know about: instead of trying to fix all RAM every 64ms, you're supposed to do it in small chunks at a time. Every dozen microseconds, RAM will self-read / self-write to refresh another row. Don't be surprised if your memory-reads randomly stall out an extra few hundred nanoseconds because of this refresh. The end. Not so hard, now is it? -------- DRAM is faster when you stream, because you open a row once, fill out all the data in a row, and then send the row back to DRAM. In effect, you only have to do a bunch of "column" writes to sense amplifiers, as opposed to opening-and-closing a bunch of different rows. ---------- So yeah, programmers should know it because its really not that hard to learn :-) And if you start measuring your program at the nanosecond level, you'll actually see these effects and start to demand explanations. ------------- EDIT: Hmmm... the more I think of it, the less its something "programmers" need to know and something "SysAdmins / DevOps need to know". An advanced Sys Admin can use these profiler tools to figure out whether they need that 6x Memory Channel computer or the 8x Memory channel computer on the next purchase. Is your code memory-bound? Or is it CPU bound? Should you buy more cores? Should you buy more LRDIMMs for higher amounts of RAM? Or is your program latency-bound and actually benefits from the lower latency of RDIMMs or even UDIMMs ? The programmers kinda don't make those decisions.
laythea 8y ago
Can't agree more. If I went to a software job and started reading that stuff I'd get moved into hardware :) I suppose that's a credit to all the engineers who build all the middle layers that allow software engineers to float along at an abstract and more productive level.
3JPLW 8y ago
Indeed, there are a limited number of hard-hitting topics that are definitely must-haves: * Rough latency timings * Caching * Prefetching * Sequential vs. random access * N-dimensional layouts (row/column major and arbitrarily strided) * Design of cache-oblivious algorithms * SIMD-able access patterns * False sharing * Instruction cache & code size * Branch prediction and speculative execution I'd be curious to hear what else folks would put on (or remove from) this list.
- dragontamer 8y ago
  Good list. I'd add "Virtual Memory" to that list. In particular, the TLB cache, memory pages (4kB, 2MB "Large Pages", 2GB "Huge Pages). Although x86 specific, I'd also add x86-64 has 48-bit physical pointers: the top 16-bits are basically ignored by the current virtual memory system. I dunno if the whole Page Directory / Directory Tables / etc. etc. needs to be fully explained, but programmers should have an overall good idea what they are. There's lots of things to do with Virtual Memory. And anyone who actually reads profiler data needs to understand what the heck that TLB Cache Hits performance counter means. > * False sharing More specifically the MESI Protocol (although that's an abstraction), and cache lines should be taught. False Sharing comes as an understanding after you understand those other two concepts. A CPU Core holds a cache line in Exclusive state so that it can write to it. A 2nd CPU Core attempts to gain access, but it cannot until the 1st core releases control (by writing data back to memory and setting the line to the Invalid state). The knowledge of the 64-byte cache line is more general than just false sharing: it helps understand why alignment can be an issue (a load/store across a 64-byte cache line would require 2-reads by the memory controller), etc. etc.
  - jawnv6 8y ago
    how is MESI "an abstraction"? why is it the only model proffered instead of MESIF/MOESI? and "the top 16-bits are basically ignored" is a funny way to spell "general protection exception on linear memory reference in non-canonical space" but sure, guess we're just handwaving here
    - 8y ago
WallWextra 8y ago
Mazel tov. There are also people who went to "Principal" without being able to program at all.
gdy 8y ago
You should get acquainted with 'What Every Computer Scientist Should Know About Floating-Point Arithmetic' after which similar articles are named. I'd bet you would suggest to rename it too.
- saberience 8y ago
  Computer Science is a different discipline to Software Engineering. I would argue that sure, if you want to be a PhD, academic, researcher etc, in Computer Science, then reading said article might be useful. For engineers working at companies whose goal is to get product to market faster, coding ruby on rails, javascript, making web apps, websites, it's far more useful knowing your web frameworks well and being able to iterate fast, than knowing the intimate mathematical details of floating point arithmetic. Computer Science != engineering. It's like saying knowing physics is the same thing as knowing how to build a building.
  - nickpsecurity 8y ago
    The referenced work talks about floating point errors that both computer scientists and field engineers need to avoid. Many folks in both camps screw up on floating point code without such guidance. So, at least that part is potentially helpful. You just proved their point about how you'd reply, though.
    - saberience 8y ago
      Proved their point? Their point didn't make much sense. What actual engineers NEED to know about floating point numbers could be put on a single A4 page. It doesn't have to be 50 pages, where many of those pages are full of equations that 99% of engineers don't understand. My point is, because you seem to have trouble grasping this. Most real-world programmers, working at startups or bigger companies, making their WPF applications or Spring applications, or react/jquery/angular apps, or Swift IOS apps, etc, do not NEED to know 99% of this document. For most real-life cases all they need to know is "Use the decimal type if you're dealing with money" or something along those lines. Why do I know I'm correct? Because I work in an organization with 800-1000 engineers, and I know a fair few of them myself, and I bet maybe 1 person in the org has read this doc fully (and even this is a stretch). But the company stills makes billions in revenue every year like clockwork and the world keeps on spinning. Hence, this document is for theorists and academics, not for the average engineer making enterprise business applications. If it is for an engineer, it's for someone making extremely niche mathematical software or something equally arcane.
amelius 8y ago
I'm sure some programmers don't need to know this. For example an Erlang or Haskell programmer can't do anything about memory layout anyway, so this knowledge would be of no immediate practical use.
- yetihehe 8y ago
  > For example an Erlang or Haskell programmer can't do anything about memory layout anyway No, but we can make algorithms that will do things in other ways than standard. Knowing memory layout and behaviour enables programmer to invent better ways to do things. Of course those algorithms are very application specific, but at least it increases solution space. Erlangers mantra here is "profile this" so that you know which one is actually better.
- agentultra 8y ago
  Not exactly. There's plenty of ways of looking at GC and controlling allocation and lifetimes in Haskell is definitely possible. And when necessary the C FFI is useful to have to write small bits of code to take advantage of particular layouts and drive them from code. See [0]. [0] https://www.youtube.com/watch?v=b4bb8EP_pIE https://www.youtube.com/watch?v=b4bb8EP_pIE
ontologiae 8y ago
Every programmers should know something about memory, because software are becoming bloat [1] [2] because of a new generation of programmers who doesn't optimize their code. So, of course, you had a wonderful career, but it doesn't prove that you write efficient code [1] http://www.rntz.net/post/against-software-development.html http://www.rntz.net/post/against-software-development.html [2] http://tonsky.me/blog/disenchantment/ http://tonsky.me/blog/disenchantment/
- esmi 8y ago
  It really all comes down to what variable one is trying to optimize. I noticed neither article mentioned money as a possible variable to optimize. Here is another take to consider: https://www.joelonsoftware.com/2001/03/23/strategy-letter-iv-bloatware-and-the-8020-myth/ https://www.joelonsoftware.com/2001/03/23/strategy-letter-iv... "In 1993, given the cost of hard drives in those days, Microsoft Excel 5.0 took up about $36 worth of hard drive space. In 2000, given the cost of hard drives in 2000, Microsoft Excel 2000 takes up about $1.03 in hard drive space. In real terms, it’s almost like Excel is actually getting smaller!"
- cortesoft 8y ago
  I think the argument is that most programmers don't need to write efficient code.
- saberience 8y ago
  Is it better for my company if the software is delayed to market by 6 months because we were writing the most optimal code possible in low-level languages gaining us 5% more speed on our application? Most programmers are working for a business whose goal is to make money. Usually that involves adding more features or producing the product faster (as in development time). As far as optimal or efficient goes, all the company cares about is: "Is it fast enough so that people still buy/use the product?" Premature optimization is wasteful, both a waste of time and money. Unless you're doing it for fun, trying to eke out 1ms on a website load at the cost of excessive dev time and risk of bugs, is just silly. Don't forget, trying to make super optimal code often introduces bugs as you now have code that is more complex and/or uses more low level constructs. I've seen many deadlocks and race conditions due to engineers trying to optimize code unnecessarily.
wuxb 8y ago
I write user-level C/C++ programs and think that this book is 100% relevant. Yes the details about the transistors can be skipped for first-time readers. But eventually these explain what programmers should expect from the machine, and why.
twtw 8y ago
No, it isn't supposed to be ironic. The title says "should," after all. Who says every programmer should knows these things about memory? Obviously not you. Ulrich Drepper does - I bet if you asked him, he would say everyone should know these things, but would concede that almost no programmers do and most programmers don't need to.
deleted 8y ago
[deleted]
lxe 8y ago
I work in frontend after switching gears from embedded systems a few years back. Knowing some level of detail about how computers work is invaluable at all layers of the stack: I can make informed trade-offs between practical performance of code running on an actual computer and the cost of high-level language concerns and features.
cozzyd 8y ago
Maybe it's because I'm a physicist and not a software engineer, but it's nice to know how things work all the way down even if in the end, a phenomenological model is all you need to do your work.
mrfusion 8y ago
It’s fair game for a python/java/c# interview though! /s

gautamdivgi 8y ago

It depends. I find that most programmers not knowing some of this happens to be the bane of most performance and reliability issues. Take Java for example - pretty sure the Java dev needs to know how to optimize their jvm (memory settings, etc.). They would need to know direct memory, etc. and how it relates to the operating system. This nastier your traffic profile the more important the tuning described in that document are. Like all good "books" I don't remember all of it but keep coming back to refer to it. But again this is speaking from my experience as an SRE/SDE (or a perf/reliability engineer).

dragontamer 8y ago

I would argue that the overclocker needs to know more about these details than most programmers, yes. (Overclockers actually tweak these values to maximize the performance of their computer). But any high-performance programmer needs to understand the RAS / CAS / PRE cycle, if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. If you are accessing RAM randomly, you better be sure its within L3 cache (or nearer). I've done some experiments, and "streaming" data from beginning to end can be 2x to 3x faster than random access on modern DDR4 RAM. Understanding the RAS / CAS / PRE cycle helps me understand why streaming data to RAM is faster. And understanding that cells are simply capacitors helps me understand why the RAS / CAS / PRE cycle is necessary in DRAM.

0815test 8y ago
So, RAM is the new tape, cache is the new RAM?
a1369209993 8y ago
> if only to understand WHY the "streaming" of data is efficient, while random-access is very inefficient. I think what they're saying is that 99.9% of programmers don't need to know why, they just need to know that streaming access is 2-3x faster than random even without any memory stalls.
cortesoft 8y ago
I am all for knowing things just to know things...but why does a programmer need to know WHY the streaming of data is efficient?
- dragontamer 8y ago
  Because fundamentally, the DRAM is a major component of the computer, just as CPU cores are a major component of the computer. Now, I'd personally explain things in a far more simple manner than what was described in the PDF. Here are the facts that programmers need to know: 1. DRAM stores data in very tiny capacitors. These tiny capacitors have two properties: they run out of electricity in just 64ms. And second, they run out of electricity after a SINGLE read operation. 2. DRAM has a temporary location called "sense amplifiers" where data is stored during a refresh or a read. These sense amplifiers can hold data permanently. 3. This "temporary read" is called Row-open (or RAS). Reading from an already open row is called a Column-read (CAS). Sending the data back to DRAM proper is called Precharge (PRE). Remember, the sense amplifiers must be clear before they can read from a new row. (The old data in DRAM was destroyed when you read it with the RAS operation) 4. I guess there's a periodic refresh you should know about: instead of trying to fix all RAM every 64ms, you're supposed to do it in small chunks at a time. Every dozen microseconds, RAM will self-read / self-write to refresh another row. Don't be surprised if your memory-reads randomly stall out an extra few hundred nanoseconds because of this refresh. The end. Not so hard, now is it? -------- DRAM is faster when you stream, because you open a row once, fill out all the data in a row, and then send the row back to DRAM. In effect, you only have to do a bunch of "column" writes to sense amplifiers, as opposed to opening-and-closing a bunch of different rows. ---------- So yeah, programmers should know it because its really not that hard to learn :-) And if you start measuring your program at the nanosecond level, you'll actually see these effects and start to demand explanations. ------------- EDIT: Hmmm... the more I think of it, the less its something "programmers" need to know and something "SysAdmins / DevOps need to know". An advanced Sys Admin can use these profiler tools to figure out whether they need that 6x Memory Channel computer or the 8x Memory channel computer on the next purchase. Is your code memory-bound? Or is it CPU bound? Should you buy more cores? Should you buy more LRDIMMs for higher amounts of RAM? Or is your program latency-bound and actually benefits from the lower latency of RDIMMs or even UDIMMs ? The programmers kinda don't make those decisions.

laythea 8y ago

Can't agree more. If I went to a software job and started reading that stuff I'd get moved into hardware :) I suppose that's a credit to all the engineers who build all the middle layers that allow software engineers to float along at an abstract and more productive level.

3JPLW 8y ago

Indeed, there are a limited number of hard-hitting topics that are definitely must-haves: * Rough latency timings * Caching * Prefetching * Sequential vs. random access * N-dimensional layouts (row/column major and arbitrarily strided) * Design of cache-oblivious algorithms * SIMD-able access patterns * False sharing * Instruction cache & code size * Branch prediction and speculative execution I'd be curious to hear what else folks would put on (or remove from) this list.

dragontamer 8y ago
Good list. I'd add "Virtual Memory" to that list. In particular, the TLB cache, memory pages (4kB, 2MB "Large Pages", 2GB "Huge Pages). Although x86 specific, I'd also add x86-64 has 48-bit physical pointers: the top 16-bits are basically ignored by the current virtual memory system. I dunno if the whole Page Directory / Directory Tables / etc. etc. needs to be fully explained, but programmers should have an overall good idea what they are. There's lots of things to do with Virtual Memory. And anyone who actually reads profiler data needs to understand what the heck that TLB Cache Hits performance counter means. > * False sharing More specifically the MESI Protocol (although that's an abstraction), and cache lines should be taught. False Sharing comes as an understanding after you understand those other two concepts. A CPU Core holds a cache line in Exclusive state so that it can write to it. A 2nd CPU Core attempts to gain access, but it cannot until the 1st core releases control (by writing data back to memory and setting the line to the Invalid state). The knowledge of the 64-byte cache line is more general than just false sharing: it helps understand why alignment can be an issue (a load/store across a 64-byte cache line would require 2-reads by the memory controller), etc. etc.
- jawnv6 8y ago
  how is MESI "an abstraction"? why is it the only model proffered instead of MESIF/MOESI? and "the top 16-bits are basically ignored" is a funny way to spell "general protection exception on linear memory reference in non-canonical space" but sure, guess we're just handwaving here
  - 8y ago

WallWextra 8y ago

Mazel tov. There are also people who went to "Principal" without being able to program at all.

gdy 8y ago

You should get acquainted with 'What Every Computer Scientist Should Know About Floating-Point Arithmetic' after which similar articles are named. I'd bet you would suggest to rename it too.

saberience 8y ago
Computer Science is a different discipline to Software Engineering. I would argue that sure, if you want to be a PhD, academic, researcher etc, in Computer Science, then reading said article might be useful. For engineers working at companies whose goal is to get product to market faster, coding ruby on rails, javascript, making web apps, websites, it's far more useful knowing your web frameworks well and being able to iterate fast, than knowing the intimate mathematical details of floating point arithmetic. Computer Science != engineering. It's like saying knowing physics is the same thing as knowing how to build a building.
- nickpsecurity 8y ago
  The referenced work talks about floating point errors that both computer scientists and field engineers need to avoid. Many folks in both camps screw up on floating point code without such guidance. So, at least that part is potentially helpful. You just proved their point about how you'd reply, though.
  - saberience 8y ago
    Proved their point? Their point didn't make much sense. What actual engineers NEED to know about floating point numbers could be put on a single A4 page. It doesn't have to be 50 pages, where many of those pages are full of equations that 99% of engineers don't understand. My point is, because you seem to have trouble grasping this. Most real-world programmers, working at startups or bigger companies, making their WPF applications or Spring applications, or react/jquery/angular apps, or Swift IOS apps, etc, do not NEED to know 99% of this document. For most real-life cases all they need to know is "Use the decimal type if you're dealing with money" or something along those lines. Why do I know I'm correct? Because I work in an organization with 800-1000 engineers, and I know a fair few of them myself, and I bet maybe 1 person in the org has read this doc fully (and even this is a stretch). But the company stills makes billions in revenue every year like clockwork and the world keeps on spinning. Hence, this document is for theorists and academics, not for the average engineer making enterprise business applications. If it is for an engineer, it's for someone making extremely niche mathematical software or something equally arcane.

amelius 8y ago

I'm sure some programmers don't need to know this. For example an Erlang or Haskell programmer can't do anything about memory layout anyway, so this knowledge would be of no immediate practical use.

yetihehe 8y ago
> For example an Erlang or Haskell programmer can't do anything about memory layout anyway No, but we can make algorithms that will do things in other ways than standard. Knowing memory layout and behaviour enables programmer to invent better ways to do things. Of course those algorithms are very application specific, but at least it increases solution space. Erlangers mantra here is "profile this" so that you know which one is actually better.
agentultra 8y ago
Not exactly. There's plenty of ways of looking at GC and controlling allocation and lifetimes in Haskell is definitely possible. And when necessary the C FFI is useful to have to write small bits of code to take advantage of particular layouts and drive them from code. See [0]. [0] https://www.youtube.com/watch?v=b4bb8EP_pIE https://www.youtube.com/watch?v=b4bb8EP_pIE

ontologiae 8y ago

Every programmers should know something about memory, because software are becoming bloat [1] [2] because of a new generation of programmers who doesn't optimize their code. So, of course, you had a wonderful career, but it doesn't prove that you write efficient code [1] http://www.rntz.net/post/against-software-development.html http://www.rntz.net/post/against-software-development.html [2] http://tonsky.me/blog/disenchantment/ http://tonsky.me/blog/disenchantment/

esmi 8y ago
It really all comes down to what variable one is trying to optimize. I noticed neither article mentioned money as a possible variable to optimize. Here is another take to consider: https://www.joelonsoftware.com/2001/03/23/strategy-letter-iv-bloatware-and-the-8020-myth/ https://www.joelonsoftware.com/2001/03/23/strategy-letter-iv... "In 1993, given the cost of hard drives in those days, Microsoft Excel 5.0 took up about $36 worth of hard drive space. In 2000, given the cost of hard drives in 2000, Microsoft Excel 2000 takes up about $1.03 in hard drive space. In real terms, it’s almost like Excel is actually getting smaller!"
cortesoft 8y ago
I think the argument is that most programmers don't need to write efficient code.
saberience 8y ago
Is it better for my company if the software is delayed to market by 6 months because we were writing the most optimal code possible in low-level languages gaining us 5% more speed on our application? Most programmers are working for a business whose goal is to make money. Usually that involves adding more features or producing the product faster (as in development time). As far as optimal or efficient goes, all the company cares about is: "Is it fast enough so that people still buy/use the product?" Premature optimization is wasteful, both a waste of time and money. Unless you're doing it for fun, trying to eke out 1ms on a website load at the cost of excessive dev time and risk of bugs, is just silly. Don't forget, trying to make super optimal code often introduces bugs as you now have code that is more complex and/or uses more low level constructs. I've seen many deadlocks and race conditions due to engineers trying to optimize code unnecessarily.

wuxb 8y ago

I write user-level C/C++ programs and think that this book is 100% relevant. Yes the details about the transistors can be skipped for first-time readers. But eventually these explain what programmers should expect from the machine, and why.

twtw 8y ago

No, it isn't supposed to be ironic. The title says "should," after all. Who says every programmer should knows these things about memory? Obviously not you. Ulrich Drepper does - I bet if you asked him, he would say everyone should know these things, but would concede that almost no programmers do and most programmers don't need to.

deleted 8y ago

[deleted]

lxe 8y ago

I work in frontend after switching gears from embedded systems a few years back. Knowing some level of detail about how computers work is invaluable at all layers of the stack: I can make informed trade-offs between practical performance of code running on an actual computer and the cost of high-level language concerns and features.

cozzyd 8y ago

Maybe it's because I'm a physicist and not a software engineer, but it's nice to know how things work all the way down even if in the end, a phenomenological model is all you need to do your work.

mrfusion 8y ago

It’s fair game for a python/java/c# interview though! /s