6 ms·

Exactly. This is a significant portion of a graduate-level computer architecture course. You average Rails/Node/etc. programmer doesn't need to know half of wha

Exactly. This is a significant portion of a graduate-level computer architecture course. You average Rails/Node/etc. programmer doesn't need to know half of what's covered during the 3 weeks of an undergraduate architecture course that's devoted to the memory system. How many programmers in 2012 are ever even going to need to cache-optimize their code? 25%? 10%? The content is fascinating, but the title is pretty wrong.

sho_hn 14y ago
The title isn't need, it's should. Both are debatable of course, but I'd argue that should is a lot more defensible: Abstractions are leaky, and even if you can get by without knowing what's going on down there, you'll at the very least sharpen your mind and be more surefooted if you do. What you're saying is a bit like the but I'll never need math in life argument. You're going to say that math is a lot more fundamental to existence then computer memory is of course, and that's true, but if you're looking at just computers, the way the memory system works is a fundamental part of the whole not entirely unlike math is to understanding nature. If you really want to know what you're dealing with and make sure you're not limiting yourself, you want to be inquisitive and go there. Basically, I don't really get why the gut reaction to these sort of things (especially if they actually have meat; there's of course a lot of crummy pretenders) is so often to declare them pointless. Yes, you need to filter your information intake, but still, better err on the side of knowing too much.
- prophetjohn 14y ago
  No one is declaring this information pointless, but it's certainly a lot less relevant to Every Programmer than it was when this was written in 2007. I've actually got an okay understanding of this stuff, but the number of times that I've split for(int i = 0; i < 10; i++) { j += i; k += i; l += i; } into for(int i = 0; i < 10; i++) j += i; for(int i = 0; i < 10; i++) k += i; for(int i = 0; i < 10; i++) l += i; to reduce cache misses is very small because with today's hardware, it very often just doesn't matter. Obviously there are many programmers whose primary job is to do this, because they need every clock cycle they can squeeze out of the thing. They're just not representative of Every Programmer.
  - vilya 14y ago
    I'd argue it's more important now than when the white paper was first written. Where processor speed or memory size used to be the bottleneck, increasingly these days it's memory bandwidth instead, so managing that effectively is key to writing high performance programs.
    - cbsmith 14y ago
      I was going to say something almost precisely like this. The main thing I was going to add is that it is both memory bandwidth and latency that have become important bottlenecks, and often the latter is more important. You know how people were saying "disk is the new tape"? Guess what the new disk is? RAM. Much as with disk latencies, RAM latencies haven't really improved that much over the last decade or so, though memory bandwidth and CPU power have. Guess what that means? ;-) ..but you can only know this if you understand what's going on underneath the hood. So if you don't, no need to worry; you won't even know when the moment of your utter irrelevance arrives.
- garenp 14y ago
  My typical response to the "I'll never need to know X" argument is similar, which is that having as broad and diverse understanding of a system as possible gives you imagination. Without imagination, you will not be able to solve certain problems, not know where to even look when some arise, and at worse be totally unaware that better solutions exist. That said, the OP's context is that of an "interview style" question which raises mental red flags for me. It's a common problem for interviewers to base their assessment of a candidate based on whether or not the interviewee has the same subset of knowledge that they do. In that case, I'd be quick to point out that: * knowledge != intelligence * knowledge != capability * knowledge != skills * correlation != causation
romaniv 14y ago
They don't need to cache-optimize, but knowing how stuff really works can help prevent some pathological cases of premature optimization in higher-level languages. I've read most of this during my undegrad OS course, and found it absolutely fascinating.
- sho_hn 14y ago
  That's an awesome point I really wish I had made myself, so I'll just shamelessly rehash it to drive it home: Knowing how the system actually works down to a low level can help you avoid inadvertently writing code that will perform badly, even if you're dealing with code isn't otherwise required to perform above "reasonable" and thus doesn't call for low-level tuning. Let's take database queries as an example: Knowing something about how the database is implemented will help you avoid writing queries that are pathologically slow, even if the queries you need to write aren't critical path enough to benefit from going to great length to tune them to the implementation.
ajross 14y ago
I hear this so much, but it's just wrong. Here's a "interview style" question which is hopefully practical enough to show why: Let's say you have a giant in-memory balanced binary tree (for your data store, maybe) with mean depth of ... 27, say. And you're deployed on a machine with 4 DRAM channels populated with (for the sake of argument) 7-7-7-23 DDR3 memory. Give me a ballpark estimate for how many reads per second you can achieve from that tree. My experience is that virtually no one can answer this question well, and most programmer's intuition is wildly off (they think it's much faster than it is). And even the ones who get it mostly right usually just use "100 cycles" as their assumption of DRAM latency and get tripped up on the subtleties (like the interaction with the number of channels vs. precharge overlap; or the already-active row latency for sequential access which this question doesn't cover). Reading this paper will teach you how to answer that question from first principles. Surely that's worth a few hours of your time. At the very least it's worth more than arguing on the internet about how you don't need to know it. (edit: A bonus followup question -- hopefully just as practical -- would be: "So maybe binary trees aren't great. Given a node data size of N, what is the most appropriate fanout to use in a B tree implementation?". This pulls knowledge of caching architectures into the mix.)
- klochner 14y ago
  Surely that's worth a few hours of your time. At the very least it's worth more than arguing on the internet about how you don't need to know it. That's a great point. Regarding your interview question, the kind of optimization you're getting into isn't something 99% of 'programmers' will ever touch.
  - ajross 14y ago
    I suspect you can spin the demographics in lots of directions. But I'd hope that a far higher percentage of startup technical founders (the audience to which the comment was directed) would need to do non-trivial performance estimation in the course of building their products.
    - efuquen 14y ago
binarycrusader 14y ago
If you work in the game programming or systems programming fields, then definitely. Look at some of the presentations that developers made at GDCONF this past year or other years where they talk about the importance of using structures of arrays instead of arrays of structures. In brief (and greatly simplified) you may be able to fit an entire array into the processor cache and iterate over it quickly, since accessing each element of the array would only potentially bring in a pointer address. Whereas accessing an array of structures would cause the cache to be blown as each large structure attempted to be brought into cache and didn't quite fit. Here's one of the presentations: http://www.research.scea.com/research/pdfs/gdc2006_kokkevis_vangelis_physics.pdf

sho_hn 14y ago

The title isn't need, it's should. Both are debatable of course, but I'd argue that should is a lot more defensible: Abstractions are leaky, and even if you can get by without knowing what's going on down there, you'll at the very least sharpen your mind and be more surefooted if you do. What you're saying is a bit like the but I'll never need math in life argument. You're going to say that math is a lot more fundamental to existence then computer memory is of course, and that's true, but if you're looking at just computers, the way the memory system works is a fundamental part of the whole not entirely unlike math is to understanding nature. If you really want to know what you're dealing with and make sure you're not limiting yourself, you want to be inquisitive and go there. Basically, I don't really get why the gut reaction to these sort of things (especially if they actually have meat; there's of course a lot of crummy pretenders) is so often to declare them pointless. Yes, you need to filter your information intake, but still, better err on the side of knowing too much.

prophetjohn 14y ago
No one is declaring this information pointless, but it's certainly a lot less relevant to Every Programmer than it was when this was written in 2007. I've actually got an okay understanding of this stuff, but the number of times that I've split for(int i = 0; i < 10; i++) { j += i; k += i; l += i; } into for(int i = 0; i < 10; i++) j += i; for(int i = 0; i < 10; i++) k += i; for(int i = 0; i < 10; i++) l += i; to reduce cache misses is very small because with today's hardware, it very often just doesn't matter. Obviously there are many programmers whose primary job is to do this, because they need every clock cycle they can squeeze out of the thing. They're just not representative of Every Programmer.
- vilya 14y ago
  I'd argue it's more important now than when the white paper was first written. Where processor speed or memory size used to be the bottleneck, increasingly these days it's memory bandwidth instead, so managing that effectively is key to writing high performance programs.
  - cbsmith 14y ago
    I was going to say something almost precisely like this. The main thing I was going to add is that it is both memory bandwidth and latency that have become important bottlenecks, and often the latter is more important. You know how people were saying "disk is the new tape"? Guess what the new disk is? RAM. Much as with disk latencies, RAM latencies haven't really improved that much over the last decade or so, though memory bandwidth and CPU power have. Guess what that means? ;-) ..but you can only know this if you understand what's going on underneath the hood. So if you don't, no need to worry; you won't even know when the moment of your utter irrelevance arrives.
garenp 14y ago
My typical response to the "I'll never need to know X" argument is similar, which is that having as broad and diverse understanding of a system as possible gives you imagination. Without imagination, you will not be able to solve certain problems, not know where to even look when some arise, and at worse be totally unaware that better solutions exist. That said, the OP's context is that of an "interview style" question which raises mental red flags for me. It's a common problem for interviewers to base their assessment of a candidate based on whether or not the interviewee has the same subset of knowledge that they do. In that case, I'd be quick to point out that: * knowledge != intelligence * knowledge != capability * knowledge != skills * correlation != causation

romaniv 14y ago

They don't need to cache-optimize, but knowing how stuff really works can help prevent some pathological cases of premature optimization in higher-level languages. I've read most of this during my undegrad OS course, and found it absolutely fascinating.

sho_hn 14y ago
That's an awesome point I really wish I had made myself, so I'll just shamelessly rehash it to drive it home: Knowing how the system actually works down to a low level can help you avoid inadvertently writing code that will perform badly, even if you're dealing with code isn't otherwise required to perform above "reasonable" and thus doesn't call for low-level tuning. Let's take database queries as an example: Knowing something about how the database is implemented will help you avoid writing queries that are pathologically slow, even if the queries you need to write aren't critical path enough to benefit from going to great length to tune them to the implementation.

ajross 14y ago

I hear this so much, but it's just wrong. Here's a "interview style" question which is hopefully practical enough to show why: Let's say you have a giant in-memory balanced binary tree (for your data store, maybe) with mean depth of ... 27, say. And you're deployed on a machine with 4 DRAM channels populated with (for the sake of argument) 7-7-7-23 DDR3 memory. Give me a ballpark estimate for how many reads per second you can achieve from that tree. My experience is that virtually no one can answer this question well, and most programmer's intuition is wildly off (they think it's much faster than it is). And even the ones who get it mostly right usually just use "100 cycles" as their assumption of DRAM latency and get tripped up on the subtleties (like the interaction with the number of channels vs. precharge overlap; or the already-active row latency for sequential access which this question doesn't cover). Reading this paper will teach you how to answer that question from first principles. Surely that's worth a few hours of your time. At the very least it's worth more than arguing on the internet about how you don't need to know it. (edit: A bonus followup question -- hopefully just as practical -- would be: "So maybe binary trees aren't great. Given a node data size of N, what is the most appropriate fanout to use in a B tree implementation?". This pulls knowledge of caching architectures into the mix.)

klochner 14y ago
Surely that's worth a few hours of your time. At the very least it's worth more than arguing on the internet about how you don't need to know it. That's a great point. Regarding your interview question, the kind of optimization you're getting into isn't something 99% of 'programmers' will ever touch.
- ajross 14y ago
  I suspect you can spin the demographics in lots of directions. But I'd hope that a far higher percentage of startup technical founders (the audience to which the comment was directed) would need to do non-trivial performance estimation in the course of building their products.
  - efuquen 14y ago

binarycrusader 14y ago

If you work in the game programming or systems programming fields, then definitely. Look at some of the presentations that developers made at GDCONF this past year or other years where they talk about the importance of using structures of arrays instead of arrays of structures. In brief (and greatly simplified) you may be able to fit an entire array into the processor cache and iterate over it quickly, since accessing each element of the array would only potentially bring in a pointer address. Whereas accessing an array of structures would cause the cache to be blown as each large structure attempted to be brought into cache and didn't quite fit. Here's one of the presentations: http://www.research.scea.com/research/pdfs/gdc2006_kokkevis_vangelis_physics.pdf