6 ms·
The software development final exam: Computer Architecture and Operating Systems
- cperciva 14y agoSome FAQs: I'll be posting the third and fourth parts at 9 AM UTC on Wednesday and Thursday (and submitting the links to HN). I got just over 100 responses to part 1, and I'm almost halfway through grading them. I'll be sending out a batch of emails (form emails, I'm afraid, to save time) once I've caught up with the part 1 grading. I will be posting my answers, my grading scheme, and an analysis of the results next week. I will not disclose any individual's performance (except to them) without their permission.
- jsnell 14y agoThanks for taking the time to do this, I think it's an interesting experiment (even if one can't read too much into the results due to the sampling bias).
- peteretep 14y agoI am enjoying these, but as a poster on your previous thread predicted, you're moving further away from anything resembling what 99% of today's programmers ever see... Need a car analogy? Fine: you pitched this as "what every driver should know about driving", and you're talking about the intricacies of combustion engines. That is to say, it's interesting, may pin together some pieces of other knowledge, but of no real practical use to anyone who doesn't also fit in to the "and I'm a mechanic" demographic.
- Ensorceled 14y agoI like that analogy but it's worse than you say: 2 of the questions are about drift racing (mutexs) 2 of the questions are about the design of the fuel injection control chip 1 question is a subtle question about the fuel injection control chip
- antidoh 14y ago"What every pilot should know" might be a better analogy. Pilots are more directly responsible for keeping their vehicle within safe conditions, and to do that they need to know more about their vehicles than car drivers. Programmers affect and determine the operation of the computers their code runs on much more than a driver driving inside the lines.
- randomdata 14y agoI really like the first analogy, actually. A driver who knows all the details of how each engine works will be able to choose the car that provides the best fuel economy and performance, but in the end, the driver's status will really only be judged on the style and features of the car they choose. i.e. the developer who chooses the slower zeroarray algorithm that still completes in reasonable time, but provides a better user interface and better feature set is still the better software developer (but perhaps the poorer computer scientist).
- kamaal 14y agoI am pretty sure pilots don't master fluid mechanics in detail.
- jrajav 14y agoI'm excited and motivated to learn by these posts, and I can tell others feel the same way, but I share this sentiment. To me, it even seems a little too fair to compare it to drivers and mechanics. It's more like a pitch to drivers about how to be a better car mechanic, when the target demographic actually includes drivers, stock car drivers, car mechanics, automotive designers, motorcyclists, motorcycle mechanics, pilots, submarine engineers, rocket scientists...
- deleted 14y ago[deleted]
- thaumaturgy 14y agoColin, I've been programming for about 25 years now, in more languages and architectures than I can ever readily remember, and you're making me feel like a noob. Thank you for that. :-)
- nandemo 14y agoThat's the spirit! I don't understand why people get so defensive. I have a BS and a master's degree in CS and I didn't know about MESI cache coherence. And I've never run into it in 7+ years of professional experience. But that doesn't mean it's a useless question. I don't feel compelled to learn all about it right now, but at least I'll get acquainted with the Wikipedia-level facts. One day I might study it further.
- mgkimsal 14y ago"But that doesn't mean it's a useless question." Is it a useful question, just because it was asked?
- michael_h 14y agoI have liked these questions, even if I think a few of them are a little narrow. It's certainly sparked a bit of motivation for me. However, the intro blog post is worded very poorly, which is what I'm assuming is causing a lot of the problems. It reads less as a call to personal improvement and more as a shaming/institutional snobbery - whether that was intended or not.
- wrath 14y agoI don't know if I should feel "okay" about not knowing the answers or if I should feel "dumb". Is this something as a system programmer today really need to know?
- jaimzob 14y agoNo. They can be interesting things to read about, and knowing a little about the themes each question comes from (i.e. how does a processor cache work) will make you a better programmer, but don't worry about cramming your head with trivia such as "what does MESI stand for?"
- LinaLauneBaer 14y agoCan someone explain why zeroarray2 is so much slower than zeroarray1 (regarding question 2)? I have profiled the two functions and it turns out for inner for-loop and the line which zeros out the entry in the array are much slower... (optimisations turned off). The only real difference is the name of the variables. Right? Has this something to do with memory alignment?
- andyjohnson0 14y agoIn zeroarray1 the address to be zeroed goes linearly from 0 to (1024^2)-1. This generates less cache misses because the address is much more likely to already be in the cache, which is in turn because the address only differes by one from the previous value of the address. zeroarray2 jumps around (0, 1024, 2048, 4096, ..., 1, 1025, 2049, 4097, ...) in a way that intentionally makes life difficult for the cpu's cache manager, and generating lots of cache misses.
- dagw 14y agoAlso worth noting that this is language dependent. What you say is true in C (which is admittedly what the question was about), but not true in for example Fortran. edit: Ignore. I'm wrong, see below
- andyjohnson0 14y agoI'm not doubting what you say, but could you elaborate on why Fortran is different? I could understand a difference for two-dimensional (or more) arrays, where different languages lay-out the array contents in memory differently. Does Fortran lay-out the contents of one dimensional arrays in an unusual way?
- dagw 14y agoSorry, I screwed up. I skimmed the question and mis-remebered it when making my post. I thought the question looked like: for(i=0... for(j=0... A[i][j]=0; vs for(i=0... for(j=0... A[j][i]=0; In which case C vs Fortran makes a difference to the way multidimensional arrays are stored in memory.
- philbarr 14y agoFor those who are reading this and are new to software development, please don't let these questions put you off or take them as some sort of test of your skills. The questions pertain only to the poster's particular CS degree and to his particular experience, which for some reason he thinks is the only CS degree and experience that is valid. Also note that you will come across people like this quite a lot in the software industry and you have to be careful not to let them get under your skin. They can be very difficult to work with. If you are a team lead and identify someone like this on your team then it can be very difficult to place them and you often have to gently coerce them into their own "special" area where they feel their talents are best used.
- arethuza 14y ago"They can be very difficult to work with." Don't you think that is a bit harsh? From what I have seen of Colin's work and contributions here on HN he seems like a pretty knowledgeable and decent chap to me. I rather enjoyed myself contemplating these questions - although I haven't had the nerve to submit completed answers!
- philbarr 14y agoThe red flag was this line in the blog that led to the questions: "If you can't answer the majority of the questions on these four papers, and you're working or intend to work as a software developer, you should ask yourself why". This assumes an air of authority that the poster clearly cannot posses; that is, he does not have the right to judge whether someone is a valid software developer or not. Someone claiming this level of superiority seems to fit the personality type I described. Of course - I could well be wrong but that's what it sounded like to me.
- ygra 14y agoNote that this refers to all four parts, not just the second. In this case one might lack detailed low-level knowledge of threading and a particular library used to do so on POSIX systems which may have no relation to performance in the other parts.
- Ensorceled 14y agoI graduated with good marks from a great school. And so far I'd have been, at best, 7 out of 10 on graduation day. After 20 successful years in the industry, I might get 4 or 5 depending on time constraints since I'd have to figure out some stuff from first principles. Some of these questions aren't even necessarily relevant in getting a computer science degree let alone software development.
- bentoner 14y agoWhat's the best textbook for this material (caching, concurrency etc.)?
- mdkess 14y agoI found this http://www.akkadia.org/drepper/cpumemory.pdf http://www.akkadia.org/drepper/cpumemory.pdf to be a very good starting point for getting into serious low level development. It's a difficult read, so go through it slowly and with a pad of paper for notes, but very information dense and useful.
- ww520 14y agoThere aren't much relationship to OS concepts. The lock question can be arguably related but it's more for concurrency. They are pretty light weight questions. OS questions would involve process/thread/fiber, scheduling, priority, multitasking, memory management, virtual memory, IPC, deadlock, distributed deadlock, interrupt/signal, input system, output system, GUI system, file system, disk system, IO caching, driver model, kernel space and user space, privilege and non-privilege operation, security model, user/process permission and privilege, file permission, authentication, authorization, etc.
- cperciva 14y agoI was aiming for questions relevant to general-purpose developers, not questions relevant to operating system developers -- and a small number of questions at that. I covered locking, race conditions, virtual memory, and processes; I think those are much more relevant to most developers than knowing the internals of how filesystems work or how interrupts are routed.
- mmcnickle 14y agoI think you have fallen into the trap by following your goal of wanting "questions which are easy to mark as right or wrong". You end up writing questions that are extremely narrow in scope and don't provide any opportunity for any interesting discussion. Anyone could memorise the states in MESI cache coherence, but does that really test their understanding of the material? That so many of the answers are one google away shows how shallow these questions are. All you're doing here is testing knowledge retention. I'll admit that I couldn't answer many of these questions off hand; I hope the criticism doesn't come across as bitter.
- randomdata 14y agoI did get the feeling the questions were chosen, perhaps unintentionally, to create a natural bias towards those who have a formal CS education. Contrived example: Someone who came up with a quicksort algorithm independently, and is completely aware of its operating conditions, but chose to call it mysort instead, would be unable to answer the question in the last test despite having all the theoretical knowledge needed. There is, of course, still something to be said about being able to use terminology that is shared with others, but that changes the nature of the test in ways I'm not sure the author intended.
- JoeAltmaier 14y agoA good point, considering a large portion of practicing programmers are self-taught.
- iamtempleton 14y agoIs this true in 2012? I would love to see a study or something. Not doubting you, just curious.
- geebee 14y agoSo far, I've been enjoying reading these questions. However, I think that the format prevents using them to answer what might be a more interesting question - how long would it take someone with a different (non-CS) academic background to answer the question (with understanding, obviously, not through cut and paste). In other words, rather than asking whether someone knows the answer, ask how well prepared they are to research and understand the question.
- cperciva 14y agoYou're misunderstanding the purpose of these questions. It isn't to determine whether you are able to figure out the answers to these questions -- as many people have pointed out, some of these are simple "type into google" questions. The point is that knowing the answers or being able to figure them out on your own is, I believe, strongly correlated with having a good understanding of the entire area the question is drawn from.
- geebee 14y agoI didn't write the questions, so the purpose of them really isn't up to me. However, I don't really agree that they are simple "type into google" questions. For example, suppose you let someone google around for a half hour to try to answer the question about how to determine if a graph is bipartite. Now, you spend about a half an hour doing an oral exam to see how well they understand what they just regurgitated. I suspect that you would see a wide range of performance, but that people with certain academic backgrounds might do much better than others. That's the "more interesting question" that I had in mind. Actually, suppose someone had never take graph theory came up with a novel but ultimately flawed attempt at an algorithm. That might be a stronger sign of talent in this area than someone who had taken the class and was able to reproduce an algorithm (even if that student showed a genuine understanding of it).
- cperciva 14y agoI did say that some of the questions are "type into google" questions. The bipartite-graph question isn't -- but most people will have never seen that particular question in class, either. That one is a "can you take material you should know and come up with something new" question... just like the TLB question in part 2 and another question in part 3.
- psykotic 14y agoI sent in answers to help with your survey, but I'm highly skeptical of the statistical methodology. You are selecting for people who at least think they know the answers. My prediction is that you are going to see a rosy picture of the state of CS knowledge among software developers, even though the reality is probably closer to the exact opposite.
- cperciva 14y agoI'm asking everybody to submit answers, and there are plenty of people submitting "I don't know"s.
- tmoertel 14y agoEven though you asked everybody to submit answers, you should expect people who expect themselves to do poorly to opt out. So your sample, I suspect, is strongly biased toward the answers from people who expected not to do poorly. And that bias is indeed likely to paint a rosy picture of the reality.
- cperciva 14y agoQuite true -- I'll definitely have to be careful in how I interpret the data. I'm hoping that the issue I'm most interested in -- comparing how people do on different questions, and comparing how different people do, will be less strongly biased by the poor sampling method.