4 ms·

I agree. His recent ACM article is an example: provocative and even unpleasant, but making a point which, if correct, could hardly be more important; and I thin

by gruseom 16y ago

I agree. His recent ACM article is an example: provocative and even unpleasant, but making a point which, if correct, could hardly be more important; and I think it is probably correct. What do you think it is that makes Varnish so fast? I've read the article (http://varnish-cache.org/wiki/ArchitectNotes http://varnish-cache.org/wiki/ArchitectNotes) but I'm curious to hear about it from people's experience. Sometimes the stated reasons for something being great are not the same as the real reasons.

jacquesm 16y ago
Using system calls the way they are intended to and building a server the way UNIX is meant to be used. It's a textbook example of how to do a thing like this.
- gruseom 16y ago
  Fair enough, but I want to know the key principles about how these are intended to be used. Or are you saying I should just read Stevens? The major point I took from the articles is that you should let the kernel manage what's in RAM -- it's going to do it anyway, so don't fight it -- and instead of building complicated paging and caching mechanisms, focus on laying out your data to take advantage of how the kernel already does it. What I want to know is, what are the other such major points? p.s. Good luck tomorrow :)
akkartik 16y ago
Ah, here's the article: http://queue.acm.org/detail.cfm?id=1814327 http://queue.acm.org/detail.cfm?id=1814327 Especially interesting to me because I spent 5 years on memory issues in grad school, steeped in the idea that number of comparisons or number of instructions per unit input is often a red herring. Unfortunately the alternative is a much more complex model to juggle in your head.
- gruseom 16y ago
  (Sorry for not including the link to that one.) Can you expand on this? What do you think of the article's argument and how does it relate to what you studied? Perhaps this should be in its own thread, but I'm really curious about this because, right or wrong, what he's advocating is not a common design style.
  - akkartik 16y ago
    Oh, he's right. More often than not, it's uncommon just because we're doing it wrong. I spent a good couple of years thinking up layout optimizations like the one in Figure 6. I think that's fairly common design style, when people choose to think about this stuff. The challenge is coming up with consistently better layouts for a specific application. One benefit of java, for example, is that you can move things around without worrying about pointer arithmetic like in C. And his rant about Figure 7 resonates. Again, it's brain-dead obvious that you should take the memory hierarchy of your system into account because you waste millions of cycles waiting on a disk read, but it's really hard to do. Our education system doesn't help, and neither do mainstream tools. I think of it as concentric revolutions. The pentium's algorithm for processing instructions out-of-order first showed up in a mainframe 30 years before (http://en.wikipedia.org/wiki/Tomasulo_algorithm http://en.wikipedia.org/wiki/Tomasulo_algorithm). The internet gradually rediscovers what desktop devs gradually rediscovered what the minicomputer devs originally dreamt up. Most of the time it's not common yet simply because people haven't gotten around to it. It's basically YAGNI on the scale of industry segments. A successful startup will gradually make its infrastructure more stable and higher-performance after it's ensured it'll be around. And a successful new technology will first validate its reason for being before gradually trying to use what previous technologies have learned. Sorry this is poorly written. I'm not sure what parts of this are non-obvious, so feel free to ask more questions.

by gruseom 16y ago

jacquesm 16y ago

Using system calls the way they are intended to and building a server the way UNIX is meant to be used. It's a textbook example of how to do a thing like this.

gruseom 16y ago
Fair enough, but I want to know the key principles about how these are intended to be used. Or are you saying I should just read Stevens? The major point I took from the articles is that you should let the kernel manage what's in RAM -- it's going to do it anyway, so don't fight it -- and instead of building complicated paging and caching mechanisms, focus on laying out your data to take advantage of how the kernel already does it. What I want to know is, what are the other such major points? p.s. Good luck tomorrow :)

akkartik 16y ago

Ah, here's the article: http://queue.acm.org/detail.cfm?id=1814327 http://queue.acm.org/detail.cfm?id=1814327 Especially interesting to me because I spent 5 years on memory issues in grad school, steeped in the idea that number of comparisons or number of instructions per unit input is often a red herring. Unfortunately the alternative is a much more complex model to juggle in your head.

gruseom 16y ago
(Sorry for not including the link to that one.) Can you expand on this? What do you think of the article's argument and how does it relate to what you studied? Perhaps this should be in its own thread, but I'm really curious about this because, right or wrong, what he's advocating is not a common design style.
- akkartik 16y ago
  Oh, he's right. More often than not, it's uncommon just because we're doing it wrong. I spent a good couple of years thinking up layout optimizations like the one in Figure 6. I think that's fairly common design style, when people choose to think about this stuff. The challenge is coming up with consistently better layouts for a specific application. One benefit of java, for example, is that you can move things around without worrying about pointer arithmetic like in C. And his rant about Figure 7 resonates. Again, it's brain-dead obvious that you should take the memory hierarchy of your system into account because you waste millions of cycles waiting on a disk read, but it's really hard to do. Our education system doesn't help, and neither do mainstream tools. I think of it as concentric revolutions. The pentium's algorithm for processing instructions out-of-order first showed up in a mainframe 30 years before (http://en.wikipedia.org/wiki/Tomasulo_algorithm http://en.wikipedia.org/wiki/Tomasulo_algorithm). The internet gradually rediscovers what desktop devs gradually rediscovered what the minicomputer devs originally dreamt up. Most of the time it's not common yet simply because people haven't gotten around to it. It's basically YAGNI on the scale of industry segments. A successful startup will gradually make its infrastructure more stable and higher-performance after it's ensured it'll be around. And a successful new technology will first validate its reason for being before gradually trying to use what previous technologies have learned. Sorry this is poorly written. I'm not sure what parts of this are non-obvious, so feel free to ask more questions.