6 ms·

A conclusion reached by measurement is not premature. This looks like an attempt to write a better server than the 80/20 rule allows. If he's wrong and only one

A conclusion reached by measurement is not premature. This looks like an attempt to write a better server than the 80/20 rule allows. If he's wrong and only one polling method is useful in production, the live servers will pick the good one and nobody will suffer because he jumped to conclusions. Since he's written Mongrel, I trust that he has a reason to worry about polling that may not have appeared in the post

jacquesm 16y ago
> A conclusion reached by measurement is not premature. That's just plain wrong. Premature optimisation does not refer to having to measure before you optimise, it refers to optimising things that in practice may have little or no effect on the actual performance of the program. By doing these tests in isolation instead of while running on a profiling kernel under production load it is very well possible that the bottleneck will not be the polling code at all but something entirely different. I'd say that this is a textbook example of what premature optimisation is all about. Assuming you have a finite budget of time to spend on a project any optimisations done that take time out of that budget that could have been spent more effective elsewhere is premature. Now there is a chance that this would have been the bottleneck in the completed system, but before you've got a complete system you can't really tell. My guess based on real world experience with lots of system level code that used both, including web servers, video servers, streaming audio servers and so on is that the overhead of poll/epoll will be relatively minor compared to other parts of the code and the massive amount of IO that typically follows a poll or an epoll call. If you have 10K sockets open then typically poll/epoll will return a large number of 'active' descriptors, you'll then be doing IO on all of those for that single call to poll/epoll. Each of those IO calls is probably going to be as much or more work to process than the poll call was.
- mfukar 16y ago
  If you have 10K sockets open then typically poll/epoll will return a large number of 'active' descriptors, ...<snip> That's half true - it doesn't hold for low ATR traffic (lots of hanging connections, clients that GET something, spend time elsewhere while in the meantime the browser keeps the connection alive). In short, there's nothing typical about it because, while those two kinds of loads have been studied extensively in both bibliography and practice, their combination and the practical consequences are not well understood, afaik. Links to relevant studies are more than welcome, of course.
  - jacquesm 16y ago
    Large in an absolute sense, say 3K for a pool of 10K sockets. That's a sizeable number of active connections to deal with after a single system call. Typically for each of those fds you'll then do a read or a write. So the epoll/poll overhead is fairly small, with epoll coming out a bit faster than poll in that situation.
- jakevoytko 16y ago
  "I need polling" => "Here are my options, which one is better?" => "They're good for different things" => "I'll pick the best one for the environment" is a reasonable design process. More so than some decisions that I make! Yes, there's a fixed time budget. But you're suggesting selectively ignoring evidence when designing a program, preferring random guessing and pattern matching to actual numbers. Should he have collected them to begin with? Maybe not, but sometimes you can't help your curiosity on a hobby project :). I understand your concern about this hypothetical production system, but the fact of the matter is that there is no production system right now, and no way to measure how it will handle certain things, but there are benchmark numbers. Better than nothing, I say!
  - llimllib 16y ago
    > => "I'll pick the best one for the environment" Which, in reality is, "I'll spend a lot of design and implementation effort designing a new one which may or may not improve the measurable, global performance of my new web server because it's not yet at the point where I can benchmark these sorts of things to verify that I'm not wasting a whole ton of effort that could be better spent by deciding that epoll is fast enough." Maybe Zed knows from his previous server experience that {e}poll is where he hits a bottleneck; it's just that if there's any chance that it's not, he could be wasting a bunch of time implementing "superpoll". (Or maybe he just wants to do it because it's neat, or because it's innovative (which it is), or for any number of other reasons. I'm just pointing out that he's doing much more than picking "the best one for the environment")
    - axod 16y ago
      ... or it's good "PR" so people get behind the project. It screams "I know what I'm doing!!! I'm even optimizing this!!!". ;)
    - zedshaw 16y ago
      It's an idea I had after actually measuring. If it doesn't work then I tried something out. What you really should be getting from it though is that epoll is not faster. It is not O(1). It is not faster on smaller vs. larger lists of FDs. Pretty much all the things you were told as advantages of epoll are total crap. The only advantage of epoll is it's O(N=active) when poll is O(N=total). That's it. So at a minimum I've done some education and spent some time learning something.
- zedshaw 16y ago
  Again with this idea that Mongrel2 isn't working. You sir have no freaking idea what you're talking about. "That's just plain wrong. Premature optimisation does not refer to having to measure before you optimise, it refers to optimising things that in practice may have little or no effect on the actual performance of the program." No, that's just plain wrong. Premature optimisation is actually implementing something convoluted thinking it's optimized without knowing whether it actually is or not. It's voodoo cargo cult science. It's going against occam's razor. There's nothing in Mongrel2 that's premature optimized. It's all very simple algorithms chosen for the right task, and later on I'll be testing them to see if they're still right. So your claim that this is premature optimization is just a buzzword and completely offensive. I took a long time to actually test my ideas before implementing them. That's the total inverse of premature optimization. What is total voodoo junk science is most of what you say. So far I haven't seen one set of data or any scientific experimental design or even a single testable hypothesis backing what you claim. It's all just rhetoric. Until you've got hard numbers backing what you say, everything you're saying is inferior to what I'm doing: science.
  - jacquesm 16y ago
    So, if it's working why not throw a load of real world traffic at it in stead of this 'science' that you're performing here ? After all, that is where the rubber meets the road and it would be a very easy way to determine if your hunch is right or not. Epoll was specifically created with that sort of workloads in mind, your 'surprising' conclusion is not rooted in the fact that epoll is somehow behaving in a way that is contrary to expectation, in fact it behaves exactly as it is designed to do. Benchmarking it like this is nothing like the real world, and that's where epoll shines, not when you test it the way you just did. As for the numbers, we're serving about 10Gbps continuously using a combination of varnish and java code to several million uniques daily, html, images, video. Poll over epoll is a run race, as far as I'm concerned you're wasting your time with this. But by all means, ignore all this and do what you have to, those are the lessons learned best anyway, and it's your time, not mine. If you feel like getting another view on this I'd suggest to contact the author of Varnish, he really knows his stuff and he might be able to convince you where I can not.
axod 16y ago
FWIW, I've written an async webserver which handles a few thousand concurrent users, and does thousands of HTTP reqs/second. I've never seen poll/epoll as a bottleneck, but I'm using Java NIO which really seems to work extremely well (I don't remember how/which it uses, I think epoll). Maybe Java does some of this cool stuff already so perhaps I'm shielded from the pain of dealing with things directly. In the past I've written Java NIO code that dealt with around 60,000 concurrent connections pretty well. The time spent doing poll seemed to be completely insignificant. CPU usage was negligible. It'd be good to see some numbers though - for example: For average mongrel application, 40% of CPU time is spent in poll / average of 30ms latency is due to poll etc. But I'm skeptical those numbers are true. That was my point. If you don't start with those numbers and measurements, optimizations like this, whilst interesting, may end up being of no real use to anyone.
- zedshaw 16y ago
  > But I'm skeptical those numbers are true. That was my point. Yeah, I'm skeptical those numbers are true too, but then again we're talking about totally different numbers.

jacquesm 16y ago

> A conclusion reached by measurement is not premature. That's just plain wrong. Premature optimisation does not refer to having to measure before you optimise, it refers to optimising things that in practice may have little or no effect on the actual performance of the program. By doing these tests in isolation instead of while running on a profiling kernel under production load it is very well possible that the bottleneck will not be the polling code at all but something entirely different. I'd say that this is a textbook example of what premature optimisation is all about. Assuming you have a finite budget of time to spend on a project any optimisations done that take time out of that budget that could have been spent more effective elsewhere is premature. Now there is a chance that this would have been the bottleneck in the completed system, but before you've got a complete system you can't really tell. My guess based on real world experience with lots of system level code that used both, including web servers, video servers, streaming audio servers and so on is that the overhead of poll/epoll will be relatively minor compared to other parts of the code and the massive amount of IO that typically follows a poll or an epoll call. If you have 10K sockets open then typically poll/epoll will return a large number of 'active' descriptors, you'll then be doing IO on all of those for that single call to poll/epoll. Each of those IO calls is probably going to be as much or more work to process than the poll call was.

mfukar 16y ago
If you have 10K sockets open then typically poll/epoll will return a large number of 'active' descriptors, ...<snip> That's half true - it doesn't hold for low ATR traffic (lots of hanging connections, clients that GET something, spend time elsewhere while in the meantime the browser keeps the connection alive). In short, there's nothing typical about it because, while those two kinds of loads have been studied extensively in both bibliography and practice, their combination and the practical consequences are not well understood, afaik. Links to relevant studies are more than welcome, of course.
- jacquesm 16y ago
  Large in an absolute sense, say 3K for a pool of 10K sockets. That's a sizeable number of active connections to deal with after a single system call. Typically for each of those fds you'll then do a read or a write. So the epoll/poll overhead is fairly small, with epoll coming out a bit faster than poll in that situation.
jakevoytko 16y ago
"I need polling" => "Here are my options, which one is better?" => "They're good for different things" => "I'll pick the best one for the environment" is a reasonable design process. More so than some decisions that I make! Yes, there's a fixed time budget. But you're suggesting selectively ignoring evidence when designing a program, preferring random guessing and pattern matching to actual numbers. Should he have collected them to begin with? Maybe not, but sometimes you can't help your curiosity on a hobby project :). I understand your concern about this hypothetical production system, but the fact of the matter is that there is no production system right now, and no way to measure how it will handle certain things, but there are benchmark numbers. Better than nothing, I say!
- llimllib 16y ago
  > => "I'll pick the best one for the environment" Which, in reality is, "I'll spend a lot of design and implementation effort designing a new one which may or may not improve the measurable, global performance of my new web server because it's not yet at the point where I can benchmark these sorts of things to verify that I'm not wasting a whole ton of effort that could be better spent by deciding that epoll is fast enough." Maybe Zed knows from his previous server experience that {e}poll is where he hits a bottleneck; it's just that if there's any chance that it's not, he could be wasting a bunch of time implementing "superpoll". (Or maybe he just wants to do it because it's neat, or because it's innovative (which it is), or for any number of other reasons. I'm just pointing out that he's doing much more than picking "the best one for the environment")
  - axod 16y ago
    ... or it's good "PR" so people get behind the project. It screams "I know what I'm doing!!! I'm even optimizing this!!!". ;)
  - zedshaw 16y ago
    It's an idea I had after actually measuring. If it doesn't work then I tried something out. What you really should be getting from it though is that epoll is not faster. It is not O(1). It is not faster on smaller vs. larger lists of FDs. Pretty much all the things you were told as advantages of epoll are total crap. The only advantage of epoll is it's O(N=active) when poll is O(N=total). That's it. So at a minimum I've done some education and spent some time learning something.
zedshaw 16y ago
Again with this idea that Mongrel2 isn't working. You sir have no freaking idea what you're talking about. "That's just plain wrong. Premature optimisation does not refer to having to measure before you optimise, it refers to optimising things that in practice may have little or no effect on the actual performance of the program." No, that's just plain wrong. Premature optimisation is actually implementing something convoluted thinking it's optimized without knowing whether it actually is or not. It's voodoo cargo cult science. It's going against occam's razor. There's nothing in Mongrel2 that's premature optimized. It's all very simple algorithms chosen for the right task, and later on I'll be testing them to see if they're still right. So your claim that this is premature optimization is just a buzzword and completely offensive. I took a long time to actually test my ideas before implementing them. That's the total inverse of premature optimization. What is total voodoo junk science is most of what you say. So far I haven't seen one set of data or any scientific experimental design or even a single testable hypothesis backing what you claim. It's all just rhetoric. Until you've got hard numbers backing what you say, everything you're saying is inferior to what I'm doing: science.
- jacquesm 16y ago
  So, if it's working why not throw a load of real world traffic at it in stead of this 'science' that you're performing here ? After all, that is where the rubber meets the road and it would be a very easy way to determine if your hunch is right or not. Epoll was specifically created with that sort of workloads in mind, your 'surprising' conclusion is not rooted in the fact that epoll is somehow behaving in a way that is contrary to expectation, in fact it behaves exactly as it is designed to do. Benchmarking it like this is nothing like the real world, and that's where epoll shines, not when you test it the way you just did. As for the numbers, we're serving about 10Gbps continuously using a combination of varnish and java code to several million uniques daily, html, images, video. Poll over epoll is a run race, as far as I'm concerned you're wasting your time with this. But by all means, ignore all this and do what you have to, those are the lessons learned best anyway, and it's your time, not mine. If you feel like getting another view on this I'd suggest to contact the author of Varnish, he really knows his stuff and he might be able to convince you where I can not.

axod 16y ago

FWIW, I've written an async webserver which handles a few thousand concurrent users, and does thousands of HTTP reqs/second. I've never seen poll/epoll as a bottleneck, but I'm using Java NIO which really seems to work extremely well (I don't remember how/which it uses, I think epoll). Maybe Java does some of this cool stuff already so perhaps I'm shielded from the pain of dealing with things directly. In the past I've written Java NIO code that dealt with around 60,000 concurrent connections pretty well. The time spent doing poll seemed to be completely insignificant. CPU usage was negligible. It'd be good to see some numbers though - for example: For average mongrel application, 40% of CPU time is spent in poll / average of 30ms latency is due to poll etc. But I'm skeptical those numbers are true. That was my point. If you don't start with those numbers and measurements, optimizations like this, whilst interesting, may end up being of no real use to anyone.

zedshaw 16y ago
> But I'm skeptical those numbers are true. That was my point. Yeah, I'm skeptical those numbers are true too, but then again we're talking about totally different numbers.