5 ms·

Scalable Network Programming

41 points by kirubakaran 18y ago

signa11 18y ago
oldie but a goldie. obligatory non-scribd link (http://bulk.fefe.de/scalable-networking.pdf http://bulk.fefe.de/scalable-networking.pdf). if you are in this business get 'network algorithmics'. read it, be one with it ! it is sicp of networking.
- yagibear 18y ago
  "sicp of networking" is a bit over the top. IMHO it gives undue emphasis to Varghese's research interest in packet classifiers, and is filled with typos.
strlen 18y ago
Should be a mandatory read for anyone who wants to talk about "scaling". There's lot of armchair discussion of this topic from people who have never looked at UNIX networking from a below-the-surface level (or don't even know about select(), epoll and the various UNIX IPC mechanisms).
axod 18y ago
Interesting read, but I didn't see the case that you just don't use processes or threads covered. Did I miss it?
- strlen 18y ago
  The discussion of epoll/select does talk about the event-driven scenario and there's the discussion of simplest web browser (one procession for all connections). Memcached is one example of a purely event driven application. Another example (also by Brad Fitzpatrick) is perlbal (written in Perl and using epoll).
  - axod 18y ago
    Sure, I just expected the doc to go a little more like: 1 process per connection (bad) -> 1 thread per connection (better?) -> 1 process (bingo!)
    - neilc 18y ago
      ~1 process per core, that is (unless the daemon is completely I/O bound).
      - rcoder 18y ago
        This is only true if you think your application can do a better job of scheduling than the underlying OS kernel. In many (if not most) cases, this is false. For HTTP servers hosting static content, you may be able to out-perform the OS thread scheduler. For most non-trivial apps, you're probably wrong. Small fork()-ed processes can still compete with clever poll(), /dev/epoll, or kqueue()-based servers, if you can keep each instance lightweight.
        staunch 18y ago
        > This is only true if you think your application can do a better job of scheduling than the underlying OS kernel. In many (if not most) cases, this is false. Why wouldn't you be able to virtually always do a better job of scheduling events (when you control all the code competing for resources) than a generic OS scheduler? > Small fork()-ed processes can still compete with clever poll(), /dev/epoll, or kqueue()-based servers, if you can keep each instance lightweight. Do you know of any examples of ultra highly scalable fork()ing servers?
        rcoder 18y ago
        > Why wouldn't you be able to virtually always do a better job of scheduling events (when you control all the code competing for resources) than a generic OS scheduler? Simply put, I would say that "you" (where "you" is an average webapp developer) probably don't understand scheduling, event-driven programming, or memory management as well as the average kernel developer. I of course don't mean this to extend to anyone in this thread. Imagine, though, giving your average PHP or ASP developer, who may struggle to implement a basic sort algorithm, the problem of implementing cooperative multitasking in a scalable way. > Do you know of any examples of ultra highly scalable fork()ing servers? Under real-world workloads, I still consider Apache to be "highly scalable." In my experience, given 1:1 investment in hardware for front-end and database servers, the RDBMS craps out long before the webapp, so being able to accept another 5000 incoming connections is only going to hurt you.
- abstractbill 18y ago
  Yeah, I thought for a minute I had skipped ahead a few pages by mistake. One minute it's all about thousands of processes or threads, and the next it talks about select/poll/epoll/kqueue without really introducing the idea properly (but other than that I thought it was great!). Incidentally, the justin.tv chat server uses one thread per cpu, and epoll for its network io, and currently scales to a little over 10,000 connections per cpu (see http://irc.justin.tv/ http://irc.justin.tv/ for other numbers). It's the most scalable irc server that I'm aware of. I keep meaning to write something up about it...
davidw 18y ago
It's been a while since I've checked, but I think he's mistaken that Apache is one process per connection. I think it fires up multiple processes just because that's best if you take a bit to serve each request. He doesn't really cover that aspect of things much. Serving static files quick is, if not quite a solved problem, not terribly hard. Doing applications, with significant processing overhead, is a bit more difficult.
- aristus 18y ago
  So is anyone interested in an essay about that? I've got a few topics on the hook. I used to run a search engine that served 600 million queries per month.
  - bluelu 18y ago
    Now that sounds interesting. Why did you quit or did you sell your search engine? I'm doing some search stuff myself (I haven't launched so far) and it would be nice to be able to get some tips. However I do not expect to reach 600 million queries per month, at least not in the first month ;)
    - aristus 18y ago
      I only ran the initial architecture & buildout and then moved to sysop as they hired people. The rest of the business was not my business. :) After a good few years we hit financial trouble and were bought up by Yahoo for pennies.
      - bluelu 18y ago
        Must have been an interesting job though. :)
huhtenberg 18y ago
http://www.kegel.com/c10k.html http://www.kegel.com/c10k.html is more concise and it has a better coverage of available options. It is the canonical reference for anyone asking "how to handle 10000 clients".
- neilc 18y ago
  It's a great read, although it hasn't been updated since 2006, so it should be read with care.
Hexstream 18y ago
Executive summary of the first 53 pages: epoll is great, the other solutions range from not-so-good to terrible.

41 points by kirubakaran 18y ago

oldie but a goldie. obligatory non-scribd link (http://bulk.fefe.de/scalable-networking.pdf http://bulk.fefe.de/scalable-networking.pdf). if you are in this business get 'network algorithmics'. read it, be one with it ! it is sicp of networking.

yagibear 18y ago
"sicp of networking" is a bit over the top. IMHO it gives undue emphasis to Varghese's research interest in packet classifiers, and is filled with typos.

strlen 18y ago

Should be a mandatory read for anyone who wants to talk about "scaling". There's lot of armchair discussion of this topic from people who have never looked at UNIX networking from a below-the-surface level (or don't even know about select(), epoll and the various UNIX IPC mechanisms).

axod 18y ago

Interesting read, but I didn't see the case that you just don't use processes or threads covered. Did I miss it?

strlen 18y ago
The discussion of epoll/select does talk about the event-driven scenario and there's the discussion of simplest web browser (one procession for all connections). Memcached is one example of a purely event driven application. Another example (also by Brad Fitzpatrick) is perlbal (written in Perl and using epoll).
- axod 18y ago
  Sure, I just expected the doc to go a little more like: 1 process per connection (bad) -> 1 thread per connection (better?) -> 1 process (bingo!)
  - neilc 18y ago
    ~1 process per core, that is (unless the daemon is completely I/O bound).
    - rcoder 18y ago
      This is only true if you think your application can do a better job of scheduling than the underlying OS kernel. In many (if not most) cases, this is false. For HTTP servers hosting static content, you may be able to out-perform the OS thread scheduler. For most non-trivial apps, you're probably wrong. Small fork()-ed processes can still compete with clever poll(), /dev/epoll, or kqueue()-based servers, if you can keep each instance lightweight.
      - staunch 18y ago
        > This is only true if you think your application can do a better job of scheduling than the underlying OS kernel. In many (if not most) cases, this is false. Why wouldn't you be able to virtually always do a better job of scheduling events (when you control all the code competing for resources) than a generic OS scheduler? > Small fork()-ed processes can still compete with clever poll(), /dev/epoll, or kqueue()-based servers, if you can keep each instance lightweight. Do you know of any examples of ultra highly scalable fork()ing servers?
        rcoder 18y ago
        > Why wouldn't you be able to virtually always do a better job of scheduling events (when you control all the code competing for resources) than a generic OS scheduler? Simply put, I would say that "you" (where "you" is an average webapp developer) probably don't understand scheduling, event-driven programming, or memory management as well as the average kernel developer. I of course don't mean this to extend to anyone in this thread. Imagine, though, giving your average PHP or ASP developer, who may struggle to implement a basic sort algorithm, the problem of implementing cooperative multitasking in a scalable way. > Do you know of any examples of ultra highly scalable fork()ing servers? Under real-world workloads, I still consider Apache to be "highly scalable." In my experience, given 1:1 investment in hardware for front-end and database servers, the RDBMS craps out long before the webapp, so being able to accept another 5000 incoming connections is only going to hurt you.
abstractbill 18y ago
Yeah, I thought for a minute I had skipped ahead a few pages by mistake. One minute it's all about thousands of processes or threads, and the next it talks about select/poll/epoll/kqueue without really introducing the idea properly (but other than that I thought it was great!). Incidentally, the justin.tv chat server uses one thread per cpu, and epoll for its network io, and currently scales to a little over 10,000 connections per cpu (see http://irc.justin.tv/ http://irc.justin.tv/ for other numbers). It's the most scalable irc server that I'm aware of. I keep meaning to write something up about it...

davidw 18y ago

It's been a while since I've checked, but I think he's mistaken that Apache is one process per connection. I think it fires up multiple processes just because that's best if you take a bit to serve each request. He doesn't really cover that aspect of things much. Serving static files quick is, if not quite a solved problem, not terribly hard. Doing applications, with significant processing overhead, is a bit more difficult.

aristus 18y ago
So is anyone interested in an essay about that? I've got a few topics on the hook. I used to run a search engine that served 600 million queries per month.
- bluelu 18y ago
  Now that sounds interesting. Why did you quit or did you sell your search engine? I'm doing some search stuff myself (I haven't launched so far) and it would be nice to be able to get some tips. However I do not expect to reach 600 million queries per month, at least not in the first month ;)
  - aristus 18y ago
    I only ran the initial architecture & buildout and then moved to sysop as they hired people. The rest of the business was not my business. :) After a good few years we hit financial trouble and were bought up by Yahoo for pennies.
    - bluelu 18y ago
      Must have been an interesting job though. :)

huhtenberg 18y ago

http://www.kegel.com/c10k.html http://www.kegel.com/c10k.html is more concise and it has a better coverage of available options. It is the canonical reference for anyone asking "how to handle 10000 clients".

neilc 18y ago
It's a great read, although it hasn't been updated since 2006, so it should be read with care.

Hexstream 18y ago

Executive summary of the first 53 pages: epoll is great, the other solutions range from not-so-good to terrible.