12 ms·

10 Million Concurrent Connections – The Kernel is the Problem

257 points by snaky 13y ago

wmf 13y ago
I think the original source is easier to read: http://c10m.robertgraham.com/p/manifesto.html http://c10m.robertgraham.com/p/manifesto.html
gioele 13y ago
> The problem with packets is they go through the Unix kernel. The network stack is complicated and slow. The path of packets to your application needs to be more direct. Don’t let the OS handle the packets. Why don't you focus on simplifying the network stack instead? The stack is complicated and slow for a reason: it takes care of many things. I will believe that you can do better when you provide the same amount of functionality (QoS, firewall, probing, tracing). If you say that you can do without all these additional features, why don't you go and optimize the stack so that the most basic code path is smaller and faster?
- toki5 13y ago
  If you have one specialized need (i.e. one specific path for packets to travel), then it is just as valid an approach to trash every other path. To turn your own question around: Why bother trying to optimize a stack that contains a lot of stuff you don't strictly need? The talk doesn't deal with general-purpose servers that perform multiple roles; it says "if you have a singular role, here is how you hyper-optimize to support that role at a huge scale."
  - gioele 13y ago
    > If you have one specialized need (i.e. one specific path for packets to travel), then it is just as valid an approach to trash every other path. You only have one specialized need at the very beginning. Then you start seeing the need for "just another little feature". And soon you will start replicating big parts of the network stack. Call it Greenspun's tenth rule for the network, if you want. Also, most of the time, what you "just need" happens to be also the common need of many other users out there. Joining forces to fix a single code path is a much better investment than redoing things in userland for the sake of it.
- lmm 13y ago
  Would linux accept those patches?
  - yid 13y ago
    It doesn't have to -- you can just have a loadable kernel module that takes over the network stack.
    - wmf 13y ago
      Linux developers are very opposed to having their lovingly crafted network stack replaced with something crappy and/or proprietary (see TOE).
    - lmm 13y ago
      Such a thing would be very difficult to maintain given that linux refuses to provide a stable API.
      - yid 13y ago
        Agreed, but it's possible (in theory, much like writing your own network stack).
- wmf 13y ago
  This is a worthwhile topic of research and there are some papers at HotOS about it, but I don't blame people for taking shortcuts if they can't solve the general case.
- PeterisP 13y ago
  Well, the whole point of the article is that for some needs even the option of extra functionality hurts - you want to remove every extra 'if', every unneeded (for you) field in a data structure to make it fit in cache better, etc.
abraininavat 13y ago
First of all, what a disorganized article. Who the hell edited this? Secondly, the article surmises that the kernel is the problem, which seems right. Then it makes a leap to state that the way to do this is to write your own driver. Who said? I certainly don't agree. You are still running a multi-user kernel, but you've now sacrificed its stability by running a parallel network stack. Linux wasn't written to work in that way and it's hard to know what you're getting into when you subvert your OS in that way. This article would be much better if it talked about other options out there. For example.. Why not take out the middle-man entirely with something like OpenMirage (http://www.openmirage.org/ http://www.openmirage.org/)? Get rid of the lowest-common-denominator performance you get using a general-purpose OS and make your application the OS. Link in whatever OS-like services (network, filesystem, etc) you may need. Talk to the hardware directly without going through a middle-man and without ugly hacks.
- jsnell 13y ago
  Well, he's mostly suggesting using a user-space networking stack provided by your vendor (Intel's DPDK). But writing your own userspace driver from scratch is actually very simple -- I've done it for Intel's 1Gbps and 10Gbps cards before the DPDK was made publicly available. We're talking ~1000 lines of user-space code, and less than a month of work. Writing your own userspace networking stack is of course more complicated, depending on exactly which level of protocol support you need. Supporting say ARP or EtherChannel is trivial, while a production quality and interoperable TCP stack will be man-years of work. But having written systems doing 10M TCP connections at 10Gbps, I strongly believe that you want to relegate the kernel into the role of a management system. Having the system split across the kernel and user-space will lead to unacceptable performance, pretty much no matter where you split things. And having the full application in the kernel would be a nightmare for development, debugging, and deployment. (And I sure as hell am not going to choose a pre-alpha research project over Linux as the base of telecoms systems.)
  - clgbrst 13y ago
    Have you personnally studied the code of this research project?
jtchang 13y ago
Well of course it is the problem. The kernel does lots of other stuff. If you were to ask someone to build a vehicle that can go really fast you might end up with a car. But ideally you'd really want a rocket. I'm sure that there are systems out there that serve web pages with only bare metal. Where the "kernel" exists only as an architectural stub. Why shouldn't the NIC serve web pages directly?
- PeterisP 13y ago
  For example, there are systems that do video streaming directly from NAND-memory to NIC, bypassing the kernel to ensure high throughput (say, multiple 100Gbps optic links) and no jitter - linux handles the control part (data management, initiating new streams, etc), but the data part is outside of it.
minimax 13y ago
Don’t scribble data all over memory via pointers. Each time you follow a pointer it will be a cache miss: [hash pointer] -> [Task Control Block] -> [Socket] -> [App]. That’s four cache misses. Incidentally, game programmers have been spreading the gospel on this issue for several years now. See for more: http://macton.smugmug.com/gallery/8936708_T6zQX#!i=593426709&k=ZX4pZ http://macton.smugmug.com/gallery/8936708_T6zQX#!i=593426709... http://research.scee.net/files/presentations/gcapaustralia09/Pitfalls_of_Object_Oriented_Programming_GCAP_09.pdf http://research.scee.net/files/presentations/gcapaustralia09... http://www.altdevblogaday.com/2011/11/07/an-example-in-data-oriented-design-sound-parameters/ http://www.altdevblogaday.com/2011/11/07/an-example-in-data-...
- dietrichepp 13y ago
  I couldn't help but laugh reading the first link. Sure, there are some good ideas for optimization. But (1) no profiling, and (2) there are a couple really bogus suggestions. For example, suggestion #33, to shift by a constant amount instead of a variable amount. You see, it only looks like you're shifting by a variable amount. This is one of those things that compilers have been optimizing for years and are very good at: strength reduction wrt variables that depend on the loop variable. You'll also see game programmers do things like "x >> 4" instead of "x/16", because "x >> 4" is faster. It is faster in assembly language, but your compiler already knows that and you are just making the code harder to read every time you do a micro-optimization. Game programmers spread the gospel on lots of premature optimization nonsense, in my opinion, and are mostly inexperienced when it comes to writing mantainable code. It's a kind of hazard of the industry. Performance problems means cutting beloved features, and rather than doing any maintenance you just start a new game from scratch. (Not universally, of course. There are a few programmers professionally working on engines.)
  - pasquinelli 13y ago
    everything you said is super true. hella super true.
  - snaky 13y ago
    >mostly inexperienced when it comes to writing mantainable code Mostly. Just like any other average guys in any industry. But game industry is continuously bringing back many interesting technologies, especially performance-related. Like LuaJIT, rapidly becoming mainstream and widely used, GPU-assisted calculation, used even in PostgreSQL - https://wiki.postgresql.org/wiki/PGStrom https://wiki.postgresql.org/wiki/PGStrom - and so much more.
tankenmate 13y ago
On the face of it this is a nonsensical problem; a 10Gbps ethernet connection is not going to need 10M concurrent tcp connections; ethernet + tcp/ip protocol overhead is approx 4% (62 bytes overhead from 1542 bytes max per packet, no jumbo packets over the wider internet yet), the average UK broadband connection is now over 12Mbps (http://media.ofcom.org.uk/2013/03/14/average-uk-broadband-speeds-hit-double-figures/ http://media.ofcom.org.uk/2013/03/14/average-uk-broadband-sp...), that gives approximately 800 connections to fill the pipe. Even at a paltry 1% bandwidth usage per connection and 4x10Gbps adaptors in the server that is still 320,000 connections, I have done 150k connections on a high end desktop Linux machine. Available memory (4kx2 socket buffers; you do want connections to be as fast as possible no?) and bandwidth will be a limit to the number of connections long before you get to 10M connections. You are far better off buying multiple machines (redundancy) in more than one location (even more redundancy) before heading off to load yourself up with technical debt that could choke a zoo full of elephants. The Linux kernel has come a long way in the last few years with improvements to SMP scaling of sockets, zero copy, and large numbers of file handles; make use of it! The level of technical skill being applied to fix kernel problems is probably more expensive than you can afford.
- snaky 13y ago
  >Available memory (4kx2 socket buffers; you do want connections to be as fast as possible no?) When 1U commodity x86 server built on C602 Intel chipset can be packed with 768 GB RAM..
- zokier 13y ago
  There are usecases that use minimal to no bandwidth but a lot of connections. IMAP-IDLE comes to mind as a premier example. Essentially anything that requires "push" capability these days relies on open connections, other examples are websockets and instant messaging. Also the number of concurrent users is on the rise because of the always-on nature of cellphones. While 10M sounds bit far fetched today, I think that using available bandwidth as a measure for connection count is misguided.
  - tankenmate 13y ago
jacob019 13y ago
writing custom drivers for each NIC is a blocker. Could 10M be achieved with raw sockets?
- alexkus 13y ago
  libnetfilter_queue for rx, raw socket for tx. That was going to be the basis for a project to handle large numbers of concurrent connections. Still on the drawing board though.
tzury 13y ago
a different version of the same presentation, a more "class(y)" one... http://www.youtube.com/watch?v=ZFvnPAIo4F0 http://www.youtube.com/watch?v=ZFvnPAIo4F0
JoeAltmaier 13y ago
Embedded programmers have been scheduling I/O like this for decades as well. A thread is a lot on an embedded kernel.
- spartango 13y ago
  The point about statically allocating "huge pages" aligns with my experience as an embedded programmer as well. Dynamic allocation is simply a no-no on those systems.
  - JoeAltmaier 13y ago
    Usually because of the runtime - locks, garbage collection strike when you can't afford it (realtime buffering). So my approach is a heap-cache usually. Calculate the log2 size, choose a heap bucket, if its empty THEN go to the heap. Never frees so never garbage collects (neither does single-large-allocation so no loss), just relinks freed blocks on the cache bucket by size for simple reuse. It soon comes to a working-set and never takes longer than a critical-section and link/unlink, at least once it settles down.
flametroll 13y ago
Here is an interesting concept that thinks 'outside the box': http://www.youtube.com/watch?v=gq1vDG-st1k http://www.youtube.com/watch?v=gq1vDG-st1k
- mindjiver 13y ago
  Interesting how similar this is an advanced 4G cell network. In previous generations of networks there was always a "controller" node taking care of the handover between the cells. This of course didn't scale very well when there was a spike in traffic usage. So in 4G cell networks there are no controllers and all handovers are done via the cells instead. Interesting to see this thinking applied to something slightly different, thanks for the link!
haberman 13y ago
> The kernel is the problem I had an epiphany one day when I realized that the kernel is nothing but a library with an expensive calling convention. The only reason we bother calling the kernel at all is because it has privileges that userspace programs don't have, and it uses those privileges to control/mediate access to shared resources. The downside of the kernel is that there's no way around it. You can't "opt out" of any of its decisions. With any normal library, if had an O(n^2) algorithm somewhere, or wasn't optimized for your use case, or just generally got in the way, you could choose not to link against it. User-space libraries are democratic in this respect; you vote with your linker line. But with the kernel, it's "my way or the highway." The kernel is the only way to the hardware. Here's an unfortunate example: O_DIRECT is one of those few ways that you can sidestep the kernel. With O_DIRECT you can bypass the kernel's page cache, which there are very good reasons for wanting to do. But Linus's opinion is that "I should have fought back harder": https://lkml.org/lkml/2007/1/10/233 https://lkml.org/lkml/2007/1/10/233 He thinks it's unfortunate that you can sidestep the kernel, because "You need a buffer whatever IO you do, and it might as well be the page cache." But what if you want better memory accounting, or better isolation between users, or a different data structure, or any number of other tweaks to the page cache implementation? Well thankfully we have O_DIRECT. Otherwise, your only choice would be to try to convince the Linux kernel maintainers to integrate your change, after you've tweaked it so that it's suitable for absolutely everyone else that uses Linux, and given it an interface that Linux is willing to support forever. Good luck with that. The kernel has always been the problem. User-space is where it's at.
- fulafel 13y ago
  > The downside of the kernel is that there's no way around it. You can't "opt out" of any of its decisions. Well, you can sidestep most of it. For example, re the c10k/c10m problem, you can the transport protocol impl to userspace. See example here how they did it with SCTP: http://www.cs.ubc.ca/labs/dsg/mpi-sctpuserspace_ICCCN2012.pdf http://www.cs.ubc.ca/labs/dsg/mpi-sctpuserspace_ICCCN2012.pd... - you could do the same with TCP if you don't care about having a TCP-enabled kernel. The other example of course is all the protocols built on top of UDP. This still has the NIC driver shoveling at least the raw IP packets via the kernel driver, but you could sidestep that too by using the hypervisor interface to route the NIC pci space and interrupts to your program (like Xen uses).
chrismealy 13y ago
Where are people getting these super cheap servers?
byte_bach 13y ago
Well, sorry to say I don't buy the "Unix was designed as a phone switch control plane" nonsense at all. Here is Dennis Ritchie: "From the point of view of the group that was to be most involved in the beginnings of Unix (K. Thompson, Ritchie, M. D. McIlroy, J. F. Ossanna), the decline and fall of Multics had a directly felt effect. We were among the last Bell Laboratories holdouts actually working on Multics, so we still felt some sort of stake in its success. More important, the convenient interactive computing service that Multics had promised to the entire community was in fact available to our limited group, at first under the CTSS system used to develop Multics, and later under Multics itself. Even though Multics could not then support many users, it could support us, albeit at exorbitant cost. We didn't want to lose the pleasant niche we occupied, because no similar ones were available; even the time-sharing service that would later be offered under GE's operating system did not exist. What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form. We knew from experience that the essence of communal computing, as supplied by remote-access, time-shared machines, is not just to type programs into a terminal instead of a keypunch, but to encourage close communication. " Unix was designed for people, not for Bell System Switches: http://cm.bell-labs.com/cm/cs/who/dmr/hist.html http://cm.bell-labs.com/cm/cs/who/dmr/hist.html
m1ke 13y ago
Anyone can recommend a good place to start reading about *nix kernel for non-programmers? Or any other programing concepts that are being referred in the post? I'm pretty green sysop guy and want to try and understand better the post (currently I'm getting maybe 10-20%)
tmzt 13y ago
This presentation hints at one possible implementation of this, specifically for VoIP/RTP. http://www.cluecon.com/presentation/building-conferenceivr-platform-nodejs-sipjs-and-webrtc/ http://www.cluecon.com/presentation/building-conferenceivr-p... I would love to see actual working code for IP/SIP/RTP in node.js using packet sockets, if anybody knows of an open source project.
anon1685 13y ago
If this became a real problem, wouldn't the pragmatic solution be to distribute the load across multiple machines? I fail to see how handling 10M connections on a single machine has any merit, taking into account reliability for example, apart from being able to brag about it.
zerop 13y ago
What should be I doing to handle thousands of quick and short lived connections?

257 points by snaky 13y ago

wmf 13y ago

I think the original source is easier to read: http://c10m.robertgraham.com/p/manifesto.html http://c10m.robertgraham.com/p/manifesto.html

gioele 13y ago

> The problem with packets is they go through the Unix kernel. The network stack is complicated and slow. The path of packets to your application needs to be more direct. Don’t let the OS handle the packets. Why don't you focus on simplifying the network stack instead? The stack is complicated and slow for a reason: it takes care of many things. I will believe that you can do better when you provide the same amount of functionality (QoS, firewall, probing, tracing). If you say that you can do without all these additional features, why don't you go and optimize the stack so that the most basic code path is smaller and faster?

toki5 13y ago
If you have one specialized need (i.e. one specific path for packets to travel), then it is just as valid an approach to trash every other path. To turn your own question around: Why bother trying to optimize a stack that contains a lot of stuff you don't strictly need? The talk doesn't deal with general-purpose servers that perform multiple roles; it says "if you have a singular role, here is how you hyper-optimize to support that role at a huge scale."
- gioele 13y ago
  > If you have one specialized need (i.e. one specific path for packets to travel), then it is just as valid an approach to trash every other path. You only have one specialized need at the very beginning. Then you start seeing the need for "just another little feature". And soon you will start replicating big parts of the network stack. Call it Greenspun's tenth rule for the network, if you want. Also, most of the time, what you "just need" happens to be also the common need of many other users out there. Joining forces to fix a single code path is a much better investment than redoing things in userland for the sake of it.
lmm 13y ago
Would linux accept those patches?
- yid 13y ago
  It doesn't have to -- you can just have a loadable kernel module that takes over the network stack.
  - wmf 13y ago
    Linux developers are very opposed to having their lovingly crafted network stack replaced with something crappy and/or proprietary (see TOE).
  - lmm 13y ago
    Such a thing would be very difficult to maintain given that linux refuses to provide a stable API.
    - yid 13y ago
      Agreed, but it's possible (in theory, much like writing your own network stack).
wmf 13y ago
This is a worthwhile topic of research and there are some papers at HotOS about it, but I don't blame people for taking shortcuts if they can't solve the general case.
PeterisP 13y ago
Well, the whole point of the article is that for some needs even the option of extra functionality hurts - you want to remove every extra 'if', every unneeded (for you) field in a data structure to make it fit in cache better, etc.

abraininavat 13y ago

First of all, what a disorganized article. Who the hell edited this? Secondly, the article surmises that the kernel is the problem, which seems right. Then it makes a leap to state that the way to do this is to write your own driver. Who said? I certainly don't agree. You are still running a multi-user kernel, but you've now sacrificed its stability by running a parallel network stack. Linux wasn't written to work in that way and it's hard to know what you're getting into when you subvert your OS in that way. This article would be much better if it talked about other options out there. For example.. Why not take out the middle-man entirely with something like OpenMirage (http://www.openmirage.org/ http://www.openmirage.org/)? Get rid of the lowest-common-denominator performance you get using a general-purpose OS and make your application the OS. Link in whatever OS-like services (network, filesystem, etc) you may need. Talk to the hardware directly without going through a middle-man and without ugly hacks.

jsnell 13y ago
Well, he's mostly suggesting using a user-space networking stack provided by your vendor (Intel's DPDK). But writing your own userspace driver from scratch is actually very simple -- I've done it for Intel's 1Gbps and 10Gbps cards before the DPDK was made publicly available. We're talking ~1000 lines of user-space code, and less than a month of work. Writing your own userspace networking stack is of course more complicated, depending on exactly which level of protocol support you need. Supporting say ARP or EtherChannel is trivial, while a production quality and interoperable TCP stack will be man-years of work. But having written systems doing 10M TCP connections at 10Gbps, I strongly believe that you want to relegate the kernel into the role of a management system. Having the system split across the kernel and user-space will lead to unacceptable performance, pretty much no matter where you split things. And having the full application in the kernel would be a nightmare for development, debugging, and deployment. (And I sure as hell am not going to choose a pre-alpha research project over Linux as the base of telecoms systems.)
- clgbrst 13y ago
  Have you personnally studied the code of this research project?

jtchang 13y ago

Well of course it is the problem. The kernel does lots of other stuff. If you were to ask someone to build a vehicle that can go really fast you might end up with a car. But ideally you'd really want a rocket. I'm sure that there are systems out there that serve web pages with only bare metal. Where the "kernel" exists only as an architectural stub. Why shouldn't the NIC serve web pages directly?

PeterisP 13y ago
For example, there are systems that do video streaming directly from NAND-memory to NIC, bypassing the kernel to ensure high throughput (say, multiple 100Gbps optic links) and no jitter - linux handles the control part (data management, initiating new streams, etc), but the data part is outside of it.

minimax 13y ago

Don’t scribble data all over memory via pointers. Each time you follow a pointer it will be a cache miss: [hash pointer] -> [Task Control Block] -> [Socket] -> [App]. That’s four cache misses. Incidentally, game programmers have been spreading the gospel on this issue for several years now. See for more: http://macton.smugmug.com/gallery/8936708_T6zQX#!i=593426709&k=ZX4pZ http://macton.smugmug.com/gallery/8936708_T6zQX#!i=593426709... http://research.scee.net/files/presentations/gcapaustralia09/Pitfalls_of_Object_Oriented_Programming_GCAP_09.pdf http://research.scee.net/files/presentations/gcapaustralia09... http://www.altdevblogaday.com/2011/11/07/an-example-in-data-oriented-design-sound-parameters/ http://www.altdevblogaday.com/2011/11/07/an-example-in-data-...

dietrichepp 13y ago
I couldn't help but laugh reading the first link. Sure, there are some good ideas for optimization. But (1) no profiling, and (2) there are a couple really bogus suggestions. For example, suggestion #33, to shift by a constant amount instead of a variable amount. You see, it only looks like you're shifting by a variable amount. This is one of those things that compilers have been optimizing for years and are very good at: strength reduction wrt variables that depend on the loop variable. You'll also see game programmers do things like "x >> 4" instead of "x/16", because "x >> 4" is faster. It is faster in assembly language, but your compiler already knows that and you are just making the code harder to read every time you do a micro-optimization. Game programmers spread the gospel on lots of premature optimization nonsense, in my opinion, and are mostly inexperienced when it comes to writing mantainable code. It's a kind of hazard of the industry. Performance problems means cutting beloved features, and rather than doing any maintenance you just start a new game from scratch. (Not universally, of course. There are a few programmers professionally working on engines.)
- pasquinelli 13y ago
  everything you said is super true. hella super true.
- snaky 13y ago
  >mostly inexperienced when it comes to writing mantainable code Mostly. Just like any other average guys in any industry. But game industry is continuously bringing back many interesting technologies, especially performance-related. Like LuaJIT, rapidly becoming mainstream and widely used, GPU-assisted calculation, used even in PostgreSQL - https://wiki.postgresql.org/wiki/PGStrom https://wiki.postgresql.org/wiki/PGStrom - and so much more.

tankenmate 13y ago

On the face of it this is a nonsensical problem; a 10Gbps ethernet connection is not going to need 10M concurrent tcp connections; ethernet + tcp/ip protocol overhead is approx 4% (62 bytes overhead from 1542 bytes max per packet, no jumbo packets over the wider internet yet), the average UK broadband connection is now over 12Mbps (http://media.ofcom.org.uk/2013/03/14/average-uk-broadband-speeds-hit-double-figures/ http://media.ofcom.org.uk/2013/03/14/average-uk-broadband-sp...), that gives approximately 800 connections to fill the pipe. Even at a paltry 1% bandwidth usage per connection and 4x10Gbps adaptors in the server that is still 320,000 connections, I have done 150k connections on a high end desktop Linux machine. Available memory (4kx2 socket buffers; you do want connections to be as fast as possible no?) and bandwidth will be a limit to the number of connections long before you get to 10M connections. You are far better off buying multiple machines (redundancy) in more than one location (even more redundancy) before heading off to load yourself up with technical debt that could choke a zoo full of elephants. The Linux kernel has come a long way in the last few years with improvements to SMP scaling of sockets, zero copy, and large numbers of file handles; make use of it! The level of technical skill being applied to fix kernel problems is probably more expensive than you can afford.

snaky 13y ago
>Available memory (4kx2 socket buffers; you do want connections to be as fast as possible no?) When 1U commodity x86 server built on C602 Intel chipset can be packed with 768 GB RAM..
zokier 13y ago
There are usecases that use minimal to no bandwidth but a lot of connections. IMAP-IDLE comes to mind as a premier example. Essentially anything that requires "push" capability these days relies on open connections, other examples are websockets and instant messaging. Also the number of concurrent users is on the rise because of the always-on nature of cellphones. While 10M sounds bit far fetched today, I think that using available bandwidth as a measure for connection count is misguided.
- tankenmate 13y ago

jacob019 13y ago

writing custom drivers for each NIC is a blocker. Could 10M be achieved with raw sockets?

alexkus 13y ago
libnetfilter_queue for rx, raw socket for tx. That was going to be the basis for a project to handle large numbers of concurrent connections. Still on the drawing board though.

tzury 13y ago

a different version of the same presentation, a more "class(y)" one... http://www.youtube.com/watch?v=ZFvnPAIo4F0 http://www.youtube.com/watch?v=ZFvnPAIo4F0

JoeAltmaier 13y ago

Embedded programmers have been scheduling I/O like this for decades as well. A thread is a lot on an embedded kernel.

spartango 13y ago
The point about statically allocating "huge pages" aligns with my experience as an embedded programmer as well. Dynamic allocation is simply a no-no on those systems.
- JoeAltmaier 13y ago
  Usually because of the runtime - locks, garbage collection strike when you can't afford it (realtime buffering). So my approach is a heap-cache usually. Calculate the log2 size, choose a heap bucket, if its empty THEN go to the heap. Never frees so never garbage collects (neither does single-large-allocation so no loss), just relinks freed blocks on the cache bucket by size for simple reuse. It soon comes to a working-set and never takes longer than a critical-section and link/unlink, at least once it settles down.

flametroll 13y ago

Here is an interesting concept that thinks 'outside the box': http://www.youtube.com/watch?v=gq1vDG-st1k http://www.youtube.com/watch?v=gq1vDG-st1k

mindjiver 13y ago
Interesting how similar this is an advanced 4G cell network. In previous generations of networks there was always a "controller" node taking care of the handover between the cells. This of course didn't scale very well when there was a spike in traffic usage. So in 4G cell networks there are no controllers and all handovers are done via the cells instead. Interesting to see this thinking applied to something slightly different, thanks for the link!

haberman 13y ago

> The kernel is the problem I had an epiphany one day when I realized that the kernel is nothing but a library with an expensive calling convention. The only reason we bother calling the kernel at all is because it has privileges that userspace programs don't have, and it uses those privileges to control/mediate access to shared resources. The downside of the kernel is that there's no way around it. You can't "opt out" of any of its decisions. With any normal library, if had an O(n^2) algorithm somewhere, or wasn't optimized for your use case, or just generally got in the way, you could choose not to link against it. User-space libraries are democratic in this respect; you vote with your linker line. But with the kernel, it's "my way or the highway." The kernel is the only way to the hardware. Here's an unfortunate example: O_DIRECT is one of those few ways that you can sidestep the kernel. With O_DIRECT you can bypass the kernel's page cache, which there are very good reasons for wanting to do. But Linus's opinion is that "I should have fought back harder": https://lkml.org/lkml/2007/1/10/233 https://lkml.org/lkml/2007/1/10/233 He thinks it's unfortunate that you can sidestep the kernel, because "You need a buffer whatever IO you do, and it might as well be the page cache." But what if you want better memory accounting, or better isolation between users, or a different data structure, or any number of other tweaks to the page cache implementation? Well thankfully we have O_DIRECT. Otherwise, your only choice would be to try to convince the Linux kernel maintainers to integrate your change, after you've tweaked it so that it's suitable for absolutely everyone else that uses Linux, and given it an interface that Linux is willing to support forever. Good luck with that. The kernel has always been the problem. User-space is where it's at.

fulafel 13y ago
> The downside of the kernel is that there's no way around it. You can't "opt out" of any of its decisions. Well, you can sidestep most of it. For example, re the c10k/c10m problem, you can the transport protocol impl to userspace. See example here how they did it with SCTP: http://www.cs.ubc.ca/labs/dsg/mpi-sctpuserspace_ICCCN2012.pdf http://www.cs.ubc.ca/labs/dsg/mpi-sctpuserspace_ICCCN2012.pd... - you could do the same with TCP if you don't care about having a TCP-enabled kernel. The other example of course is all the protocols built on top of UDP. This still has the NIC driver shoveling at least the raw IP packets via the kernel driver, but you could sidestep that too by using the hypervisor interface to route the NIC pci space and interrupts to your program (like Xen uses).

chrismealy 13y ago

Where are people getting these super cheap servers?

byte_bach 13y ago

Well, sorry to say I don't buy the "Unix was designed as a phone switch control plane" nonsense at all. Here is Dennis Ritchie: "From the point of view of the group that was to be most involved in the beginnings of Unix (K. Thompson, Ritchie, M. D. McIlroy, J. F. Ossanna), the decline and fall of Multics had a directly felt effect. We were among the last Bell Laboratories holdouts actually working on Multics, so we still felt some sort of stake in its success. More important, the convenient interactive computing service that Multics had promised to the entire community was in fact available to our limited group, at first under the CTSS system used to develop Multics, and later under Multics itself. Even though Multics could not then support many users, it could support us, albeit at exorbitant cost. We didn't want to lose the pleasant niche we occupied, because no similar ones were available; even the time-sharing service that would later be offered under GE's operating system did not exist. What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form. We knew from experience that the essence of communal computing, as supplied by remote-access, time-shared machines, is not just to type programs into a terminal instead of a keypunch, but to encourage close communication. " Unix was designed for people, not for Bell System Switches: http://cm.bell-labs.com/cm/cs/who/dmr/hist.html http://cm.bell-labs.com/cm/cs/who/dmr/hist.html

m1ke 13y ago

Anyone can recommend a good place to start reading about *nix kernel for non-programmers? Or any other programing concepts that are being referred in the post? I'm pretty green sysop guy and want to try and understand better the post (currently I'm getting maybe 10-20%)

tmzt 13y ago

This presentation hints at one possible implementation of this, specifically for VoIP/RTP. http://www.cluecon.com/presentation/building-conferenceivr-platform-nodejs-sipjs-and-webrtc/ http://www.cluecon.com/presentation/building-conferenceivr-p... I would love to see actual working code for IP/SIP/RTP in node.js using packet sockets, if anybody knows of an open source project.

anon1685 13y ago

If this became a real problem, wouldn't the pragmatic solution be to distribute the load across multiple machines? I fail to see how handling 10M connections on a single machine has any merit, taking into account reliability for example, apart from being able to brag about it.

zerop 13y ago

What should be I doing to handle thousands of quick and short lived connections?