7 ms·
The C10K problem
- deleted 13y ago[deleted]
- kev009 13y agoYes.
- spacemanmatt 13y agoLast time I built a server project from scratch it was on a 300MHz quad-core Xeon base, which hosted around 150 simultaneous web users. It took some effort to make it scale like that but it was worth it. My hardware maintenance was low because we really maximized the software capacity. In modern times, RAM and CPUs are more than 10 times bigger and faster, but I am seeing people get around 25 times LESS out of them, because they choose terrible tools, don't benchmark, and generally don't care. Now (different company) our app server pool has 150 nodes and each serves 4 users. The application complexity is significantly smaller than what I did 13 years ago. I sincerely doubt any of my coworkers have read this document. It shows.
- fiatmoney 13y ago"It took some effort to make it scale" It's often a lot easier to rely on horizontal scaling, especially on a strategic level when you're deciding whether to trust your organization's technical ability (which really means management's hiring ability) or your ability to call AWS and spin up another hundred servers. "Easier" doesn't mean "cheaper" or "faster" in the long run, but it is lower-variance for sure.
- philwelch 13y agoHorizontal scaling isn't just about throwing hardware at the problem. It's about distributing your application over n number of nodes without single points of failure.
- jcrites 13y agoI'd argue there's one more criterion for horizontal scalability: an increase in hardware capacity (number of nodes N) must yield a proportional increase in system throughput. In other words, horizontal scalability means being _able_ to throw hardware at the problem :-) Anyway, I agree with fiatmoney's sentiment that if you're in a corporate environment where your goal is growth, it's often best to focus on ensuring that your systems can grow along with demand, in a confident, easy, operations-free way -- as opposed to ensuring you've eked every bit of performance out of the hardware you have. In many business projects, the cost of human engineering time exceeds the cost of hardware. Even when considering large-scale systems, it makes sense to scrutinize optimizations. If you had time to implement only one of these two equal-effort projects, which would you choose? (a) An optimization project expected to reduce a recurring yearly cost by $1 million (b) A new feature development project expected to earn an additional $2 million yearly recurring revenue? I've found it effective to look at all engineering decisions as business decisions. From a business perspective, the mistake at the center of fallacies like premature optimization is the failure to consider opportunity cost. (On the other hand sometimes pushing a technical project as far as you can is its own reward. Not everything in life has to be about business. Whether or not the results are widely relevant to typical businesses, I look forward to seeing the research into the C10K and C10M problems!)
- justincormack 13y agoPeople are often too optimistic about these additional $2m, while cutting yearly cost by $1m would allow you to hire several more developers, or increase your runway to profitability. There is a business case for both...
- sokoloff 13y ago> If you had time to implement only one of these two equal-effort projects, which would you choose? Impossible to answer without knowing the gross margin of your business. For a big-box retailer (low margins), take the first. For a high-(gross)-margin business, take the second.
- davidw 13y agoA lot of the calculation is the $$$ you make per connection. For a SaaS kind of thing, you can generally keep that high enough to not worry about squeezing every last drop of efficiency out of things. A much bigger worry for most startup-ish companies is finding product market fit.
- DrJ 13y agoWhile I would agree that trying to squeeze every last drop of efficiency may not be worth the time/money for the company. I think the parent is implying that people are wasting a lot of extra resources due to poor tools, poor coding, poor profiling, poor maintenance, etc. I would agree though that for startups, the problem is finding the place to $$$, it is important to know what the cost per user/connection/instance is and being able to control or monitor it.
- gberger 13y ago(1999)?
- wging 13y agoArguably (2009) at the earliest. Was certainly originally written earlier but it seems like something of a living document. The first sentence after the table of contents is: "See Nick Black's execellent Fast UNIX Servers page for a circa-2009 look at the situation."
- jleader 13y agoIf you scroll down to the bottom, there's a changelog describing some changes applied during 2003-2011, and then "Copyright 1999-2014", which might give a few clues about how old the document is.
- vangale 13y agoI was definitely using this page as a resource in 2000-2001. I was going to try and use Python Medusa module (precursor of Twisted) to scale connections.
- Aloisius 13y agoMmm. It has been updated over the years, but of course, 10K was relatively easy even back then. For fun one day, I turned off the search index on one of the production machines and hit over 90K at Napster without much trouble (production ran at ~36K). And that was on little dual processor Pentium 2 machines.
- lugg 13y agoIf you dont have a blog I think you should blog some stories from those days. I'd be really interested to hear them.
- mturmon 13y agoYes, here is the original 1999 snapshot. It has been updated a bunch over the years. http://web.archive.org/web/19990508164301/http://www.kegel.com/c10k.html http://web.archive.org/web/19990508164301/http://www.kegel.c... I think Dan wrote this page originally when he was at Disappearing, Inc. Later he was at Google (2004-10).
- dontdieych 13y agoIf you decide to publish long serious article to web, please consider use css for better readability. Don't rely on browser default css too much. They are very busy to make javascript fast. Especially, please don't use 100% width. Sure, our displays have far more width than height. But that width is good for movie, not text. Put any books on your monitor. Then reduce text line width. https://www.readability.com/articles/zrgovuxr https://www.readability.com/articles/zrgovuxr TW;DR: Too Wide, Didn't Read.
- blago 13y agoThere is a bookmarklet for this: javascript:(function(){ document.body.style.width='600px'; })()
- dontdieych 13y agoMostly I'm fine with default css but only 100% width. This could be helpful in many old article like this. Thanks
- ricardobeat 13y agoThis article is 15 years old. There wasn't much in the way of CSS back then, much less responsive design. But the brilliant thing is it still works perfectly, because it's so simple! Pop your tab to a window and resize, or use a readability browser extension/website. Oh, you just did that.
- elwell 13y agohaha yeah
- Joeboy 13y agoI can pretty much always read sites that don't bother to override the browser defaults, whereas people's CSS efforts often make things hard to read. It's obviously possible to make sites nicer using CSS, but if you're just trying to let people read your words rather than show off your design skills, leaving it to the browser's defaults is absolutely fine.
- pyalot2 13y agoThe 10k problem is still not solved. A run of the mill VPS typically manages around 1k simultaneous connections and 10k requests/s. That is, over plain http. Over TLS this goes down to about 100 connections and 600 requests/s. A cheap amazon instance will usually be around 1/5th of all that. With a bit of tweaking you can get the plain http case a bit higher, but the TLS route will not get much better, because TLS isn't written with speed in mind (the handshake is slow and expensive) and the prevalent implementation (OpenSSL) is not written for high performance servers (it basically dictates that you run blocking sockets in a thread per connection). Unfortunately spdy and various http2 proposals rely on TLS (in order to punch trough proxies), which means going back in server performance about 10 years. So it is of little surprise that companies have started offering "cloud" solutions, because the typical VPS can't handle todays high traffic internet over TLS (worse than plain http by a factor of 10) and the typical cloud server is worse than a VPS by a factor 5, creating a 50x performance degradation, artificially. Obviously when faced with the question of running 10 servers, or 100, most small companies turn to the even worse "cloud" solution, requiring even more servers (500). The whole affair is a sodding mess, and we're wasting massive amounts of energy and capital on insisting on doing things inefficiently. This is because by rights our VPS servers should easily be able to break trough the 10k limit in every way, but it can't because the OS wastes a lot of time running an inefficient network stack as well as that TLS and OpenSSL can't be bothered to get their act together. And that is how, in the year of our lord 2014, more than 15 years after somebody writing the 10k problem article, and after webservers becoming at least 128x faster then back in the 90ties, most websites out there can still not stand up to serious traffic, take ages to load, and are hosted by an infrastructure (routers and whatnot) that easily buckle under even light DDOSing.
- wbsun 13y agoWhen it goes to transactions and stateful sessions, C10K, or even C1K is far from solved yet.
- pyalot2 13y agoBut that's a bit absurd isn't it? A recent core i7 reaches 124850 MIPS. That means that if it takes 1/1000th of a second to handle a connection, the CPU could execute 124 million instructions during that time. At 10/10000th it could still execute 12 million instructions. A minimal asynchronous connection handling certainly doesn't require more than say 10'000 instructions, so our servers are under-delivering on their performance at least by a factor of 1200x, perhaps even by a factor of 12000x. Our servers should be able to surpass C1k easily, even C10k shouldn't tax them. They should, by rights, only be taxed by the C10m problem.
- snaky 13y ago2013 update: The C10M problem It's time for servers to handle 10 million simultaneous connections, don't you think? After all, computers now have 1000 times the memory as 15 years ago when the first started handling 10 thousand connections. Today (2013), $1200 will buy you a computer with 8 cores, 64 gigabytes of RAM, 10-gbps Ethernet, and a solid state drive. Such systems should be able to handle: - 10 million concurrent connections - 10 gigabits/second - 10 million packets/second - 10 microsecond latency - 10 microsecond jitter - 1 million connections/second http://c10m.robertgraham.com/p/manifesto.html http://c10m.robertgraham.com/p/manifesto.html
- peterwwillis 13y agoRead the "Other Performance Metric Relationships" part at the bottom of this page[1]. Basically, just because your machine may be able to physically hold 10 million connections open, does not mean your machine could handle opening 10 million connections in a reasonable amount of time, much less handling 10 million transactions in a reasonable amount of time. If you can't open that many connections at once, or process that many transactions, just being able to keep them open becomes moot. This article[2] breaks down the issues fairly well. In order to handle this kind of traffic, you have to basically redesign huge swaths of technology that exist because we don't want to have to implement these things more than once. I don't see how anyone would invest in this without a specific itch to be scratched (like deep packet inspection). [1] http://www.cisco.com/web/about/security/intelligence/network_performance_metrics.html http://www.cisco.com/web/about/security/intelligence/network... [2] http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html http://highscalability.com/blog/2013/5/13/the-secret-to-10-m...
- jrmenon 13y agoBig fan of kqueue() mentioned in the article. IIRC with sockets, it not only tells you if the socket fd is ready (say for non-blocking read), but even informs the number of bytes available to read which allows you write efficient code (i.e. not to some fixed buffer where you may need to loop in again to see if more data needs to be read). Also I think for files/directories, you can listen for any changes that occur. Wish this was officially available in Linux.