9 ms·
500,000 Requests/Sec – Modern HTTP Servers Are Fast
- jamesu 15y agoA comparison with older hardware would have been nice. Also using localhost is not a very good indicator of real-world performance.
- krakensden 15y agoThe flipside is that as time goes on, this benchmark becomes less impressive. The server in question had 24GB of RAM and two CPUs with 12 cores each.
- mthreat 15y agoI'm interested in more info on your linux TCP tuning, especially how you decided to tcp_tw_recycle and tcp_fin_timeout (and how you decided setting the former to 1 is safe)
- rdoherty 15y agoSigh, another completely synthetic benchmark with big iron: A dual Intel Xeon X5670 with 24GB of RAM from SoftLayer. The X5670 has 6 cores @ 2.93 GHz, 2 threads per core, /proc/cpuinfo shows 24 CPUs. Serving a plain HTML page over localhost via nginx. A medium length blog post with pretty much no real-world information.
- prolepunk 15y agoSo it is a pretty obvious link bait for whatever company they are advertising.
- getsat 15y agonginx was probably using around 20mb ram, too. 24gb is ridiculously unnecessary, but it was probably the least the author could get with the rest of the specs.
- rdl 15y agoThat's a commodity 2CPU machine these days, and about a month from being dethroned by the new E5 Xeons. Serving over multiple 10GE to front end cache servers physically colocated with it would be realistic -- not sure how much of a hit that would be vs. over localhost. Static vs. dynamic content is a reasonable optimization to make for an application under heavy load.
- rdoherty 15y agoThat's a good point about the hardware. I was initially comparing it to cloud hosting where you'd most likely not have access to multiple cores.
- jules 15y agoStill, for comparison a huge site like reddit is serving around 1000 pageviews per second. The hardware in this post isn't that good, nowadays you can do quite a bit better. If you could get even 0.02% of these pageviews per second out of more modern hardware that's already 100 pageviews per second. Only a handful of sites need more than that. For example HN certainly does not. And AFAIK HN runs single core on a language (Arc) that compiles to Scheme, not the fastest in town. Since reddit is only about 70x larger than HN, we are probably close to the point where with a lot of optimization a site like reddit could be run on a single large box (if we're not there already).
- ma2rten 15y agoDude... I am sorry, but your reply does not make sense on many levels: I don't know if reddit could run on a single server, but this benchmark does nothing to prove it. A pageview on reddit consist of several http requests. A http requests translates into several requests into a database backend. In this syntactic benchmark he is requesting the same static file over and over again, so nginx and/or linux can cache it in memory (or CPU cache). Also even if they could put reddit on a single server, it would require a lot of optimization and engineering time is more expensive than hardware. It would also make adding new futures in the future a pain in ass, because you would have to be very greedy with CPU time (and other resources like memory, bandwidth, etc...). Also you want a multi-server setup for redundancy and handling extra capacity. EDIT: I don't if you also planning to put the database on the same server? Web servers are basically stateless so they are easy to scale. It would be interesting to calculate how much disks that server would need just to handle the IO throughput. Full disclosure: This is some speaking who used to work for social network with 9M users and 3,000 servers.
- jules 15y agoThis benchmark does not, but the fact that Hacker News which is just 70x smaller but conceptually similar runs on a single core on a slow language does. I completely agree that it's probably not be a good idea, but the fact that it's possible or almost possible is still interesting in my opinion. If nothing else the power of modern hardware puts the obsession with horizontal scalability into perspective, since most sites are nowhere near the size of reddit. Even though a pageview on reddit translates to several HTTP requests, only one of those requests has to serve dynamic HTML. The requests that get the JS, CSS and images can go to a content delivery network (and they probably already do). You'd probably indeed want to put the database on the same server, with the vast majority of DB accesses hitting main memory (as you can easily get a server with hundreds of gigabytes or even terabytes of RAM nowadays -- it helps that reddit's access pattern is very heavily concentrated on new posts). Of course you have to replicate to multiple servers to prevent data loss. 9M users and 3,000 servers sounds like a lot. What kind of servers are these? What is the size of the database, roughly?
- sciurus 15y agoBig iron? That server couldn't cost more than $5,000.
- kev009 15y agoBig iron? I guess if you've been spoofed into the cloud hysteria and you pay for an anemic node at the same price a dedicated runs.
- NeutronBoy 15y agoIn a real world app, I think you'd be considered lucky if the web-server is your bottleneck, rather than your DB or network connection.
- RegEx 15y agoAs someone who has only worked on extremely small sites, I've never had to deal with scaling/bottlenecks. Do you have any recommended reads on handling DB bottlenecks and the like so I can be prepared when that day comes? Thanks!
- charliepark 15y agoGregg Pollock made a series of videos (underwritten by RPM (https://rpm.newrelic.com/) https://rpm.newrelic.com/)), called Scaling Rails (http://railslab.newrelic.com/scaling-rails http://railslab.newrelic.com/scaling-rails). It's phenomenal. That link shows the "contents" on the left-hand sidebar. Start at the bottom and work your way up the list.
- beambot 15y agoThat looks phenomenal. Is it equally-applicable to, say, Django development? Or is there a sister-series?
- Luyt 15y agoJacob Kaplan-Moss did a talk on 'Django Deployment' which is really more about scalability (reverse proxies in front of your webapp) and reliability (multiple database backends etc). http://ontwik.com/python/django-deployment-workshop-by-jacob-kaplan-moss/ http://ontwik.com/python/django-deployment-workshop-by-jacob...
- alexlitov 15y agoQuickly looking over SoftLayer's price sheet, it looks to be a little over $1k a month for Intel 5670 with 24GB of RAM
- shingen 15y agoSoftlayer is a lot more expensive than most hosts these days. They're closer to being in Rackspace territory now. You can get a dual E5-2609 with 32gb of ram + bandwidth from a good host like WebNX for $369 / month. And if you wanted to spend $1k per month, you can get a dual e5-2687W (16 cores @ 3.1ghz) with 128gb ram; with a 4x SAS RAID 10, plus a 120gb ssd drive, plus a 2tb backup freebie, plus 100mbps unmetered bandwidth (30tb). That literally blows the doors off Softlayer.
- CPlatypus 15y agoLiterally? Did you literally fall asleep in English class?
- shingen 15y agoDouche much? Looking over your comments on HN, that latest comment seems to be the peak of your contribution here. In over two years you've picked up 394 karma points. I'm going with the assumption that all of those karma points are from being a pedant about the English language. You're such a valuable member of this community! Rock on.
- halayli 15y agonginx saturates at ~18k req/sec / core with a latency of ~24ms. This saturation is not coming from nginx in particular but from OS limits (mode switching, stack copying etc.. for read/write system calls). There is nothing new in "modern HTTP servers". They are event-driven programs and this has existed for a long time.
- fpp 15y agoAs you rightly said - the key limit if you get to such high I/O levels is first with OS processing - the implementation of the network stack on the iron - with performance above 50k the limit comes from the "pipes" moving the data up from the network card to the OS layer where the web server sits (and providing the layer 4 / TCP services). Hence in telcom environments where you get such requirements the TCP/IP stack is processed closer to the card - but no out-of-the-box web servers could handle such streams - most of that is handled with e.g. C-libs closely bound to the networking HW. With such specialized NW cards (16 core card) plus libs you get to > 1'500'000 teardowns p.s. / connections established and roughly 15Gbps - there are now 100Gbps solutions on the market on the top-end - your actual throughput depends mostly what kind of processing you're doing. That kind of equipment is of course not normally used for web serving - more for (transparent) proxying and inspection / traffic shaping, etc as more-and-more telcom cores are completely TCP/IP based.
- halayli 15y agoI think using a generic OS for IO operations is not a good idea in general. We all do it but there got to be a better solution. We don't need kernel/userland isolation that introduces mode switching, stack copying, copying arguments etc... In such an OS, the environment is under our supervision and processes are trusted.
- fpp 15y agoWith the telcom example I've provided above there is no generic OS used for that - the protocol (e.g. TCP, IP or even HTTP) is (pre-)processed within the hardware and not further up in the OS. Generally Linux / Unix (carrier grade / HA) is used within such environments as the OS. Context switches (user land / kernel) are some of the most expensive operations so they are to be avoided as often as possible. What you're pointing to are features you generally find in RTOS (real-time OS) solutions - and these kind of cards / platforms allow using them. Look up the Trillium platform from CCPU for example how such a stack / system is layered - they use Wind River PNE-LE (Linux edition for network equipment) for the HA & non-HA protocols.
- ghempton 15y agoThe ironic part is that his blog is hosted on wordpress.
- acidx 15y agoMy toy web server can achieve similar performance using a much more modest hardware (Core i7 2640 laptop), using way less RAM (a few dozen kilobytes). Granted this is also being tested on localhost with static content (no disk I/O) -- but shows that event-driven servers are not that novel or difficult to write: my code weighs around 1700 LOC of (might I say) readable C. Static file serving is also fast (using sendfile(), etc), but needs an overhaul to achieve usable concurrency. Currently there's a ~4x performance drop while serving files, but I'm working on this. (The sources are at http://github.com/lpereira/lwan http://github.com/lpereira/lwan by the way.)
- stuhood 15y agoSince this is almost pure sendfile() work (aside from the headers), it really doesn't seem like a very useful example... there isn't much static content left on the web.
- morsch 15y agoJust to give a different point of view: I'm sure you're right per request, which I guess is what matters here. But per volume, most content on the web is static.
- qwe1234_troll 15y agoModern HTTP Servers Are Fast... ...when they aren't doing anything. Fascinating news!