10 ms·
How to handle 1000s of concurrent users on a 360MB VPS
- jschuur 15y agoWorth noting the the story is from Dec '09.
- gaius 15y agoPeople were doing this with Squid years before that, in the 90s.
- iphoneedbot 15y agoYeah.. squid is actually great. I do wish squid is a packaged that just needs to be turned on for most VPS
- stock_toaster 15y agoAt the end of the article, why is apache even being used anymore? Just use nginx and php-fpm (or php-cgi/fcgi).
- ramidarigaz 15y agoThat's what I use. I thought about the method in the article, but it seemed too complicated, and nginx + fcgi is plenty fast for me. Are there any performance benefits to having Apache as well?
- stock_toaster 15y agoPerformance benefits? No, I wouldn't think so. I can think of too many performance downsides to it. The only benefits I can think of offhand would be if you had a requirement for something that Apache specifically provided, or custom modules you needed run. Something like suphp, modsecurity, or shared hosting configuration, etc. All policy type stuff, not performance.
- mfjordvald 15y agoNone at all. It's purely because most people aren't willing to invest the time it would take to not need .htaccess. In fact by not using PHP-FPM you lose out on awesome stuff like the slow log!
- geuis 15y agoI use nginx exclusively as my server and I run node instances and wordpress behind it. Why even use apache?
- etherealG 15y agomind if I ask what you use to tie wordpress in to nginx? cgi?
- ognyankulev 15y agoAs written in another comment, nginx + php5-fpm is a great combo, no need for Apache at all. I run both Drupal and WordPress (the latter against my will :-P ) in such a setup. I think the only reason people use nginx -> Apache mod_php is because there's a lot of documentation and knowledge about Apache. nginx requires a little bit different mental model how HTTP requests are served but it's worth it. EDIT: You can think of PHP5-FPM as Apache+mod_php but only serving PHP requests. So nginx handles static files and URL magic and when there's request for PHP, it passes it to PHP5-FPM, just like it would pass it to Apache+mod_php. This is oversimplifed but it's kind of like that.
- chopsueyar 15y agoI had a similar configuration for a Wordpress blog, and another Linode instance running ExpressionEngine. I have nginx configured with php-fpm to use sockets to communicate between each other.
- muppetman 15y agoYou can do the same on a really high traffic Drupal site using the boost module. It's basically a module that will, for non-logged in users, generate a static html page and serve that instead. PHP isn't even called, the .htaccess just serves up the cached content. And that's using Apache!
- tobylane 15y agoI've heard a lot of bad things about htaccess, is its slowness comparable to php over static, or apache over nginx?
- true_religion 15y agohtaccess is just a configuration file used by Apache to determine what should be served to whom.
- tobylane 15y agoWhich can slow things down, seems much less of a problem now than when I last dealt with them. There's a lot to read on using them to speed websites up. Minus two for an on topic question?
- true_religion 15y agoThe amount of slow-down is relative to what you actually write in the htaccess file. You can have two very different files that actually do the same thing.
- handelaar 15y ago.htaccess is read and parsed on every request. Putting the same directives in the server's virtual host config is a ton more efficient. Me, I use php-fastcgi daemons with APC and nginx, with no Apaches. The speed bump and the drop in memory requirements were staggering. The main 'cost' is figuring out how to convert rewrite rules from one type to another, but if you're using a widely-known app, someone's already done it for you.
- mise 15y agoI've been on Apache servers for years, so I come from that frame of mind. My managed VPSs are running Apache, and one of them is running nginx for static resources. I think my managing host (Servint) doesn't even support nginx. How, for argument's sake, would I move my PHP/MySQL sites to run on a managed nginx server?
- ComputerGuru 15y agoAt the risk of stating the obvious, you would move to a better VPS that charges as much or less for a better package :)
- chopsueyar 15y agoHe means Linode.
- gsharma 15y agoRepost: http://news.ycombinator.com/item?id=970682 http://news.ycombinator.com/item?id=970682 Interesting comments on the original thread.
- stock_toaster 15y agooh wow. I didn't even notice the posted date on the original article until I followed the repost link you provided, and saw how long ago that had been posted.
- nwmcsween 15y agonginx is actually pretty bad at all benchmarks I've thrown at it, you can see some here: http://www.webhostingtalk.com/showthread.php?t=918355 http://www.webhostingtalk.com/showthread.php?t=918355 this was on an 8 core intel Xeon 5520 server system it maxed 8 cores (18ghz) @ 7500 requests per second for static content.
- SaltwaterC 15y agoFirst of all, there's no such thing as 18GHz. The GHz don't sum up. It's like saying that if a woman makes a child in 9 months, two women can make a child in 4.5 months. It doesn't work that way. Moving forward to the actual issue. Something is really broken into your configuration. The first thing is the file AIO. But this is Linux's fault. The file AIO is broken by design. It bypasses the kernel file cache. Which is a pretty large performance hit per se. The second thing is the fact that I can easily get 55k req/s on a Q9400 with nginx, without even touching the sysctl parameters of a Ubuntu 10.04 x86_64. The only actual difference is nginx 1.0.x instead of the ancient 0.7.x you would actually find into the standard repos aka the production version I publish into my company's private repo.
- mfjordvald 15y agoI haven't read through the forum thread as there are ten pages and much of it seems off-topic. But the requests per second doesn't even matter here, the fact that you were capable of maxing the CPU at all shows me you did something wrong. I easily get 12k requests on an i7 box and the CPU wasn't even stressed then. Eventually it was IO that limited it.
- chopsueyar 15y agoEven though it was eight cores, he only had 1024KB of system RAM.
- stock_toaster 15y agoI found that forum thread quite hard to follow, and there are blobs of ab runs sprinkled all over. I clicked a few pages through, and didn't see any configs posted, sysctl values, etc. Was it more of an ad-hoc test you were running just to get some vague ideas about performance, as apposed to a comprehensive performance comparison? My personal (anecdotal) experience shows that nginx is amazingly fast. Some nginx configs can impact performance if you aren't careful (I am thinking of try_files vs if-tests[1]), and there are various nginx tunables (worker process count, cpu affinity, etc) as well. I have personally seen issues with apachebench when there are lots of connections (not to mention linux system tunables such as max file descriptors!), iptables state tracking, etc... [1]: http://wiki.nginx.org/IfIsEvil http://wiki.nginx.org/IfIsEvil
- mmaunder 15y agoThis is surreal. I just wrote a draft blog entry about the changes in startup economics due to event-servers like node and nginx (publishing tomorrow), and I hit HN just before heading to bed and this blog entry is #1. Thanks for all the love today guys. Have an awesome memorial day weekend if you're in the States.
- PawelDecowski 15y agoOr bank holiday weekend if you're in the UK ;) I've been using nginx for about 2 years and it performs so much better than Apache. I'm not using Apache as a backend though, I use FastCGI. It performs even better.
- jrockway 15y agoWhy use nginx instead of Varnish as the frontend proxy? If you are using nginx as your http <-> fastcgi gateway, that's one thing, but if you're just using it as a reverse proxy, it seems like a lot of extra stuff to maintain. I like living on the bleeding edge, so I do varnish -> mongrel2 <-> apps instead... but varnish -> nginx <-> apps or varnish -> apache <-> apps seem equally reasonable. The key is caching and being able to handle slow users without tying up your app. All three of these setups do that. nginx to apache only saves your app server; it doesn't do much caching.
- mmaunder 15y agoI must admit I've never used varnish, but from what I gather, it's function is caching, not handling large numbers of concurrent connections. Nginx's strength is handling tens of thousands of concurrent connections with very little incremental CPU load and a tiny memory footprint. It then reverse proxies those connections to a threaded-model server occupying each thread for very little time and quickly freeing them up for other work. From what I gather, varnish uses a threaded model with 1 thread per connection which is what nginx solves. So it might make sense to put nginx in front of varnish so you get high concurrency with great caching. Nginx does offer some caching but I've played with it in high load environments and wasn't impressed with the performance.
- StavrosK 15y agoVarnish is arguably more performant than varnish. I've had amazing success with both, and I don't think both are necessary together, but they're both screaming fast. I noticed one day that varnish had served 100,000 visitors per day from the cache due to a Google mistake only because I saw it in my analytics. The server load was at 0.05 all day. I love varnish. Hard.
- flexd 15y agoVarnish is ridiculously amazing. I use it directly in front of unicorn to host some ruby apps. The performance is just amazing compared to the load avg. No need for a 'proper' web server.
- deleted 15y ago[deleted]
- chopsueyar 15y agoAre the loopback requests limited by the TCP stack? Is it possible to set this up where nginx and apache communicate via a socket?
- arianb 15y agoYes, and that's the recommended way of setting it up. :)
- ericb 15y agoDropping keepalives actually slightly hurts your best-case performance. As a config change, if you are using the prefork mpm (I wouldn't if scalability is your goal) it will generally help you though. With prefork the scaling math on worker memory * connections vs your box's memory is often the first place things will fall apart if you have keepalives on and real traffic. I suggest the worker mpm if you don't want to switch to nginx.
- jacques_chester 15y agoMissing: caching. Caching. Caching. And more caching. I speak of Wordpress and database-backed PHP applications of its ilk. Wordpress loves to spray MySQL in a thick stew of queries for every single page load. Without sensible -- preferably aggressive -- caching, your Wordpress site will die in the arse. My setup generates cached .gz files, which nginx can serve directly. It flies. It didn't used to.
- aonic 15y agoOr if you dont want to deal with Wordpress to implement the caching, use Varnish. (also mentioned few times in this thread) I've used Varnish for some high-traffic Wordpress sites by sitting it in front of Apache running Wordpress. With well planned TTLs, and logged-in cookie detection, most of the inconveniences of caching can be eliminated. Also, using Disqus or similar services for comments will be helpful. You wont have to worry about comments not showing up until the cache clears, leading to confused users. That is, if you use the javascript implementation of these comment services. Alternatively, plugins like WP Super Cache have automatic post cache flushing if there are updates, and you can also tell it to bypass cache for logged in users. But it is IO-bound.
- euroclydon 15y agoWP Super Cache will serve up cached files via PHP with the flip of a switch. While this isn't as fast as letting the server serve up cached files, it's an easy win.
- nicksergeant 15y agoIt's not missing, he specifically addressed caching at the end of the article (indicating he didn't cover it).
- getsat 15y agoThis is the best caching plugin I've found for WP: http://wordpress.org/extend/plugins/w3-total-cache/ http://wordpress.org/extend/plugins/w3-total-cache/ It supports disk-based caches and memcached.
- otterley 15y agoThis technique naïvely assumes the only thing that is slow is the client. If a backend data provider that the web server communicates with (e.g., database) is also slow, this arrangement merely adds complexity without much corresponding benefit. The origin server will still block waiting on the data provider, which can cause process starvation in the pathological case. Moreover, issues with slow clients often can be solved by raising the TCP send buffer size. As long as the response size is less than the send buffer size, it really doesn't matter how slow the client is: write() will return immediately, leaving the webserver free to serve the next request. Getting the data to the client then becomes the kernel's responsibility.
- SaltwaterC 15y agoThat would work, unless you're using a container-based VPS. OpenVZ is notorious for standing in the way of system tuning. Using nginx makes a lot of sense in this case, especially with the ey-balancer patch.
- otterley 15y agoThe TCP send buffer size can be controlled at the application level (see SO_SNDBUF in tcp(7)). There's no need to adjust kernel sysctl settings.
- SaltwaterC 15y agoBy the manual page you mentioned: http://linux.die.net/man/7/socket http://linux.die.net/man/7/socket > SO_SNDBUF: [...] The default value is set by the wmem_default sysctl and the maximum allowed value is set by the wmem_max sysctl. You can't use sysctl under OpenVZ. QED.
- otterley 15y agoFair enough - then you'll have to switch to a different VPS if you're sending > 128KB responses. Use the right tool for the job.
- iphoneedbot 15y agoThis approach really works well... I would add that off-loading MySQL to a different VPS within the same network works well too; latency within the same network is very negligible. Doing simple optimization such as moving/disabling mail and stuff you probably dont need works too.
- jacques_chester 15y agoIndeed, pushing MySQL onto a different VPS in the same network basically halved page generation times for my sites. This is partly because when a table contains a TEXT field, MySQL performs all joins against that table on disk, regardless of what fields you're selecting or joining on. Which is a problem for, oh, I dunno, every PHP blog engine / bulletin board / CMS ever.
- james4k 15y agoI just find it odd that Apache hadn't employed the likes of epoll already. IRC servers have been using async sockets like this for yeeeears.
- peterwwillis 15y agoWell, it does since apache 2.1 ('event' mpm module, replaces worker)
- SaltwaterC 15y agoEvent MPM is still a large pile of fail. Unless you're unwilling to switch from Apache, the benefits over Worker are mostly theoretical from the performance point of view. Hopefully this will change with 2.4, but only for those who didn't kick Apache out of their production machines.
- mp3geek 15y agoI use straight nginx on Fanboy Adblock (which gets hundreds of SSL connections per minute), no need for Apache.
- peterwwillis 15y agoHint: you can just run Apache twice; once with a mpm_prefork config for your apps, once with a mpm_event config to handle client connections.
- rkalla 15y agoAs a good number of folks have pointed out, this article (helpful for sure) is basically nginx + PHP-FPM. nginx servers up static content (e.g. if you are using WP-SuperCache and it's generating those static cache files for you, JS, CSS, images, etc.) then you configure a pool of warmed up PHP VM instances via PHP-FPM (Check /etc/php5/php-fpm or some equiv dir on your server). Then you setup a rule that directed PHP requests to the FPM service, that would probably look something like this in nginx: location ~ \.php$ { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME /var/www/mysite.com/$fastcgi_script_name; include fastcgi_params; } and instead of adding the layer of Apache servicing the PHP requests, you are having nginx pass through the PHP requests directly to PHP processes to run. Going from an Apache2 + FCGID configuration (somewhat similar to this, but with Apache) to nginx, I saw a 75% drop in server load. I'm almost certain this still isn't a totally tweaked out setup that someone more familiar with this process could do better, but for my needs it took my server crashing under load no matter what I did, to typically being idle most of its life (~900k/month pageviews) So I'm a big fan of nginx. I'm not saying you couldn't configure Apache to do the same... just 5 years of attempts to do so never got me anywhere with it.