9 ms·

250x Speed Improvements with Microcaching (and No New Code)

346 points by taybenlor 15y ago

alnayyir 15y ago
Typical dumb web benchmark ricing, (cf. wildly different concurrency parameters). The usage of apachebench in general is unwise to begin with, but if you're going to use it, at least show data for a variety of tests measuring StdDev across various levels of concurrency. Better off using httperf and knowing how to benchmark things. Worse still, you neglected to mention that the 99/100% cases were much much worse in high concurrency than the low concurrency "slower" test. You were hammering a server that was running on the same machine that was generating the load. Wtf? If you're aiming httperf or ab at "localhost", you're motherfucking doing it wrong. Stop benchmarking shit and blogging about it until you get a grasp of statistics. If you have content that is so read-heavy as to be virtually trivial to cache heavily, just use Varnish.
jbyers 15y ago
Nice technique. The number of distinct dynamic pages you expect to get hammered still must be regenerated within that second. With a longer window, some wp-admin or logged-in-user detection, and a third-party comment service, I could see this being a standard nginx wordpress configuration.
- mhd 15y ago
  What comment systems are you thinking about? With Disqus the rest of the page shouldn't really matter at all, whether it's dynamic, cached, static etc… Depending on one's error tolerance, maybe dynamic configuration would help (check if you're hit heavily and/or check if you're on HN/slashdot/reddit, then modify the config accordingly).
  - IanMikutel 15y ago
    What about the Facebook comment system?
    - nl 15y ago
      Same thing. Any Javascript-based comment system should be ok (because the comments are all served from a 3rd party site). It is when you are using internal wordpress comments you need to be careful.
michaelbuckbee 15y ago
This seems pretty similar to using Varnish or Squid as a reverse proxy (though likely easier to setup).
- taybenlor 15y ago
  Similar sort of approach. A lot less work to set up, especially if you're already running nginx.
deleted 15y ago
[deleted]
- dangrossman 15y ago
  APC on its own is an opcode cache, not a page/data cache. Did you write your own code to save pages into it and retrieve them? Or is there a cache plugin for WordPress you're using which uses APC as its data store?
  - deleted 15y ago
    [deleted]
wheels 15y ago
> This is fine, up until the point where you get on HN and Reddit at the same time Incidentally, you don't actually need much to handle that. Our web server is a wimpy 256 MB VPS and we've had (Wordpress) blog entries hit the front page of HN and Reddit simultaneously and weather the storm without missing a beat. An appropriately setup Apache + Wordpress SuperCache does the trick just fine. (Hint: The default Apache configuration isn't "appropriate".) You're not going to hit anywhere even close to 2k requests per second on the front page of those two.
- timdorr 15y ago
  It depends on the content. If it's something that will be visited just by HN/Reddit users: Sure, you're not going to get 2k/sec directly from them. But if it's something that's going to get shared/tweeted/retweeted/blogged/reblogged/etc., then you will be looking at huge traffic numbers. It just depends on how viral your content is and how far it spreads. It's the long tail that matters.
  - wheels 15y ago
    You've really got to hit the jackpot to get up to 2000/request per second (assuming you don't already have a high traffic site). Assuming about 20 requests per pageview, that's more than a quarter million pageviews per hour. I've had things go moderately viral (thousands of tweets / retweets) and you don't get anywhere near that. Pre-tuning for hitting the jackpot is in most cases going to be premature optimization.
    - seanp2k 15y ago
      ...and you know what they say about premature optimization; "if you write something that people want to read and your site is down, you will look like an idiot" wait, no, wait, that's not right at all. Maybe: "W3 Total Cache will save your site and takes all of 5 minutes to configure" Hmm, nope. Perhaps: "Cloudflare is free and you can probably sweet talk your webhost into setting up the whole thing for you" No, no, no, no. Wait, what were we talking about?
    - 15y ago
mopoke 15y ago
I like this solution and am definitely tempted to give it a go. Anyone got any thoughts on the best way to do this on a page with personalisation? (and this is really simple personalisation - one section of the page changes depending on whether you're logged in or not). My solution would probably be to have the personalised section load as an async request but then you'd need to make sure that the async request can handle the same load as the microcached content. Any other ideas?
- splitrocket 15y ago
  That's how I've done it in the past. Depending on how many personalized requests you are doing, compared to the resource utilization savings you'll get from caching the rest of your content, you'll probably come out ahead.
- _phred 15y ago
  You can put any nginx variable into the proxy_cache_key setting, including cookies and query strings. I've used this to cache localized versions of each page on a site based off of a "language" cookie. Async is a good approach for what you need, but it's good to know what nginx's caching mechanism can do.
splitrocket 15y ago
I've been doing essentially the same thing with my wordpress install. Went from a few hundred reqs/second on a cheap linode to over 4k reqs/second. (I think the limit was the benchmarking tool, not nginx) I've got the nginx config if anyone is interested.
- dbalatero 15y ago
  I'd be interested in this, yeah! My GitHub username is @dbalatero if you want to send it via that.
- splitrocket 15y ago
  Here it is: http://alchemi.st/nginx-wordpress-network-and-fastcgi-cache-the-ultimate-guide/ http://alchemi.st/nginx-wordpress-network-and-fastcgi-cache-... You can change the caching ttl to meet your needs.
brown9-2 15y ago
Is it fair to compare those ab benchmarks when the concurrency value is different for each (4 and 500)? Would be curious to see how the original config handled 500 concurrent requests.
- splitrocket 15y ago
  My experience doing something very similar is that the original config would not handle 500 concurrent requests. It's a fair comparison insofar as the uncached app probably couldn't serve that many concurrent requests in a reasonable amount of time.
patio11 15y ago
A national radio campaign is likely to bring hundreds or thousands of visitors spread over hours, not hundreds of thousands of visitors spread over seconds, so I probably would not take any particular action to harden a site in anticipation of it. It is a poor use of engineering resources and adds technical risk with no corresponding benefit. (n.b. Pasting code you got from a blog post, particularly code marked as kinda broken, is not a risk-free endeavor! I love nginx, don't get me wrong, but paste in snippets from two different blog posts and watch the sparks fly if you don't understand how nginx handles, e.g., location priority.)
- adgar 15y ago
  Not to mention that he's compromising on accuracy. Yeah, 1 second is tiny, but why give it up if you don't have to? Especially those of you reading who want to try this on a potentially more dynamic site. Edit: downvoted with no explanation, cute. And readers here think they're above Reddit standards. There's no reason to compromise the integrity of a system to any extent before you have to. As others have demonstrated, that need was clearly not met here.
  - maratd 15y ago
    > Not to mention that he's compromising on accuracy. Yeah, 1 second is tiny, but why give it up if you don't have to? While NGINX may use very little CPU and have minimal hard drive access, PHP (or Java/Python/Ruby/etc.) will hit those resources hard. Look, I'm not one of those environmentalist wackos who pisses his pants when somebody forgets to turn off a light ... but ... if you're keeping your BLOG server a hundred times busier than it has to be just so that your users don't experience a staleness of 1 second ... you're an asshole. A military or medical application, fine. But a blog? Sorry, no.
    - scott_s 15y ago
      you're an asshole. Not necessary.
      - 15y ago
crikli 15y ago
Is this using nginx in lieu of Apache or nginx in front of apache, acting as a reverse proxy or similar?
nirvana 15y ago
"If you have personalized pages (ie: majority logged-in users) this approach isn't going to work. " I've considered this problem, and am working on a solution for nirvana[1]. The biggest challenge to this project has been to take a language (coffeescript) that is sequential and run it in a distributed environment, without the programmer having to know distributed programming. One of the techniques I'm applying is making a response (in this case, a web page) the result of a collection of components, which are rendered separately in the same context. (EG: The context is the headers of the request, plus the user record if the user is logged in, etc.) So, the request comes in, the components are loaded from the cache, they are executed (in parallel) all with a copy of the state, their results are aggregated and that result can run thru templating to produce a webpage that is returned. The idea then becomes, instead of executing the code for every component in every request, if the component has no context specific requirements (e.g.: it is the same for every user, it's a static element, or it's dynamic, and but doesn't need to be generated every time) .. then it can be flagged as cacheable. The caching would also have a staleness factor (Eg: 1m, 5m 10m).[2] My hope is that you can have pages that are custom per user, but that also contain heavy impact results (say a graph produced by an expensive operation), where the results come form cache, the static components come from cache, but the user specific parts are dynamically generated each request. This component approach not only lets the code be rendered in parallel, and often not even rendered, but instead pulled from cache, but it should allow for more convenient re-use of common elements and features across a site. I hadn't considered caching for just 1s, though. Will have to think about that. [1] Nirvana is CoffeeScript web development backed by erlang and Riak. Instantly distributed coffeescript. It will be open source, hopefully soon. Follow @nirvanacore on Twitter if you're interested in being notified. [2] Planned. There are some implications of this that will require tradeoffs, so initially it may just be a flag of Yes/No for "Cache for up to 1 minute." or some value like that.
- aidenn0 15y ago
  Something I have done in the past is put all the "loaded by everyone but not static" stuff in memcached. The application pushes to memcached on startup, and any POST that would change it causes the application backend to update the memcached entry(ies). This is a slightly different approach from the microcaching, but it has the advantage of being always consistent. I only do this when the stuff starts to obviously become a bottleneck, and I haven't done much with cloud hosting, where RAM seems to be more expensive than cpu cycles, so I don't know how well it would work there.
rymedia 15y ago
Very impressive.
simonw 15y ago
I really like the idea of setting a cookie that bypasses caching for a few seconds - I've heard the same technique used by Facebook, who set a cookie that ties you to the MySQL master server rather than the slave after you perform a write action so that you'll see your update without waiting for replication lag. It relies on using the Max-Age Cookie argument though, and I was under the impression that IE doesn't implement that correctly. Anyone know what the status of IE and max-age cookies is?
- Fenn 15y ago
  Hey Simon, Hmmm, you're right about the IE issues (http://mrcoles.com/blog/cookies-max-age-vs-expires/ http://mrcoles.com/blog/cookies-max-age-vs-expires/). I haven't looked into how possible/hard expires would be in nginx config, but you could probably hack it in with http://wiki.nginx.org/HttpUserIdModule http://wiki.nginx.org/HttpUserIdModule
  - simonw 15y ago
    The problem with the expires argument is that you have to set it to a specific date, but I don't think you can be sure what timezone / clock setting the user's browser has - which means that it's pretty much impossible to reliably set a cookie that expires in a couple of seconds time. Hopefully I'm wrong - I'd love to know if there's a workaround for that issue.
todsul 15y ago
The big issue here is the application type. This only works well for a very particular type of application. That is, a highly dynamic site that is NOT dependent on user logins. 1) If the site is only moderately dynamic, you can just use plain Nginx and set fastcgi_cache to a few minutes or hours. Much less load on the server. I like to keep things simple, I wouldn't even bother with Apache. Porting rewrites to Nginx is super simple. 2) If the site is customised on a per-user basis, 'microcaching' will break the site and have disastrous consequences. Every user will see the system customised for whichever user primed the cache. My primary website is user based. That means this 'microcaching' concept wont work at all. It would be catastrophic. That's where Varnish comes in with ESI. I really wish I didn't have to use Varnish. It's slower than Nginx, it adds another layer of complexity, and in testing, it seems slightly flaky. But what Varnish+ESI allows is caching of parts of my page that aren't user specific. I.e. header, footer, etc. If you want to see my test results of Nginx vs Varnish, see http://todsul.com/nginx-varnish http://todsul.com/nginx-varnish
- chlee 15y ago
  I thought about incorporating Varnish into my side-project as well. However, Varnish seems to be a [potential] hassle when you account for VCL, ESI, and etc. It also adds an extra layer or indirection and complexity as you've mentioned. With regards to caching user specific parts of your page, have you considered template caching [1, 2]? This way you can apply different caching policies to different parts of your pages w/o using varnish and ESI. With that said, why would one choose Varnish + ESI (or reverse proxy in other words) over template caching + memcached/redis/riak? Can someone explain the pro's and con's of both approaches? [1] http://www.makotemplates.org/docs/caching.html http://www.makotemplates.org/docs/caching.html [2] http://jinja.pocoo.org/docs/extensions/ http://jinja.pocoo.org/docs/extensions/
  - slyall 15y ago
    1. A reverse proxy will usually be faster since it doesn't have to execute an "code" (even a cut-down framework). 2. Some sites will be "read-only" and your really can serve everyone with exactly the same content. 3. Legacy codebase/programmers may not be able to use more scalable techniques so a reverse proxy is the best solution.
bbrizzi 15y ago
Anyone know of an equivalent in Apache?
mike-cardwell 15y ago
The few times I've been on the front page of Slashdot it has eclipsed the traffic that I've had from being on the front page of Reddit. Hacker News barely causes a blip.
jbarham 15y ago
Given that I'm currently personally setting up a caching cluster to host www.melbournecup.com (aka "The Race That Stops a Nation"), being able to gracefully handle a huge spike in traffic is something that is very much on my mind! :) The site itself is developed in Django and so far I'm just planning on putting a bunch of Varnish caches (behind a load balancer) in front of the Django server. I'm using the very nice Django Varnish app (https://github.com/justquick/django-varnish https://github.com/justquick/django-varnish) in the Django instance to automatically purge pages from the cache as they're updated. I'm deliberately trying to keep the setup as simple as possible, but the goal is to have a fast site and fresh content. Tips from others who have handled similar traffic loads would be very welcome!
ck2 15y ago
The only way to make wordpress fast and responsive is to bypass wordpress entirely. You don't need to do anything complex - just install wp-super-cache, set a long timeout, and most importantly add the .htaccess rules it generates to bypass wordpress entirely and serve the cached static files directly.
zzzeek 15y ago
Microcaching at the page level is of course a great idea for a dynamic app, but only works if the content being served is identical for all users - in which case why not just use static pages. Oh right, because we only know how to use Wordpress. Ditching wordpress for a static generator should be the preferred route, if possible. (use disqus or similar for comments). The vast universe of truly dynamic apps that we write in Rails or Python or whatever usually have page elements that are specific to the user's session - "Welcome John Smith" and all that (edit: oh i see he mentioned that at the bottom). So page-level caching isn't feasible there, unless like in the case of disqus you're using javascript to inject personalized content from another server. But for a really interactive web application where coarse grained solutions like this aren't feasible, I'm still a proponent of page-component level caching, something you normally do in your app layer, not the web server layer.
eli 15y ago
I think I'd rather just install Varnish than try to reinvent it in my ngnix config. As a bonus you'll get faster serving of static assets too.

346 points by taybenlor 15y ago

Typical dumb web benchmark ricing, (cf. wildly different concurrency parameters). The usage of apachebench in general is unwise to begin with, but if you're going to use it, at least show data for a variety of tests measuring StdDev across various levels of concurrency. Better off using httperf and knowing how to benchmark things. Worse still, you neglected to mention that the 99/100% cases were much much worse in high concurrency than the low concurrency "slower" test. You were hammering a server that was running on the same machine that was generating the load. Wtf? If you're aiming httperf or ab at "localhost", you're motherfucking doing it wrong. Stop benchmarking shit and blogging about it until you get a grasp of statistics. If you have content that is so read-heavy as to be virtually trivial to cache heavily, just use Varnish.

jbyers 15y ago

Nice technique. The number of distinct dynamic pages you expect to get hammered still must be regenerated within that second. With a longer window, some wp-admin or logged-in-user detection, and a third-party comment service, I could see this being a standard nginx wordpress configuration.

mhd 15y ago
What comment systems are you thinking about? With Disqus the rest of the page shouldn't really matter at all, whether it's dynamic, cached, static etc… Depending on one's error tolerance, maybe dynamic configuration would help (check if you're hit heavily and/or check if you're on HN/slashdot/reddit, then modify the config accordingly).
- IanMikutel 15y ago
  What about the Facebook comment system?
  - nl 15y ago
    Same thing. Any Javascript-based comment system should be ok (because the comments are all served from a 3rd party site). It is when you are using internal wordpress comments you need to be careful.

michaelbuckbee 15y ago

This seems pretty similar to using Varnish or Squid as a reverse proxy (though likely easier to setup).

taybenlor 15y ago
Similar sort of approach. A lot less work to set up, especially if you're already running nginx.

deleted 15y ago

[deleted]

dangrossman 15y ago
APC on its own is an opcode cache, not a page/data cache. Did you write your own code to save pages into it and retrieve them? Or is there a cache plugin for WordPress you're using which uses APC as its data store?
- deleted 15y ago
  [deleted]

wheels 15y ago

> This is fine, up until the point where you get on HN and Reddit at the same time Incidentally, you don't actually need much to handle that. Our web server is a wimpy 256 MB VPS and we've had (Wordpress) blog entries hit the front page of HN and Reddit simultaneously and weather the storm without missing a beat. An appropriately setup Apache + Wordpress SuperCache does the trick just fine. (Hint: The default Apache configuration isn't "appropriate".) You're not going to hit anywhere even close to 2k requests per second on the front page of those two.

timdorr 15y ago
It depends on the content. If it's something that will be visited just by HN/Reddit users: Sure, you're not going to get 2k/sec directly from them. But if it's something that's going to get shared/tweeted/retweeted/blogged/reblogged/etc., then you will be looking at huge traffic numbers. It just depends on how viral your content is and how far it spreads. It's the long tail that matters.
- wheels 15y ago
  You've really got to hit the jackpot to get up to 2000/request per second (assuming you don't already have a high traffic site). Assuming about 20 requests per pageview, that's more than a quarter million pageviews per hour. I've had things go moderately viral (thousands of tweets / retweets) and you don't get anywhere near that. Pre-tuning for hitting the jackpot is in most cases going to be premature optimization.
  - seanp2k 15y ago
    ...and you know what they say about premature optimization; "if you write something that people want to read and your site is down, you will look like an idiot" wait, no, wait, that's not right at all. Maybe: "W3 Total Cache will save your site and takes all of 5 minutes to configure" Hmm, nope. Perhaps: "Cloudflare is free and you can probably sweet talk your webhost into setting up the whole thing for you" No, no, no, no. Wait, what were we talking about?
  - 15y ago

mopoke 15y ago

I like this solution and am definitely tempted to give it a go. Anyone got any thoughts on the best way to do this on a page with personalisation? (and this is really simple personalisation - one section of the page changes depending on whether you're logged in or not). My solution would probably be to have the personalised section load as an async request but then you'd need to make sure that the async request can handle the same load as the microcached content. Any other ideas?

splitrocket 15y ago
That's how I've done it in the past. Depending on how many personalized requests you are doing, compared to the resource utilization savings you'll get from caching the rest of your content, you'll probably come out ahead.
_phred 15y ago
You can put any nginx variable into the proxy_cache_key setting, including cookies and query strings. I've used this to cache localized versions of each page on a site based off of a "language" cookie. Async is a good approach for what you need, but it's good to know what nginx's caching mechanism can do.

splitrocket 15y ago

I've been doing essentially the same thing with my wordpress install. Went from a few hundred reqs/second on a cheap linode to over 4k reqs/second. (I think the limit was the benchmarking tool, not nginx) I've got the nginx config if anyone is interested.

dbalatero 15y ago
I'd be interested in this, yeah! My GitHub username is @dbalatero if you want to send it via that.
splitrocket 15y ago
Here it is: http://alchemi.st/nginx-wordpress-network-and-fastcgi-cache-the-ultimate-guide/ http://alchemi.st/nginx-wordpress-network-and-fastcgi-cache-... You can change the caching ttl to meet your needs.

brown9-2 15y ago

Is it fair to compare those ab benchmarks when the concurrency value is different for each (4 and 500)? Would be curious to see how the original config handled 500 concurrent requests.

splitrocket 15y ago
My experience doing something very similar is that the original config would not handle 500 concurrent requests. It's a fair comparison insofar as the uncached app probably couldn't serve that many concurrent requests in a reasonable amount of time.

patio11 15y ago

A national radio campaign is likely to bring hundreds or thousands of visitors spread over hours, not hundreds of thousands of visitors spread over seconds, so I probably would not take any particular action to harden a site in anticipation of it. It is a poor use of engineering resources and adds technical risk with no corresponding benefit. (n.b. Pasting code you got from a blog post, particularly code marked as kinda broken, is not a risk-free endeavor! I love nginx, don't get me wrong, but paste in snippets from two different blog posts and watch the sparks fly if you don't understand how nginx handles, e.g., location priority.)

adgar 15y ago
Not to mention that he's compromising on accuracy. Yeah, 1 second is tiny, but why give it up if you don't have to? Especially those of you reading who want to try this on a potentially more dynamic site. Edit: downvoted with no explanation, cute. And readers here think they're above Reddit standards. There's no reason to compromise the integrity of a system to any extent before you have to. As others have demonstrated, that need was clearly not met here.
- maratd 15y ago
  > Not to mention that he's compromising on accuracy. Yeah, 1 second is tiny, but why give it up if you don't have to? While NGINX may use very little CPU and have minimal hard drive access, PHP (or Java/Python/Ruby/etc.) will hit those resources hard. Look, I'm not one of those environmentalist wackos who pisses his pants when somebody forgets to turn off a light ... but ... if you're keeping your BLOG server a hundred times busier than it has to be just so that your users don't experience a staleness of 1 second ... you're an asshole. A military or medical application, fine. But a blog? Sorry, no.
  - scott_s 15y ago
    you're an asshole. Not necessary.
    - 15y ago

crikli 15y ago

Is this using nginx in lieu of Apache or nginx in front of apache, acting as a reverse proxy or similar?

nirvana 15y ago

"If you have personalized pages (ie: majority logged-in users) this approach isn't going to work. " I've considered this problem, and am working on a solution for nirvana[1]. The biggest challenge to this project has been to take a language (coffeescript) that is sequential and run it in a distributed environment, without the programmer having to know distributed programming. One of the techniques I'm applying is making a response (in this case, a web page) the result of a collection of components, which are rendered separately in the same context. (EG: The context is the headers of the request, plus the user record if the user is logged in, etc.) So, the request comes in, the components are loaded from the cache, they are executed (in parallel) all with a copy of the state, their results are aggregated and that result can run thru templating to produce a webpage that is returned. The idea then becomes, instead of executing the code for every component in every request, if the component has no context specific requirements (e.g.: it is the same for every user, it's a static element, or it's dynamic, and but doesn't need to be generated every time) .. then it can be flagged as cacheable. The caching would also have a staleness factor (Eg: 1m, 5m 10m).[2] My hope is that you can have pages that are custom per user, but that also contain heavy impact results (say a graph produced by an expensive operation), where the results come form cache, the static components come from cache, but the user specific parts are dynamically generated each request. This component approach not only lets the code be rendered in parallel, and often not even rendered, but instead pulled from cache, but it should allow for more convenient re-use of common elements and features across a site. I hadn't considered caching for just 1s, though. Will have to think about that. [1] Nirvana is CoffeeScript web development backed by erlang and Riak. Instantly distributed coffeescript. It will be open source, hopefully soon. Follow @nirvanacore on Twitter if you're interested in being notified. [2] Planned. There are some implications of this that will require tradeoffs, so initially it may just be a flag of Yes/No for "Cache for up to 1 minute." or some value like that.

aidenn0 15y ago
Something I have done in the past is put all the "loaded by everyone but not static" stuff in memcached. The application pushes to memcached on startup, and any POST that would change it causes the application backend to update the memcached entry(ies). This is a slightly different approach from the microcaching, but it has the advantage of being always consistent. I only do this when the stuff starts to obviously become a bottleneck, and I haven't done much with cloud hosting, where RAM seems to be more expensive than cpu cycles, so I don't know how well it would work there.

rymedia 15y ago

Very impressive.

simonw 15y ago

I really like the idea of setting a cookie that bypasses caching for a few seconds - I've heard the same technique used by Facebook, who set a cookie that ties you to the MySQL master server rather than the slave after you perform a write action so that you'll see your update without waiting for replication lag. It relies on using the Max-Age Cookie argument though, and I was under the impression that IE doesn't implement that correctly. Anyone know what the status of IE and max-age cookies is?

Fenn 15y ago
Hey Simon, Hmmm, you're right about the IE issues (http://mrcoles.com/blog/cookies-max-age-vs-expires/ http://mrcoles.com/blog/cookies-max-age-vs-expires/). I haven't looked into how possible/hard expires would be in nginx config, but you could probably hack it in with http://wiki.nginx.org/HttpUserIdModule http://wiki.nginx.org/HttpUserIdModule
- simonw 15y ago
  The problem with the expires argument is that you have to set it to a specific date, but I don't think you can be sure what timezone / clock setting the user's browser has - which means that it's pretty much impossible to reliably set a cookie that expires in a couple of seconds time. Hopefully I'm wrong - I'd love to know if there's a workaround for that issue.

todsul 15y ago

The big issue here is the application type. This only works well for a very particular type of application. That is, a highly dynamic site that is NOT dependent on user logins. 1) If the site is only moderately dynamic, you can just use plain Nginx and set fastcgi_cache to a few minutes or hours. Much less load on the server. I like to keep things simple, I wouldn't even bother with Apache. Porting rewrites to Nginx is super simple. 2) If the site is customised on a per-user basis, 'microcaching' will break the site and have disastrous consequences. Every user will see the system customised for whichever user primed the cache. My primary website is user based. That means this 'microcaching' concept wont work at all. It would be catastrophic. That's where Varnish comes in with ESI. I really wish I didn't have to use Varnish. It's slower than Nginx, it adds another layer of complexity, and in testing, it seems slightly flaky. But what Varnish+ESI allows is caching of parts of my page that aren't user specific. I.e. header, footer, etc. If you want to see my test results of Nginx vs Varnish, see http://todsul.com/nginx-varnish http://todsul.com/nginx-varnish

chlee 15y ago
I thought about incorporating Varnish into my side-project as well. However, Varnish seems to be a [potential] hassle when you account for VCL, ESI, and etc. It also adds an extra layer or indirection and complexity as you've mentioned. With regards to caching user specific parts of your page, have you considered template caching [1, 2]? This way you can apply different caching policies to different parts of your pages w/o using varnish and ESI. With that said, why would one choose Varnish + ESI (or reverse proxy in other words) over template caching + memcached/redis/riak? Can someone explain the pro's and con's of both approaches? [1] http://www.makotemplates.org/docs/caching.html http://www.makotemplates.org/docs/caching.html [2] http://jinja.pocoo.org/docs/extensions/ http://jinja.pocoo.org/docs/extensions/
- slyall 15y ago
  1. A reverse proxy will usually be faster since it doesn't have to execute an "code" (even a cut-down framework). 2. Some sites will be "read-only" and your really can serve everyone with exactly the same content. 3. Legacy codebase/programmers may not be able to use more scalable techniques so a reverse proxy is the best solution.

bbrizzi 15y ago

Anyone know of an equivalent in Apache?

mike-cardwell 15y ago

The few times I've been on the front page of Slashdot it has eclipsed the traffic that I've had from being on the front page of Reddit. Hacker News barely causes a blip.

jbarham 15y ago

Given that I'm currently personally setting up a caching cluster to host www.melbournecup.com (aka "The Race That Stops a Nation"), being able to gracefully handle a huge spike in traffic is something that is very much on my mind! :) The site itself is developed in Django and so far I'm just planning on putting a bunch of Varnish caches (behind a load balancer) in front of the Django server. I'm using the very nice Django Varnish app (https://github.com/justquick/django-varnish https://github.com/justquick/django-varnish) in the Django instance to automatically purge pages from the cache as they're updated. I'm deliberately trying to keep the setup as simple as possible, but the goal is to have a fast site and fresh content. Tips from others who have handled similar traffic loads would be very welcome!

ck2 15y ago

The only way to make wordpress fast and responsive is to bypass wordpress entirely. You don't need to do anything complex - just install wp-super-cache, set a long timeout, and most importantly add the .htaccess rules it generates to bypass wordpress entirely and serve the cached static files directly.

zzzeek 15y ago

Microcaching at the page level is of course a great idea for a dynamic app, but only works if the content being served is identical for all users - in which case why not just use static pages. Oh right, because we only know how to use Wordpress. Ditching wordpress for a static generator should be the preferred route, if possible. (use disqus or similar for comments). The vast universe of truly dynamic apps that we write in Rails or Python or whatever usually have page elements that are specific to the user's session - "Welcome John Smith" and all that (edit: oh i see he mentioned that at the bottom). So page-level caching isn't feasible there, unless like in the case of disqus you're using javascript to inject personalized content from another server. But for a really interactive web application where coarse grained solutions like this aren't feasible, I'm still a proponent of page-component level caching, something you normally do in your app layer, not the web server layer.

eli 15y ago

I think I'd rather just install Varnish than try to reinvent it in my ngnix config. As a bonus you'll get faster serving of static assets too.