6 ms·

Tell HN: Have a look at Varnish

26 points by eisokant 16y ago

Hey Everyone! I just wanted to share with you the link to an incredible HTTP accelerator: http://varnish-cache.org/ We've been testing a lot of different stacks for our startup (Tyba) and using NGINX (behind) + Varnish (upfront) has allowed us to go from 9 reqs/sec (loading a Wordpress site) to 2500 reqs/sec. I know it's not a novelty but I wanted to share it in the hope it helps the few people who don't know about it. Kind regards, Eiso

d_r 16y ago
More context on why Varnish is superior to Squid: http://varnish-cache.org/wiki/ArchitectNotes http://varnish-cache.org/wiki/ArchitectNotes
- jakevoytko 16y ago
  This article should be required reading for anyone building programs with massive memory consumption! To give some context, this describes how Varnish combines mmapping with a nice memory partitioning scheme to get free caching from the OS, instead of fighting the OS' tendency to cache and managing it yourself.
- sgt 16y ago
  Varnish = reverse proxy Squid = forwarding proxy
  - ggbaker 16y ago
    Squid can also be used as a reverse proxy, so it is a reasonable competitor to Varnish. (As is Apache mod_proxy.)
jacquesm 16y ago
Agreed, varnish is awesome, we use it on a home-brew CDN and it has been a fantastic experience so far. Also, the guy that wrote it (Poul Henning-Kamp) is very responsive when it comes to answering questions or fixing issues, they have an IRC channel at #varnish on irc.linpro.no. Definitely a piece of software worth looking in to when your website gets bigger.
- Andys 16y ago
  PHK has a history of smash hits; he is responsible for much of the work on FreeBSD's container-based virtualisation (jails), modular disk device framework (Geom), & disk encryption (gbde).
jessor 16y ago
Agreed, nginx+varnish (optionally on top of an existing apache setup) seems to be best practice right now for heavy traffic sites.
- piotrSikora 16y ago
  This might be most popular setup right now, but I wouldn't consider it the best practice. Why in the world would you use both nginx and varnish?
  - zepolen 16y ago
    Because until 0.7 nginx lacked a caching system, and so it made sense to use both. Now it doesn't.
    - bjpirt 16y ago
      And even now there's no way of manually expiring a cached page without using a third party unsupported plugin (IIRC) I still love Nginx though :-)
      - piotrSikora 16y ago
        I'm the author of ngx_cache_purge [1]. Would you mind explaining why do you think that it's "unsupported"? This module is fully maintained, it works with every recent nginx release and supports purging content from cache of every upstream supported by nginx (FastCGI, proxy, SCGI, uWSGI). [1] http://labs.frickle.com/nginx_ngx_cache_purge/ http://labs.frickle.com/nginx_ngx_cache_purge/
  - moe 16y ago
    Because varnish is more flexible (VCL) and more time tested. It's also likely[1] faster and more well behaved under high loads, simply because it has been around for much longer and seen more tuning in large deployments. [1] Disclaimer: I haven't benchmarked them against one another, that's merely my common-sense assessment.
zepolen 16y ago
Why would Varnish be required in that situation if you're already running nginx which is perfectly capable of caching a simple wordpress site in the same manner.
- StavrosK 16y ago
  I don't think it's required, but, really, why wouldn't you be running varnish? It's just amazing.
  - piotrSikora 16y ago
    Because it doesn't add anything useful to the mix - nginx's cache works just fine.
    - mrkurt 16y ago
      Sure it does, it adds ESI (which is fantastic). It also keeps backend connections open rather can creating/destroying them for each request. Varnish does restarts nicely too, we have requests that get passed on to another group of servers if the backend returns errors. It's really a very nice way of brokering http requests.
      - zepolen 16y ago
        I think the point is 'in this situation'. Of course there are places where Varnish's features are useful, just not here.
DavidPP 16y ago
For people who want to use NGINX caching on top of Apache/Wordpress, I found out this tutorial : http://www.myatus.co.uk/2010/06/28/a-simplified-nginx-apache-combo-with-wordpress-support/ http://www.myatus.co.uk/2010/06/28/a-simplified-nginx-apache...
japanesejay 16y ago
Thanks for sharing! We've built a drupal powered site using Varnish as well. With a few tweaks, our initial tests showed ~3000 req/sec! Good stuff!

26 points by eisokant 16y ago

d_r 16y ago

More context on why Varnish is superior to Squid: http://varnish-cache.org/wiki/ArchitectNotes http://varnish-cache.org/wiki/ArchitectNotes

jakevoytko 16y ago
This article should be required reading for anyone building programs with massive memory consumption! To give some context, this describes how Varnish combines mmapping with a nice memory partitioning scheme to get free caching from the OS, instead of fighting the OS' tendency to cache and managing it yourself.
sgt 16y ago
Varnish = reverse proxy Squid = forwarding proxy
- ggbaker 16y ago
  Squid can also be used as a reverse proxy, so it is a reasonable competitor to Varnish. (As is Apache mod_proxy.)

jacquesm 16y ago

Agreed, varnish is awesome, we use it on a home-brew CDN and it has been a fantastic experience so far. Also, the guy that wrote it (Poul Henning-Kamp) is very responsive when it comes to answering questions or fixing issues, they have an IRC channel at #varnish on irc.linpro.no. Definitely a piece of software worth looking in to when your website gets bigger.

Andys 16y ago
PHK has a history of smash hits; he is responsible for much of the work on FreeBSD's container-based virtualisation (jails), modular disk device framework (Geom), & disk encryption (gbde).

jessor 16y ago

Agreed, nginx+varnish (optionally on top of an existing apache setup) seems to be best practice right now for heavy traffic sites.

piotrSikora 16y ago
This might be most popular setup right now, but I wouldn't consider it the best practice. Why in the world would you use both nginx and varnish?
- zepolen 16y ago
  Because until 0.7 nginx lacked a caching system, and so it made sense to use both. Now it doesn't.
  - bjpirt 16y ago
    And even now there's no way of manually expiring a cached page without using a third party unsupported plugin (IIRC) I still love Nginx though :-)
    - piotrSikora 16y ago
      I'm the author of ngx_cache_purge [1]. Would you mind explaining why do you think that it's "unsupported"? This module is fully maintained, it works with every recent nginx release and supports purging content from cache of every upstream supported by nginx (FastCGI, proxy, SCGI, uWSGI). [1] http://labs.frickle.com/nginx_ngx_cache_purge/ http://labs.frickle.com/nginx_ngx_cache_purge/
- moe 16y ago
  Because varnish is more flexible (VCL) and more time tested. It's also likely[1] faster and more well behaved under high loads, simply because it has been around for much longer and seen more tuning in large deployments. [1] Disclaimer: I haven't benchmarked them against one another, that's merely my common-sense assessment.

zepolen 16y ago

Why would Varnish be required in that situation if you're already running nginx which is perfectly capable of caching a simple wordpress site in the same manner.

StavrosK 16y ago
I don't think it's required, but, really, why wouldn't you be running varnish? It's just amazing.
- piotrSikora 16y ago
  Because it doesn't add anything useful to the mix - nginx's cache works just fine.
  - mrkurt 16y ago
    Sure it does, it adds ESI (which is fantastic). It also keeps backend connections open rather can creating/destroying them for each request. Varnish does restarts nicely too, we have requests that get passed on to another group of servers if the backend returns errors. It's really a very nice way of brokering http requests.
    - zepolen 16y ago
      I think the point is 'in this situation'. Of course there are places where Varnish's features are useful, just not here.

DavidPP 16y ago

For people who want to use NGINX caching on top of Apache/Wordpress, I found out this tutorial : http://www.myatus.co.uk/2010/06/28/a-simplified-nginx-apache-combo-with-wordpress-support/ http://www.myatus.co.uk/2010/06/28/a-simplified-nginx-apache...

japanesejay 16y ago

Thanks for sharing! We've built a drupal powered site using Varnish as well. With a few tweaks, our initial tests showed ~3000 req/sec! Good stuff!