3 ms·

How are you measuring this lost ground? Varnish is still extremely popular, and designs which separate TLS from caching are also still popular.

tyingq 5y ago
Mostly anecdotal, since there's no real definitive way to tell. There's two spaces I see. - Caching within the webserver, as the caching there has improved over time. The caches in Nginx, Apache, LiteSpeed, etc, perform much better than they did in the past. - Caching at a load balancer tier. Nginx again (though in a LB context), Haproxy, etc. Both spaces seem to have less talk about Varnish than they used to, and more about other platforms. You can see a plateau, then decline (though slight) that starts for Varnish around mid-2018, here: https://trends.builtwith.com/Web-Server/Varnish https://trends.builtwith.com/Web-Server/Varnish Compare to: LiteSpeed: https://trends.builtwith.com/Web-Server/LiteSpeed https://trends.builtwith.com/Web-Server/LiteSpeed Nginx: https://trends.builtwith.com/Web-Server/nginx https://trends.builtwith.com/Web-Server/nginx (Though these kinds of surveys are tricky, as they depend on outward facing headers that don't always exist, or don't always tell you enough...like Nginx doesn't always imply the cache is in use) I also wonder what percentage hit Varnish might take if Fastly moves away. I'm sure they regret not varnish specifically, but exposing the varnish vcl directly to end users. Edit: Yes, I agree that sites like "builtwith" are flawed, and mentioned some of the reasons above. And, I didn't mean for this comment to sound like a criticism...just an observation. I noticed builtwith's chart has a similar plateau + slight decline for Apache, starting also near mid-2018.
- phkamp 5y ago
  Varnish has a pretty big market-share as "the intelligent HTTP-router" which can be used to sort traffic to piles of legacy webservers etc, and also be the central clearing-house for detecting and fixing trouble. Surveys such as builtwith, despite their hard work, can often not "see through the sandwich" and spot if there is a Varnish in it. Also, I dont know about you, but from a "I want the world to keep working" reliability point of view, I do not like it when any single piece of software, FOSS or non-FOSS, becomes too dominant. See for instance how dysfunctional the GCC-monopoly was until LLVM gave them competition. So taking builtwith at their numbers, I'm actually fine with Varnish having "stagnated" at a market share of one fifth of the worlds top 10K websites: That keeps me humble about my code quality, but does not keep me awake at night.
- acdha 5y ago
  > Though these kinds of surveys are tricky, as they depend on outward facing headers that don't always exist It's not just “don't always exist”: those headers are actively recommended against by various security guidelines so many large sites heavily use things you can only infer from other characteristics. This is also the kind of environment where I see some movement against Varnish: internal TLS requirements increase the cost of managing two services instead of one, and if you're increasingly using something like an external CDN the level of benefit from Varnish's cache declines somewhat even though the powerful request routing and manipulation features are still appealing. I've been generally wondering what it would take to be able to flip the model to something like Cloudflare's Argo Tunnel feature where you could secure internal communications by having your various web services make an _outbound_ connection to the Varnish box which all of the requests will be tunneled over so you only need to manage one certificate there rather than one for every service/container in a complex application.

tyingq 5y ago

Mostly anecdotal, since there's no real definitive way to tell. There's two spaces I see. - Caching within the webserver, as the caching there has improved over time. The caches in Nginx, Apache, LiteSpeed, etc, perform much better than they did in the past. - Caching at a load balancer tier. Nginx again (though in a LB context), Haproxy, etc. Both spaces seem to have less talk about Varnish than they used to, and more about other platforms. You can see a plateau, then decline (though slight) that starts for Varnish around mid-2018, here: https://trends.builtwith.com/Web-Server/Varnish https://trends.builtwith.com/Web-Server/Varnish Compare to: LiteSpeed: https://trends.builtwith.com/Web-Server/LiteSpeed https://trends.builtwith.com/Web-Server/LiteSpeed Nginx: https://trends.builtwith.com/Web-Server/nginx https://trends.builtwith.com/Web-Server/nginx (Though these kinds of surveys are tricky, as they depend on outward facing headers that don't always exist, or don't always tell you enough...like Nginx doesn't always imply the cache is in use) I also wonder what percentage hit Varnish might take if Fastly moves away. I'm sure they regret not varnish specifically, but exposing the varnish vcl directly to end users. Edit: Yes, I agree that sites like "builtwith" are flawed, and mentioned some of the reasons above. And, I didn't mean for this comment to sound like a criticism...just an observation. I noticed builtwith's chart has a similar plateau + slight decline for Apache, starting also near mid-2018.

phkamp 5y ago
Varnish has a pretty big market-share as "the intelligent HTTP-router" which can be used to sort traffic to piles of legacy webservers etc, and also be the central clearing-house for detecting and fixing trouble. Surveys such as builtwith, despite their hard work, can often not "see through the sandwich" and spot if there is a Varnish in it. Also, I dont know about you, but from a "I want the world to keep working" reliability point of view, I do not like it when any single piece of software, FOSS or non-FOSS, becomes too dominant. See for instance how dysfunctional the GCC-monopoly was until LLVM gave them competition. So taking builtwith at their numbers, I'm actually fine with Varnish having "stagnated" at a market share of one fifth of the worlds top 10K websites: That keeps me humble about my code quality, but does not keep me awake at night.
acdha 5y ago
> Though these kinds of surveys are tricky, as they depend on outward facing headers that don't always exist It's not just “don't always exist”: those headers are actively recommended against by various security guidelines so many large sites heavily use things you can only infer from other characteristics. This is also the kind of environment where I see some movement against Varnish: internal TLS requirements increase the cost of managing two services instead of one, and if you're increasingly using something like an external CDN the level of benefit from Varnish's cache declines somewhat even though the powerful request routing and manipulation features are still appealing. I've been generally wondering what it would take to be able to flip the model to something like Cloudflare's Argo Tunnel feature where you could secure internal communications by having your various web services make an _outbound_ connection to the Varnish box which all of the requests will be tunneled over so you only need to manage one certificate there rather than one for every service/container in a complex application.