4 ms·

i had a site with ~ 10m pageviews monthly that used to run on a $20 DigitalOcean instance. What kind of things you need to do to scale that up to 600m PV month

i had a site with ~ 10m pageviews monthly that used to run on a $20 DigitalOcean instance. What kind of things you need to do to scale that up to 600m PV monthly?

blowski 7y ago
Depends on whether the majority of views are on static blog posts, as they can be cached and served from nginx. But if you’ve got a lot of dynamic views - comments, admin, personalised - it’s much harder. Still impressive you managed to get that level of performance out of a $20 DO box.
- toast0 7y ago
  10m page views really doesn't look that big if you bring it down to pageviews/second and there's some reasonable distribution. Assuming a busy day is 1M, and that a busy hour of the busy day has 100,000, but over the hour is evenly distributed, you're looking at under 30 page views per second. Maybe peak is a little more. If this is tuananh's current site, it's Jekyl, so it's all static -- you could probably host that on a 486 (as long as there aren't any big images to clog the nic). If it's something with a little bit of dynamic content, you might need to be careful about how you structure it, but something reasonable should be fine at that level of traffic too, just need to make sure you don't spend a lot of cpu doing useless work (like some frameworks that take 60ms to output hello world)
aussieguy1234 7y ago
WordPress has alot of filters that allow you to take over/modify just about anything including internal functions. We overrode get_posts() to use ElasticSearch instead of MySQL and added lots of caching to alot of things with Memcache. I built a plugin which cached all of the HTML output of every WordPress page and all of the widgets in Memcache. On top of all the server side caching, there was Akamai in front caching every page at the network level.
nullwasamistake 7y ago
Not aussieguy, but cache everything on the network level. If you need anything dynamic, do it SPA style and bypass the WordPress backend. There's no getting around the 50-200 request/sec barrier of the codebase. The most popular sites use it as a fancy static site generator. Write your dynamic stuff in React or Vue (something client side). Then point it at a custom backend built to handle the load. For comments and the like, maybe mongo or elastic with Go/Java. I've only seen a couple sites popular enough to do this. A danger with WordPress is attackers finding a stray dynamic endpoint (not cached) and taking site down by curling or F5. Unless you have a super beefy server, it's possible to kill a site spamming F5 if you find an action not cached. This is a technical problem for all WordPress sites but only a practical one for sites large enough to be a target. Some sites the use sitemap functionality to pregenerate all pages as HTML and serve that. It's the only way to be truly secure with WordPress

i had a site with ~ 10m pageviews monthly that used to run on a $20 DigitalOcean instance. What kind of things you need to do to scale that up to 600m PV monthly?

blowski 7y ago

Depends on whether the majority of views are on static blog posts, as they can be cached and served from nginx. But if you’ve got a lot of dynamic views - comments, admin, personalised - it’s much harder. Still impressive you managed to get that level of performance out of a $20 DO box.

toast0 7y ago
10m page views really doesn't look that big if you bring it down to pageviews/second and there's some reasonable distribution. Assuming a busy day is 1M, and that a busy hour of the busy day has 100,000, but over the hour is evenly distributed, you're looking at under 30 page views per second. Maybe peak is a little more. If this is tuananh's current site, it's Jekyl, so it's all static -- you could probably host that on a 486 (as long as there aren't any big images to clog the nic). If it's something with a little bit of dynamic content, you might need to be careful about how you structure it, but something reasonable should be fine at that level of traffic too, just need to make sure you don't spend a lot of cpu doing useless work (like some frameworks that take 60ms to output hello world)

aussieguy1234 7y ago

WordPress has alot of filters that allow you to take over/modify just about anything including internal functions. We overrode get_posts() to use ElasticSearch instead of MySQL and added lots of caching to alot of things with Memcache. I built a plugin which cached all of the HTML output of every WordPress page and all of the widgets in Memcache. On top of all the server side caching, there was Akamai in front caching every page at the network level.

nullwasamistake 7y ago

Not aussieguy, but cache everything on the network level. If you need anything dynamic, do it SPA style and bypass the WordPress backend. There's no getting around the 50-200 request/sec barrier of the codebase. The most popular sites use it as a fancy static site generator. Write your dynamic stuff in React or Vue (something client side). Then point it at a custom backend built to handle the load. For comments and the like, maybe mongo or elastic with Go/Java. I've only seen a couple sites popular enough to do this. A danger with WordPress is attackers finding a stray dynamic endpoint (not cached) and taking site down by curling or F5. Unless you have a super beefy server, it's possible to kill a site spamming F5 if you find an action not cached. This is a technical problem for all WordPress sites but only a practical one for sites large enough to be a target. Some sites the use sitemap functionality to pregenerate all pages as HTML and serve that. It's the only way to be truly secure with WordPress