5 ms·

I know you're being facetious and I chuckled, but it's pretty amazing to see sites fall over under even a moderate load in 2021. A simple Go web server running

I know you're being facetious and I chuckled, but it's pretty amazing to see sites fall over under even a moderate load in 2021. A simple Go web server running on a pretty low-end machine can handle an HN-sized torrent with ease, even before sticking a CDN in front of it.

gtirloni 5y ago
It's usually a WordPress/Joomla/Magento website that can do 1-5 requests/second at most. EDIT: I'm not saying these softwares can't do more, it's just that usually on most default configurations, people don't bother to use a CDN, optimize the database, use a caching plugin, etc. They just install it, then install another 39 plugins and then ask why everything is slow. It's very common to see WordPress websites failing under load (and I've helped many to optimize their installs so I know it's possible, it's just not what the average WordPress install looks like).
- briffle 5y ago
  In this case, its appears to be their main web-page as well.
- amerkhalid 5y ago
  I used to run WordPress site on $5 vps. It could easily handle 100-500 request per second, the trick was to have a caching plugin.
  - tuananh 5y ago
    Agreed. With a very aggressive caching setup, WP can actually handle it pretty well. I used to have a 5mil pageviews per month site running on 5$ instance from DigitalOcean
  - manquer 5y ago
    For 4 years I had Static site generated in Hugo git/CI deployed to CloudFront/S3. Then the team wanted to move and moved to hubspot, it was simply harder to find and train marketing talent who is ready to learn git or even HTML. Far easier to find hubspot/WordPress designer/developers/content writers . Also it becomes easy to blame on the unfamiliar thing your company uses when things don't work for them the way they want it to. I have known teams move to back to very simple default wordpress hosting from more advanced stacks. My point is it harder to get your marketing dept to upgrade tech even if we could do it. I have learnt over the years that solutions that have to be for the people who will use everyday, even if it's poorly managed WordPress. In your example You or your SRE/devops will do all the basic configuration tunning pretty much out of the box. Your IT dept may not be able to at all or unless someone tells them to
    - lostcolony 5y ago
      So one thing I've seen very little of, but have written (to great success), is basically a wordpress/etc scraper. I was lucky in that the site was fully crawlable, but it meant that the non-technical website editors could edit things easily using a CMS, and then when they were ready, someone could deploy to prod. A CI pipeline would then crawl the site, saving all the generated static files, diff them, and upload only the parts that had changed. It meant that we -never- had naked requests hitting the underlying CMS, so our security footprint there was miniscule (to access it you'd need to already be in the network), there was no chance of failure (i.e., even with a caching layer, a bunch of updates and enough traffic could lead to a spike hitting the CMS, and in fact, if there was a bad edit in the CMS that lead to part of the site being malformed, the crawler would die, alerting us to the bad edits before going live), and we could host using any static site/CDN we wanted to, with the only downside being deploys took a while. Even that downside we could have worked around, if we'd wanted to get into the CMS internals enough to track modifications; we just really didn't want to.
      - manquer 5y ago
        It is easier today, Netlify like apps offer the best of both parts ,integrate SSG to graphical UI. You can add CI pipelines for things like say W3C html validation, translation or WCAG compliance or image/asset optimization (designers sometimes forget), AMP generation, S3/CDN like hosting etc. I was tempted to building something like like you did, but came to the same conclusion many do : it is not our core business and marketing will still have issues as the offering won't be as good as a professional one. Even a solution to bridge the two setups needs dev time to maintain. End of the day managed solutions like Hubspot comes in cheaper in total cost of ownership even though all of this kind of architecture is technically superior.
        gtirloni 5y ago
        That's what we did here. Moved from WordPress to Netlify CMS and Eleventy and users managed fine (I'm told).
- arcturus17 5y ago
  I dislike Wordpress as much as the next guy but a modest WP site with a cache or a free tier CDN in front could eat that kind of traffic for breakfast…
  - lostcolony 5y ago
    'with a cache or a free tier CDN in front' Awfully presumptive there aren't you. :P I've had software solutions delivered by consultants in past jobs that were billed as being 'complete', that were missing that. 10 rps = Django fell over, site is down. That was the first thing we fixed.
darkwater 5y ago
I guess that a default, basic WordPress install with no optimization cannot handle the HN hug of death, even in 2021
- arcturus17 5y ago
  “Optimization” would literally mean installing a cache plugin in 2 minutes. I’m no WP expert but maybe they have a good reason to leave caching out of the core.
  - DougWebb 5y ago
    I'm not a WP expert either, but I did implement a CMS for my company. Caching interferes with people editing the site and getting to see the updates immediately, and the easy way out of that is to disable caching. My guess is that default WP takes the easy way out, because of their largely untrained userbase.
    - tyingq 5y ago
      That's interesting. Most of the WP cache plugins I've seen handle that by bypassing the cache for logged in users.
    - deleted 5y ago
      [deleted]
  - caeril 5y ago
    Plenty of organizations in which the marketing or PR department maintains the company blog, not their core IT staff. This is likely their "good reason". Unpatched WP instances full of vulns are also rampant for this reason.
    - oblio 5y ago
      OP meant the Wordpress devs should at this point include the caching plugin in the core install and enable it by default.
aequitas 5y ago
Most of the times the server is not the problem, the database is. And those scale a lot harder if you don't have a proper design for reads, writes and caching. Business wants to see the typo fix to their blogpost online the very second they make it.
tyingq 5y ago
Cloudflare is in front of the blog, but they seem to have purposefully disabled caching both in the browser via Cache-Control headers[1], and apparently in Cloudflare too[2]. [1] See, for example, https://ably.com/support https://ably.com/support , which is currently still up. [2] The cf-cache-status header says "DYNAMIC", which means "This resource is not cached by default and there are no explicit settings configured to cache it."
- handrous 5y ago
  IIRC CloudFlare doesn't cache HTML by default, so if your HTML is dynamically generated (and not, at the very least, cached on your own server/application/WordPress) and you don't enable HTML caching on CF, you can still easily run into trouble. I'm pretty sure cf-cache-status is added by CF—that's not the site saying not to cache it, it's CF reporting that it's not cached (which, again, I think is the default, not something the site owner deliberately turned off).
  - PaywallBuster 5y ago
    right so Need to use a "Page Rule" telling CF to cache everything, and provide the required Cache-Control header values from app side. It's not an option in the caching settings (or need a enterprise plan?)
    - handrous 5y ago
      It's a page rule, available at every plan level (I just checked the docs). Might be a problem if you've already used up your quota of page rules at a given level, but otherwise, no big deal. Enterprise is largely about very high levels of bandwidth use, serving non-Webpage traffic, and all kinds of advanced networking features. It's also a hell of a jump in price from the $200/m top-tier "self-serve" plan (which is also the only self-serve plan with any SLA whatsoever, so, the only one any business beyond the "just testing the waters" phase should be on)
      - PaywallBuster 5y ago
darksaints 5y ago
Got curious and started looking around and found a few blog posts about being on HN front page, and it appears like you might get a traffic spike of 15-50k hits over the course of the day. That is a ridiculously low standard...basic WordPress with a caching plugin on a free tier AWS instance or raspberry pi can handle that sort of volume. If you're going to make a big deal about being contrarian with your technological choices, you either have to be really good at what you do, or be really honest about what tradeoffs you're making. I'd be extremely embarrassed if I pushed this blog post out.
- gonzo41 5y ago
  Totally agree, but hold out. The contrite follow-up blog post about why they switched to K8's will be a great read.
- leephillips 5y ago
  I’ve had a few pieces on my personal site hit the front page here, and there was no effect on site behavior at all. Just Apache on a very modest VPS, with some non-essential dynamic content handled by Django. I don’t notice the traffic spikes until I look at my server logs the next morning.
- nindalf 5y ago
  My thumb rule of how many hits a site on HN gets is roughly 100x upvotes. So 100 upvotes = 10k clicks over 8-10 hours. This is from my experience with 3 links that hit the front page with between 150-420 upvotes. I was serving static HTML off a $5 VPN and later off Cloudflare Pages. The response times were stable in both cases for all users.
KingOfCoders 5y ago
Yes, just use Netlify or something similar.
ericb 5y ago
A well configured web server easily scales really well. Now, application servers and database tiers, that's a different story. Epic novel, really.
kitd 5y ago
It may have been posted elsewhere, ofc, not just HN.

gtirloni 5y ago

It's usually a WordPress/Joomla/Magento website that can do 1-5 requests/second at most. EDIT: I'm not saying these softwares can't do more, it's just that usually on most default configurations, people don't bother to use a CDN, optimize the database, use a caching plugin, etc. They just install it, then install another 39 plugins and then ask why everything is slow. It's very common to see WordPress websites failing under load (and I've helped many to optimize their installs so I know it's possible, it's just not what the average WordPress install looks like).

briffle 5y ago
In this case, its appears to be their main web-page as well.
amerkhalid 5y ago
I used to run WordPress site on $5 vps. It could easily handle 100-500 request per second, the trick was to have a caching plugin.
- tuananh 5y ago
  Agreed. With a very aggressive caching setup, WP can actually handle it pretty well. I used to have a 5mil pageviews per month site running on 5$ instance from DigitalOcean
- manquer 5y ago
  For 4 years I had Static site generated in Hugo git/CI deployed to CloudFront/S3. Then the team wanted to move and moved to hubspot, it was simply harder to find and train marketing talent who is ready to learn git or even HTML. Far easier to find hubspot/WordPress designer/developers/content writers . Also it becomes easy to blame on the unfamiliar thing your company uses when things don't work for them the way they want it to. I have known teams move to back to very simple default wordpress hosting from more advanced stacks. My point is it harder to get your marketing dept to upgrade tech even if we could do it. I have learnt over the years that solutions that have to be for the people who will use everyday, even if it's poorly managed WordPress. In your example You or your SRE/devops will do all the basic configuration tunning pretty much out of the box. Your IT dept may not be able to at all or unless someone tells them to
  - lostcolony 5y ago
    So one thing I've seen very little of, but have written (to great success), is basically a wordpress/etc scraper. I was lucky in that the site was fully crawlable, but it meant that the non-technical website editors could edit things easily using a CMS, and then when they were ready, someone could deploy to prod. A CI pipeline would then crawl the site, saving all the generated static files, diff them, and upload only the parts that had changed. It meant that we -never- had naked requests hitting the underlying CMS, so our security footprint there was miniscule (to access it you'd need to already be in the network), there was no chance of failure (i.e., even with a caching layer, a bunch of updates and enough traffic could lead to a spike hitting the CMS, and in fact, if there was a bad edit in the CMS that lead to part of the site being malformed, the crawler would die, alerting us to the bad edits before going live), and we could host using any static site/CDN we wanted to, with the only downside being deploys took a while. Even that downside we could have worked around, if we'd wanted to get into the CMS internals enough to track modifications; we just really didn't want to.
    - manquer 5y ago
      It is easier today, Netlify like apps offer the best of both parts ,integrate SSG to graphical UI. You can add CI pipelines for things like say W3C html validation, translation or WCAG compliance or image/asset optimization (designers sometimes forget), AMP generation, S3/CDN like hosting etc. I was tempted to building something like like you did, but came to the same conclusion many do : it is not our core business and marketing will still have issues as the offering won't be as good as a professional one. Even a solution to bridge the two setups needs dev time to maintain. End of the day managed solutions like Hubspot comes in cheaper in total cost of ownership even though all of this kind of architecture is technically superior.
      - gtirloni 5y ago
        That's what we did here. Moved from WordPress to Netlify CMS and Eleventy and users managed fine (I'm told).
arcturus17 5y ago
I dislike Wordpress as much as the next guy but a modest WP site with a cache or a free tier CDN in front could eat that kind of traffic for breakfast…
- lostcolony 5y ago
  'with a cache or a free tier CDN in front' Awfully presumptive there aren't you. :P I've had software solutions delivered by consultants in past jobs that were billed as being 'complete', that were missing that. 10 rps = Django fell over, site is down. That was the first thing we fixed.

darkwater 5y ago

I guess that a default, basic WordPress install with no optimization cannot handle the HN hug of death, even in 2021

arcturus17 5y ago
“Optimization” would literally mean installing a cache plugin in 2 minutes. I’m no WP expert but maybe they have a good reason to leave caching out of the core.
- DougWebb 5y ago
  I'm not a WP expert either, but I did implement a CMS for my company. Caching interferes with people editing the site and getting to see the updates immediately, and the easy way out of that is to disable caching. My guess is that default WP takes the easy way out, because of their largely untrained userbase.
  - tyingq 5y ago
    That's interesting. Most of the WP cache plugins I've seen handle that by bypassing the cache for logged in users.
  - deleted 5y ago
    [deleted]
- caeril 5y ago
  Plenty of organizations in which the marketing or PR department maintains the company blog, not their core IT staff. This is likely their "good reason". Unpatched WP instances full of vulns are also rampant for this reason.
  - oblio 5y ago
    OP meant the Wordpress devs should at this point include the caching plugin in the core install and enable it by default.

aequitas 5y ago

Most of the times the server is not the problem, the database is. And those scale a lot harder if you don't have a proper design for reads, writes and caching. Business wants to see the typo fix to their blogpost online the very second they make it.

tyingq 5y ago

Cloudflare is in front of the blog, but they seem to have purposefully disabled caching both in the browser via Cache-Control headers[1], and apparently in Cloudflare too[2]. [1] See, for example, https://ably.com/support https://ably.com/support , which is currently still up. [2] The cf-cache-status header says "DYNAMIC", which means "This resource is not cached by default and there are no explicit settings configured to cache it."

handrous 5y ago
IIRC CloudFlare doesn't cache HTML by default, so if your HTML is dynamically generated (and not, at the very least, cached on your own server/application/WordPress) and you don't enable HTML caching on CF, you can still easily run into trouble. I'm pretty sure cf-cache-status is added by CF—that's not the site saying not to cache it, it's CF reporting that it's not cached (which, again, I think is the default, not something the site owner deliberately turned off).
- PaywallBuster 5y ago
  right so Need to use a "Page Rule" telling CF to cache everything, and provide the required Cache-Control header values from app side. It's not an option in the caching settings (or need a enterprise plan?)
  - handrous 5y ago
    It's a page rule, available at every plan level (I just checked the docs). Might be a problem if you've already used up your quota of page rules at a given level, but otherwise, no big deal. Enterprise is largely about very high levels of bandwidth use, serving non-Webpage traffic, and all kinds of advanced networking features. It's also a hell of a jump in price from the $200/m top-tier "self-serve" plan (which is also the only self-serve plan with any SLA whatsoever, so, the only one any business beyond the "just testing the waters" phase should be on)
    - PaywallBuster 5y ago

darksaints 5y ago

Got curious and started looking around and found a few blog posts about being on HN front page, and it appears like you might get a traffic spike of 15-50k hits over the course of the day. That is a ridiculously low standard...basic WordPress with a caching plugin on a free tier AWS instance or raspberry pi can handle that sort of volume. If you're going to make a big deal about being contrarian with your technological choices, you either have to be really good at what you do, or be really honest about what tradeoffs you're making. I'd be extremely embarrassed if I pushed this blog post out.

gonzo41 5y ago
Totally agree, but hold out. The contrite follow-up blog post about why they switched to K8's will be a great read.
leephillips 5y ago
I’ve had a few pieces on my personal site hit the front page here, and there was no effect on site behavior at all. Just Apache on a very modest VPS, with some non-essential dynamic content handled by Django. I don’t notice the traffic spikes until I look at my server logs the next morning.
nindalf 5y ago
My thumb rule of how many hits a site on HN gets is roughly 100x upvotes. So 100 upvotes = 10k clicks over 8-10 hours. This is from my experience with 3 links that hit the front page with between 150-420 upvotes. I was serving static HTML off a $5 VPN and later off Cloudflare Pages. The response times were stable in both cases for all users.

KingOfCoders 5y ago

Yes, just use Netlify or something similar.

ericb 5y ago

A well configured web server easily scales really well. Now, application servers and database tiers, that's a different story. Epic novel, really.

kitd 5y ago

It may have been posted elsewhere, ofc, not just HN.