8 ms·

Static sites served from the filesystem directly through any reasonable webserver are always going to be quick. 100k hits per month is nothing and a hackernews

Static sites served from the filesystem directly through any reasonable webserver are always going to be quick. 100k hits per month is nothing and a hackernews spike should not even be noticed unless you are serving up video or something large. My site sits on a $5 DO droplet and I have never even come close to hitting any kind of limit despite the occasional traffic spike. Dynamic content management systems are a convenience for the developer/maintainer of the site but push additional requirements onto the hosting CPU (and the client for javascript heavy sites). The cost of these requirements is often not clear but usually much larger than you might expect.

paulddraper 7y ago
Yep. Wordpress will eat your server if you don't configure page caching. At $CORP, the web application went down when some heavy Wordpress traffic slowed down the shared DB :/
- treebornfrog 7y ago
  I'm no programmer but I've used varnish along with nginx, seems to work well.
  - nkozyra 7y ago
    nginx has a "good enough" caching system for most use cases.
  - paulddraper 7y ago
    You need some sort of cache, whether Wordpress-specific or Varnish or CDN or something.
- BiteCode_dev 7y ago
  Yes, althought in the case of wordpress, it's also because it's really badly architectured: no internal cache, so many requests for just one page, comment are loaded even when you don't read them, etc. It's not optized at all.
  - nkozyra 7y ago
    The first time I peeked at wp code I was flabbergasted at how cavalier they were with piles of inefficient queries. At the time I was tasked with evaluating it as a "cheaper" replacement for an expensive, managed CMS. My conclusion was it would end up being more expensive to accommodate all of this inefficiency with caching galore not to mention support and dealing with a loss of dynamic web sites. I looked again last year, assuming it had greatly improved, had replaced all that with GRPC or graphql and nope, still horrendously designed. I wonder if it's intentional or just a case of support for backwards compatibly.
    - stedaniels 7y ago
      Optimising WordPress is a known and solved problem. I don't mean to undermine your comment but it is trivial for a web infrastructure ops guy to deliver on. There is a lot of fud out there on the topic so it's easy not to see the diamonds in the rough.
      - flomei 7y ago
        Totally. Made some good money, doing this for small to medium businesses. Easy work, and lots of quick wins.
      - BiteCode_dev 7y ago
        That's not the point. The point is it's not built it, because the architecture decisions were unsound. Now, don't get me wrong, I use and like Wordpress. There is not more versatile yet approachable blog/CMS/whatever out there, the ecosystem is huge and it's my go to tool when I want a quick and dirty solution. But it doesn't change the fact it has been badly designed from the start. Popularity in software doesn't reward good design. It rewards solving problems.
kijin 7y ago
Yeah, serving static files is unbelievably cheap these days. Some of my clients have nginx serving 1B+ requests per month on a single box, and the CPU doesn't even break a sweat. nginx can easily saturate a gigabit link with very little effort.
- nickjj 7y ago
  What type of hardware is that box running? Would be kind of interesting just to have a point of reference (even if it ends up being super beefy and overkill since you mentioned CPU load is low).
  - zaroth 7y ago
    The point is that the hardware doesn’t matter. RAM is faster than Ethernet. The bulk of the load will be negotiating the TLS sessions, and modern ciphers are both fast and hardware accelerated, and that’s assuming something upstream isn’t already taking care of the TLS.
  - kijin 7y ago
    A run-of-the-mill quad-core Xeon E3 with some SATA SSDs. Even that is overkill as the CPU usage never goes above 1 core. You could probably get the same job done with a $20 droplet if you're frugal, or $40 with room to spare.
dmortin 7y ago
Can't you serve a static site purely from memory and avoid hitting the filesystem altogether? If the site is not huge then all the pages could easily fit in memory and served from a memory cache.
- littlestymaar 7y ago
  When you serve from the filesystem, your content is usually in the filesystem cache in practice, and then it's being served from memory. Adding a memory cache on top of the filesystem cache is probably a waste of time for no gain (unless your workload has really specific pattern maybe…).
  - byroot 7y ago
    To add to this, most webservers will use `sendfile(2)` for static files.
    - littlestymaar 7y ago
      That wouldn't work for https though right? (unless you are behind a reverse proxy of course, but that doesn't fit to the whole discussion).
tjoff 7y ago
> Dynamic content management systems are a convenience for the developer/maintainer It isn't harder to call a script that regenerates the static files. It is just that dynamic content management systems are more common.
- toyg 7y ago
  It’s harder to scale it properly over time, because every new post takes longer to regenerate aggregated pages. Obviously this can be somewhat lightened with some hard choices (e.g. only have aggregated pages by fixed period), but it is a tradeoff that dynamic sites typically don’t have to worry about. Anybody who was around during the first blogwave in the early 2000s knows the score: after a while, with static generators you end up waiting a lot after hitting “publish”. And god help you when you change the template over the whole site. Of course, dynamic systems have their own tradeoffs, but they tend to scale better by volume content. You can then slap a simple cache in front and be done with it until views get in the hundred millions.
  - the_duke 7y ago
    Just think of the immense amount of work a compiler is doing to compile a decently sized C++ or Rust binary. If there was a need for it, static site generators could be scaled to tens or hundreds of thousands of pages and still finish in a few seconds. Caching partial results is always an option. There are generators that are decently performant, like Hugo (Go) [1] or Zola (Rust) [2]. [1] https://gohugo.io/ https://gohugo.io/ [2] https://github.com/getzola/zola https://github.com/getzola/zola
  - AndrewStephens 7y ago
    You are not wrong but it hardly matters for most blogs/product web sites. For example, I generate my blog from markdown with a custom generator written in not-very-optomised Python. There are about 400 pages currently and it only takes about 2 seconds on a 6 year old MacBook to completely build the entire site. Even if I write another 1000 blog posts each year, generating the site will still take less than a minute for the foreseeable future. Paying the cost of building the pages up front like this is still much better than having my users pay it by waiting 10s of seconds from the page to slowly be dredged out of a database. There are sites that do have tens of thousands of pages, and perhaps a content management system is appropriate for them. Horses for courses and all that.

paulddraper 7y ago

Yep. Wordpress will eat your server if you don't configure page caching. At $CORP, the web application went down when some heavy Wordpress traffic slowed down the shared DB :/

treebornfrog 7y ago
I'm no programmer but I've used varnish along with nginx, seems to work well.
- nkozyra 7y ago
  nginx has a "good enough" caching system for most use cases.
- paulddraper 7y ago
  You need some sort of cache, whether Wordpress-specific or Varnish or CDN or something.
BiteCode_dev 7y ago
Yes, althought in the case of wordpress, it's also because it's really badly architectured: no internal cache, so many requests for just one page, comment are loaded even when you don't read them, etc. It's not optized at all.
- nkozyra 7y ago
  The first time I peeked at wp code I was flabbergasted at how cavalier they were with piles of inefficient queries. At the time I was tasked with evaluating it as a "cheaper" replacement for an expensive, managed CMS. My conclusion was it would end up being more expensive to accommodate all of this inefficiency with caching galore not to mention support and dealing with a loss of dynamic web sites. I looked again last year, assuming it had greatly improved, had replaced all that with GRPC or graphql and nope, still horrendously designed. I wonder if it's intentional or just a case of support for backwards compatibly.
  - stedaniels 7y ago
    Optimising WordPress is a known and solved problem. I don't mean to undermine your comment but it is trivial for a web infrastructure ops guy to deliver on. There is a lot of fud out there on the topic so it's easy not to see the diamonds in the rough.
    - flomei 7y ago
      Totally. Made some good money, doing this for small to medium businesses. Easy work, and lots of quick wins.
    - BiteCode_dev 7y ago
      That's not the point. The point is it's not built it, because the architecture decisions were unsound. Now, don't get me wrong, I use and like Wordpress. There is not more versatile yet approachable blog/CMS/whatever out there, the ecosystem is huge and it's my go to tool when I want a quick and dirty solution. But it doesn't change the fact it has been badly designed from the start. Popularity in software doesn't reward good design. It rewards solving problems.

kijin 7y ago

Yeah, serving static files is unbelievably cheap these days. Some of my clients have nginx serving 1B+ requests per month on a single box, and the CPU doesn't even break a sweat. nginx can easily saturate a gigabit link with very little effort.

nickjj 7y ago
What type of hardware is that box running? Would be kind of interesting just to have a point of reference (even if it ends up being super beefy and overkill since you mentioned CPU load is low).
- zaroth 7y ago
  The point is that the hardware doesn’t matter. RAM is faster than Ethernet. The bulk of the load will be negotiating the TLS sessions, and modern ciphers are both fast and hardware accelerated, and that’s assuming something upstream isn’t already taking care of the TLS.
- kijin 7y ago
  A run-of-the-mill quad-core Xeon E3 with some SATA SSDs. Even that is overkill as the CPU usage never goes above 1 core. You could probably get the same job done with a $20 droplet if you're frugal, or $40 with room to spare.

dmortin 7y ago

Can't you serve a static site purely from memory and avoid hitting the filesystem altogether? If the site is not huge then all the pages could easily fit in memory and served from a memory cache.

littlestymaar 7y ago
When you serve from the filesystem, your content is usually in the filesystem cache in practice, and then it's being served from memory. Adding a memory cache on top of the filesystem cache is probably a waste of time for no gain (unless your workload has really specific pattern maybe…).
- byroot 7y ago
  To add to this, most webservers will use `sendfile(2)` for static files.
  - littlestymaar 7y ago
    That wouldn't work for https though right? (unless you are behind a reverse proxy of course, but that doesn't fit to the whole discussion).

tjoff 7y ago

> Dynamic content management systems are a convenience for the developer/maintainer It isn't harder to call a script that regenerates the static files. It is just that dynamic content management systems are more common.

toyg 7y ago
It’s harder to scale it properly over time, because every new post takes longer to regenerate aggregated pages. Obviously this can be somewhat lightened with some hard choices (e.g. only have aggregated pages by fixed period), but it is a tradeoff that dynamic sites typically don’t have to worry about. Anybody who was around during the first blogwave in the early 2000s knows the score: after a while, with static generators you end up waiting a lot after hitting “publish”. And god help you when you change the template over the whole site. Of course, dynamic systems have their own tradeoffs, but they tend to scale better by volume content. You can then slap a simple cache in front and be done with it until views get in the hundred millions.
- the_duke 7y ago
  Just think of the immense amount of work a compiler is doing to compile a decently sized C++ or Rust binary. If there was a need for it, static site generators could be scaled to tens or hundreds of thousands of pages and still finish in a few seconds. Caching partial results is always an option. There are generators that are decently performant, like Hugo (Go) [1] or Zola (Rust) [2]. [1] https://gohugo.io/ https://gohugo.io/ [2] https://github.com/getzola/zola https://github.com/getzola/zola
- AndrewStephens 7y ago
  You are not wrong but it hardly matters for most blogs/product web sites. For example, I generate my blog from markdown with a custom generator written in not-very-optomised Python. There are about 400 pages currently and it only takes about 2 seconds on a 6 year old MacBook to completely build the entire site. Even if I write another 1000 blog posts each year, generating the site will still take less than a minute for the foreseeable future. Paying the cost of building the pages up front like this is still much better than having my users pay it by waiting 10s of seconds from the page to slowly be dredged out of a database. There are sites that do have tens of thousands of pages, and perhaps a content management system is appropriate for them. Horses for courses and all that.