5 ms·

Maybe it's a sign of me getting old, but I have never understood the thought process behind building systems which server static content - newspapers, blogs - w

Maybe it's a sign of me getting old, but I have never understood the thought process behind building systems which server static content - newspapers, blogs - which require active processing for every request for every user. These are systems where the read/write ratios are often 1000:1. Ten years ago this was symptomized by systems which pulled the same content out of a database for every single request. We treated the symptoms by introducing caching servers and horizontal scaling. In modern times the symptom is displayed with heavy use of client-side rendering. Because, you know, this client-side rendering is "free" for the publisher. I don't understand how any "Engineer" can see 1000:1 R:W ratios and suggest doing all of the work on the R side of the ratio.

deleted 11y ago
[deleted]
firasd 11y ago
About static vs. dynamic rendering on the server side, the Movable Type CMS used to do exactly as you suggest and it was difficult to make changes on large sites. Imagine if you want to change a link on a sidebar and have to re-build every HTML page served by your CMS to do that. Caching dynamic pages is a bit of a “hybrid” system where you can just publish your new sidebar, then clear cache on the first request for each particular changed page. About the client side, the situation is a bit different because the JS is actually third-party software designed to be dropped in and updated easily, so it’s not just an implementation detail. Very few publishers are going to build or host their own server-side analytics or commenting systems…
- radicalbyte 11y ago
  You can do that async - have the edit experience use the db then a publish action which re-generated the site. As for monitoring/comment systems: using external sites is actually really good for performance. Third party tools do a great job of blocking them and that makes a lot of sites very fast.
- userbinator 11y ago
  Imagine if you want to change a link on a sidebar and have to re-build every HTML page served by your CMS to do that. On the other hand, doing the processing on the client side means that work would be done every single time by every single client, and the result of that work isn't even preserved. As decades of programmers have had absolutely no problem compiling software with many, many more dependent files than pages on a typical HTML site, I suspect the problem is in the implementation and not the concept. build or host their own server-side analytics or commenting systems I've not written any analytics before so I won't comment on that, but it's funny you mention commenting systems since a simple one (textboard-like) is literally less than a day's work.
  - TeMPOraL 11y ago
    > On the other hand, doing the processing on the client side means that work would be done every single time by every single client, and the result of that work isn't even preserved. Exactly! If that amount of work on server seems big, then realize that doing it client-side means dumping a million[0] times more work on the society, with all associated electricity usage. [0] - literally; if you have 100k visitors and each visited you 10 times, that's 1M times the work that should actually have been done.
- oneeyedpigeon 11y ago
  That's exactly how a site I work on operates. True, changing a link across all pages is a very slow operation (about 50 seconds for 11,000 pages) but a) that's happening in a highly structured and powerful but resource-intensive way (i.e. DOM parsing) b) it happens so rarely that I haven't bothered to optimise it. The main trade-off for an update that happens maybe once every 1-2 months taking 50 seconds is that every single page load is faster. There are other benefits from static files, too - it's much easier to work with content in flat files, 1-per-page, than the kind of complicated database / file / code relationship used by many 'normal' CMSes.
alkonaut 11y ago
It's easy to forget that html isn't the end product, the end result is lit pixels on some screen (usually). If we send json then that needs to be converted to html and that html needs to be layout and rendered. If we send html we just skip the first step. Displaying a web page isn't "free" for the client just because it's html. The question is this: what kind of overhead is a template rendering task (in js) for a modern browser, compared to the task of layout and rendering a finished html page? It will depend of course on the complexity of the data and the resulting page, as well as the browser, templating engine etc. I have no clue whether it's 1%, 10% or 50%.
- firasd 11y ago
  The slowdown caused by JS isn’t due as much to the rendering speed as the HTTP requests.
- TeMPOraL 11y ago
  > It's easy to forget that html isn't the end product, the end result is lit pixels on some screen (usually). If we send json then that needs to be converted to html and that html needs to be layout and rendered. If we send html we just skip the first step. Displaying a web page isn't "free" for the client just because it's html. JSON->HTML rendering may be only one step, but by not skipping it, we're making hundred thousands computers redo the computation that could be done once on a server. This has some electricity cost aggregated over all users, which we externalize to society because of a fad. You can't really skip the HTML->pixels step because of data size issues, but there's little excuse for client-side rendering of static content.
  - moonchrome 11y ago
    Who says you're doing JSON->HTML ? Both HTML and your data format need to get deserialized to DOM - depending on use case it can be faster/smaller to do the deserialization in JS from some format rather than using browser HTML parser. HTML is actually non-trivial to parse and isn't really optimized for anything.
- odiroot 11y ago
onewaystreet 11y ago
Because many publishers found that visitors wont wait the few extra seconds it takes to load the content on the server. They need to see something loading in their browser or they abandon the content.
- TeMPOraL 11y ago
  Then show them an AJAX spinner while fetching the static site in the background.
  - Gigablah 11y ago
    That's what one of the Blogger templates does, and I've seen people on HN complain about having to see an AJAX spinner just to load a blog.
    - TeMPOraL 11y ago
      Yeah, I'm such a complainer too. Blogs are one of the last places you should ever see such a spinner because they're just static content with sidebars. There should be little to none client-side rendering involved. And the spinner on Blogger is only a (very visible) symptom of the bigger problem - the whole site is incredibly slow and resource heavy.
      - pjmlp 11y ago
        I can understand interactive documents like notebooks (IPython and such) and games, for anything else just give me HTML + CSS.
- spacecowboy_lon 11y ago
  I think you mean publishers built a slow site and are bemused by why they don't have any traffic. Its a lack of repsect for techies and quality and not understanding that the web is not a glossy brochure that you can't optimize by shouting at a printer over the phone.
mpweiher 11y ago
Excellent point, I never understood that either, with developers being happy that their supercomputing buzzsaw can generate a couple of 100K of text in about a second. OTOH...it turns out that so-called "static" content actually also requires quite a bit of processing. Apache (or whichever server you are using) is not exactly doing nothing. And it also turns out that this sort of processing is I/O heavy, which isn't exactly the fastest path on the machine, even if all the caching done by various entities (more processing) tries and mostly succeeds in making sure very little I/O actually takes place. Add to that that moving data around nowadays is generally much more expensive than doing computation on it (several instructions per clock-cyle, multi-Gigahertz clocks, so well over a hundred clocks per main memory access, and millions per I/O, even with an SSD). In short: if we re-examine our assumptions in light of a changing environment, might a well-optimized "dynamic" stack not be faster than a "static" one? (Compare: VoltDB...). And of course you could then cache the results in memory as well.
mpweiher 11y ago
Excellent point, I never understood that either, with developers being happy that their supercomputing buzzsaw can generate a couple of 100K of text in about a second. OTOH...it turns out that so-called "static" content actually also requires quite a bit of processing. Apache (or whichever server you are using) is not exactly doing nothing. And it also turns out that this sort of processing is I/O heavy, which isn't exactly the fastest path on the machine, even if all the caching done by various entities (more processing) tries and mostly succeeds in making sure very little I/O actually takes place. Add to that that moving data around nowadays is generally much more expensive than doing computation on it (several instructions per clock-cyle, multi-Gigahertz clocks, so well over a hundred clocks per main memory access, and millions per I/O, even with an SSD). In short: if we re-examine our assumptions in light of a changing environment, might a well-optimized "dynamic" stack not be faster than a "static" one? (Compare: VoltDB...). And of course you could then cache the results in memory as well.
mseebach 11y ago
But the client side rendering isn't static, and it certainly isn't 1000:1 R:W. There are different browsers on different devices, with different resolutions, pixel densities, zoom-levels, minimum font sizes, window-sizes and orientations. Static client side rendering at 1000:1 would mean that you could render an image file once and serve it to (at least) 1000 clients.
- TeMPOraL 11y ago
  Static client side rendering means you render your site as HTML+CSS+utility JS on server once, and send a saved copy to people when asked, as opposed to client-side rendering in which you send them data, a template and JS that will build all HTML in browser.
- creshal 11y ago
  > But the client side rendering isn't static, and it certainly isn't 1000:1 R:W. There are different browsers on different devices, with different resolutions, pixel densities, zoom-levels, minimum font sizes, window-sizes and orientations. Yeees, and? Response CSS designs solve a lot of that with zero server-side computation overhead, and (compared to JS templating/render engines) minimal client-side overhead.
mreiland 11y ago
At one point gamefaqs.com would generate static pages and serve those everywhere it could. This was before cnet bought the site though, I have no idea what they do nowadays, your post just reminded me of it.
- creshal 11y ago
  It doesn't matter much, the 70+ embedded youtube videos some users put into their FAQs will kill your browser either way.
  - mreiland 11y ago
    Can you link to an example of this? I've been on gamefaqs since the very beginning and I've never seen that. In fact, the vast majority of FAQs on that site are still text.
    - creshal 11y ago
      One of the MGSV walkthroughs was my most recent run-in to this.
      - mreiland 11y ago
        no link?
Confusion 11y ago
Well, I would say the common argument is that you send an intermediate format to decouple yourself from the specifics of the receiver. HTML is an intermediate format and the client is free to render it how it sees fit for the device on which it is running, which allows compatibility with newer and different clients. It's like shipping some intermediate bytecode doesn't seem to make sense, because every client needs to compile it, until you realize it allows you to ship one format that can even be used by many different architectures and even new architectures that didn't exist when you emitted the bytecode. At the cost of a second component that needs to be present on the machine to do the compiling. When a vast majority of clients is the same or sufficiently similar, it may of course make sense to offer an optimized format, that requires less client-side processing But what would such an optimized format for the web be?
- h679496 11y ago
  Isn't it obvious? You just need an optimized templating engine that is implemented in the browser instead of js.
  - insanebits 11y ago
    Isn't HTML already templating engine for displaying text?
rythie 11y ago
Implementing caching servers and horizontal scaling is complicated and requires talented people who typically need to be on call. It's lot of complicated moving parts, which can easily go wrong outside office hours when a site unexpectedly gets popular. Where as with a static site, the complicated bit is only in the updates. A single server can handle massive numbers of requests and it's easy to scale further with a CDN.

deleted 11y ago

[deleted]

firasd 11y ago

About static vs. dynamic rendering on the server side, the Movable Type CMS used to do exactly as you suggest and it was difficult to make changes on large sites. Imagine if you want to change a link on a sidebar and have to re-build every HTML page served by your CMS to do that. Caching dynamic pages is a bit of a “hybrid” system where you can just publish your new sidebar, then clear cache on the first request for each particular changed page. About the client side, the situation is a bit different because the JS is actually third-party software designed to be dropped in and updated easily, so it’s not just an implementation detail. Very few publishers are going to build or host their own server-side analytics or commenting systems…

radicalbyte 11y ago
You can do that async - have the edit experience use the db then a publish action which re-generated the site. As for monitoring/comment systems: using external sites is actually really good for performance. Third party tools do a great job of blocking them and that makes a lot of sites very fast.
userbinator 11y ago
Imagine if you want to change a link on a sidebar and have to re-build every HTML page served by your CMS to do that. On the other hand, doing the processing on the client side means that work would be done every single time by every single client, and the result of that work isn't even preserved. As decades of programmers have had absolutely no problem compiling software with many, many more dependent files than pages on a typical HTML site, I suspect the problem is in the implementation and not the concept. build or host their own server-side analytics or commenting systems I've not written any analytics before so I won't comment on that, but it's funny you mention commenting systems since a simple one (textboard-like) is literally less than a day's work.
- TeMPOraL 11y ago
  > On the other hand, doing the processing on the client side means that work would be done every single time by every single client, and the result of that work isn't even preserved. Exactly! If that amount of work on server seems big, then realize that doing it client-side means dumping a million[0] times more work on the society, with all associated electricity usage. [0] - literally; if you have 100k visitors and each visited you 10 times, that's 1M times the work that should actually have been done.
oneeyedpigeon 11y ago
That's exactly how a site I work on operates. True, changing a link across all pages is a very slow operation (about 50 seconds for 11,000 pages) but a) that's happening in a highly structured and powerful but resource-intensive way (i.e. DOM parsing) b) it happens so rarely that I haven't bothered to optimise it. The main trade-off for an update that happens maybe once every 1-2 months taking 50 seconds is that every single page load is faster. There are other benefits from static files, too - it's much easier to work with content in flat files, 1-per-page, than the kind of complicated database / file / code relationship used by many 'normal' CMSes.

alkonaut 11y ago

It's easy to forget that html isn't the end product, the end result is lit pixels on some screen (usually). If we send json then that needs to be converted to html and that html needs to be layout and rendered. If we send html we just skip the first step. Displaying a web page isn't "free" for the client just because it's html. The question is this: what kind of overhead is a template rendering task (in js) for a modern browser, compared to the task of layout and rendering a finished html page? It will depend of course on the complexity of the data and the resulting page, as well as the browser, templating engine etc. I have no clue whether it's 1%, 10% or 50%.

firasd 11y ago
The slowdown caused by JS isn’t due as much to the rendering speed as the HTTP requests.
TeMPOraL 11y ago
> It's easy to forget that html isn't the end product, the end result is lit pixels on some screen (usually). If we send json then that needs to be converted to html and that html needs to be layout and rendered. If we send html we just skip the first step. Displaying a web page isn't "free" for the client just because it's html. JSON->HTML rendering may be only one step, but by not skipping it, we're making hundred thousands computers redo the computation that could be done once on a server. This has some electricity cost aggregated over all users, which we externalize to society because of a fad. You can't really skip the HTML->pixels step because of data size issues, but there's little excuse for client-side rendering of static content.
- moonchrome 11y ago
  Who says you're doing JSON->HTML ? Both HTML and your data format need to get deserialized to DOM - depending on use case it can be faster/smaller to do the deserialization in JS from some format rather than using browser HTML parser. HTML is actually non-trivial to parse and isn't really optimized for anything.
odiroot 11y ago

onewaystreet 11y ago

Because many publishers found that visitors wont wait the few extra seconds it takes to load the content on the server. They need to see something loading in their browser or they abandon the content.

TeMPOraL 11y ago
Then show them an AJAX spinner while fetching the static site in the background.
- Gigablah 11y ago
  That's what one of the Blogger templates does, and I've seen people on HN complain about having to see an AJAX spinner just to load a blog.
  - TeMPOraL 11y ago
    Yeah, I'm such a complainer too. Blogs are one of the last places you should ever see such a spinner because they're just static content with sidebars. There should be little to none client-side rendering involved. And the spinner on Blogger is only a (very visible) symptom of the bigger problem - the whole site is incredibly slow and resource heavy.
    - pjmlp 11y ago
      I can understand interactive documents like notebooks (IPython and such) and games, for anything else just give me HTML + CSS.
spacecowboy_lon 11y ago
I think you mean publishers built a slow site and are bemused by why they don't have any traffic. Its a lack of repsect for techies and quality and not understanding that the web is not a glossy brochure that you can't optimize by shouting at a printer over the phone.

mpweiher 11y ago

Excellent point, I never understood that either, with developers being happy that their supercomputing buzzsaw can generate a couple of 100K of text in about a second. OTOH...it turns out that so-called "static" content actually also requires quite a bit of processing. Apache (or whichever server you are using) is not exactly doing nothing. And it also turns out that this sort of processing is I/O heavy, which isn't exactly the fastest path on the machine, even if all the caching done by various entities (more processing) tries and mostly succeeds in making sure very little I/O actually takes place. Add to that that moving data around nowadays is generally much more expensive than doing computation on it (several instructions per clock-cyle, multi-Gigahertz clocks, so well over a hundred clocks per main memory access, and millions per I/O, even with an SSD). In short: if we re-examine our assumptions in light of a changing environment, might a well-optimized "dynamic" stack not be faster than a "static" one? (Compare: VoltDB...). And of course you could then cache the results in memory as well.

mpweiher 11y ago

mseebach 11y ago

But the client side rendering isn't static, and it certainly isn't 1000:1 R:W. There are different browsers on different devices, with different resolutions, pixel densities, zoom-levels, minimum font sizes, window-sizes and orientations. Static client side rendering at 1000:1 would mean that you could render an image file once and serve it to (at least) 1000 clients.

TeMPOraL 11y ago
Static client side rendering means you render your site as HTML+CSS+utility JS on server once, and send a saved copy to people when asked, as opposed to client-side rendering in which you send them data, a template and JS that will build all HTML in browser.
creshal 11y ago
> But the client side rendering isn't static, and it certainly isn't 1000:1 R:W. There are different browsers on different devices, with different resolutions, pixel densities, zoom-levels, minimum font sizes, window-sizes and orientations. Yeees, and? Response CSS designs solve a lot of that with zero server-side computation overhead, and (compared to JS templating/render engines) minimal client-side overhead.

mreiland 11y ago

At one point gamefaqs.com would generate static pages and serve those everywhere it could. This was before cnet bought the site though, I have no idea what they do nowadays, your post just reminded me of it.

creshal 11y ago
It doesn't matter much, the 70+ embedded youtube videos some users put into their FAQs will kill your browser either way.
- mreiland 11y ago
  Can you link to an example of this? I've been on gamefaqs since the very beginning and I've never seen that. In fact, the vast majority of FAQs on that site are still text.
  - creshal 11y ago
    One of the MGSV walkthroughs was my most recent run-in to this.
    - mreiland 11y ago
      no link?

Confusion 11y ago

Well, I would say the common argument is that you send an intermediate format to decouple yourself from the specifics of the receiver. HTML is an intermediate format and the client is free to render it how it sees fit for the device on which it is running, which allows compatibility with newer and different clients. It's like shipping some intermediate bytecode doesn't seem to make sense, because every client needs to compile it, until you realize it allows you to ship one format that can even be used by many different architectures and even new architectures that didn't exist when you emitted the bytecode. At the cost of a second component that needs to be present on the machine to do the compiling. When a vast majority of clients is the same or sufficiently similar, it may of course make sense to offer an optimized format, that requires less client-side processing But what would such an optimized format for the web be?

h679496 11y ago
Isn't it obvious? You just need an optimized templating engine that is implemented in the browser instead of js.
- insanebits 11y ago
  Isn't HTML already templating engine for displaying text?

rythie 11y ago

Implementing caching servers and horizontal scaling is complicated and requires talented people who typically need to be on call. It's lot of complicated moving parts, which can easily go wrong outside office hours when a site unexpectedly gets popular. Where as with a static site, the complicated bit is only in the updates. A single server can handle massive numbers of requests and it's easy to scale further with a CDN.