6 ms·

I really don't understand Wordpress as a blog platform sometimes. Why so many database queries? Why even have database queries at all- aside from perhaps commen

by tibbon 13y ago

I really don't understand Wordpress as a blog platform sometimes. Why so many database queries? Why even have database queries at all- aside from perhaps comments? Static pages make significantly more sense for almost all blog setups (Octopress ftw). Making a blog handle this amount of traffic should be trivial, but Wordpress itself is often an issue.

jacques_chester 13y ago
> Why even have database queries at all- aside from perhaps comments? Comments are non-negotiable features for many bloggers. 3rd party javascript is not the same in the eyes of many (including my little network).
- deleted 13y ago
  [deleted]
- xaritas 13y ago
  Unless things have changed dramatically recently, Hacker News uses the file system to store comments (https://news.ycombinator.com/item?id=5229522 https://news.ycombinator.com/item?id=5229522) and it seems to work well. I can imagine a few cases where having your comments in a DBMS would be a requirement, but none of them seem applicable to situations where WordPress where one might want to use WordPress anyway. Of course, this is guess from a priori reasoning. I would be genuinely interested if somebody tried file-based comments for a WordPress (or a similar platform) based site and it fell over or failed in some way.
  - jacques_chester 13y ago
    Reasonable point, insofar as it's unlike that comments will ever be joined with unrelated articles. The main reason you'd get failures in a flat file system would be the normal reasons you get a relational database in the first place. Either you get update anomalies, or you want to compact the data into single atoms, or you want ad hoc querying rather than a specific pattern of access set in advance. It doesn't help that until 5.6 MySQL's query planner joined on disk if you tried to join any two tables in which one or both has a TEXT field, regardless of engine and regardless of the actual fields selected. As you can imagine, this slows things down a bit, considering that's the core join performed for single post generation. It also makes every kind of "recent comments" plugin a performance nightmare (quite aside from their cache-busting power).
- tibbon 13y ago
  Then all you should need at most is a single query to a single table- which could be loaded asynchronously via Ajax for swift loading time and less load on the server (no slow view renders) and that response could be cached with Nginx/Varnish, and further cached on the backend with Redis. I've seen WP sites that I had to debug/fix with 200+ queries per page. That should never happen.
  - jacques_chester 13y ago
    > Then all you should need at most is a single query to a single table Wordpress divides posts + comments into 4 tables. > which could be loaded asynchronously via Ajax for swift loading time and less load on the server (no slow view renders) Wordpress can't assume any particular PHP + webserver configuration. Sometimes they're configured to send while rendering, sometimes the whole process blocks until it runs to completion. Ajax doesn't magically change that. I also dislike the idea that I should need to download what is a small application to assemble a document ... when I could just download the document straight up. When page loads get too long, I switch on pagination. > cached with Nginx/Varnish, and further cached on the backend with Redis My own setup is memcache for working objects. WP-supercache is configured to write gzip'd pages to disk and nginx is configured to serve .gz versions of page directly from disk whenever they're found. The OS file cache does the rest. > I've seen WP sites that I had to debug/fix with 200+ queries per page. When you see hundreds of queries per page, check your plugins and widgets. Wordpress is not my favourite software by any measure, but out of the box it's not that dumb.
- unknownian 13y ago
  This is exactly it for me as well. I'd use Jekyll/Pelican again if comments were somehow built-in. Disqus is still a good service for those who need it.
minimaxir 13y ago
Database features are important for the CMS aspects of Wordpress, not just the blogging aspects. Although Wordpress crashing under load is what caused me to switch to Octopress.
- RKearney 13y ago
  >Although Wordpress crashing under load is what caused me to switch to Octopress. It's all about the caching plugins. I had my blog hit 300k visits in the span of 8 hours, peaking at about 100 visits per second on a shared web host. Utilizing cashing plugins means every single person that visited my blog was served static content that refreshed itself every 10 minutes.
  - minimaxir 13y ago
    My Wordpress blog on Dreamhost shared hosting w/ WP Super Cache and Cloudflare could easily handle 100 visits/second. 200 visits/second caused blank pages half the time. 300 visits/second? Boom.
    - RKearney 13y ago
      I've never used WP Super Cache so I can't comment there, but having used W3 Total Cache I can say that the performance I get with W3 Total Cache is exactly the same as when I tried out Jekyll/Octopress since both solutions are only serving static HTML files and nothing else. Then again, 300 visits per second, estimating 250KB for the entire page (css, js, images, etc) is about 75Mbps which is going to be a lot to ask from any shared hosting provider.
      - ryandonsullivan 13y ago
        Another thing to look at is using nginx as a web server. It's a lot more lightweight than Apache. Paired with a solid SSD drive and varnish caching, and you can get some really great performance out of WordPress.
smacktoward 13y ago
You need to understand a little history to understand why WordPress is where it is today. The first blog software to get really popular was Movable Type (http://en.wikipedia.org/wiki/Movable_Type http://en.wikipedia.org/wiki/Movable_Type). MT was basically a set of Perl scripts that provided a nice GUI backend to manage content stored in a database. But the actual public-facing sites it created were all static files. When you hit "Publish," it would scan over the database and grind out a new set of static HTML files. This made scaling a Movable Type site really easy, because even back then Apache could serve static files like a monster on practically any hardware. But it meant that when you changed something on your site, that change didn't show up there immediately; you had to wait for a rebuild operation to complete and a new set of static files to be ground out to see it. The larger your site got, the longer that rebuild operation could take. "Rebuilding" became to Movable Type what "buffering" used to be to RealPlayer (remember that?). Then, after a few years of MT being the leading product in the then-rapidly growing blog market, the developers behind it decided to jack up the pricing structure for it (http://scott.yang.id.au/2004/05/license-changes-with-movable-type-3/ http://scott.yang.id.au/2004/05/license-changes-with-movable...). MT was not open source, it was proprietary, for-pay software, so if you wanted to use it you had to pay what they asked for it. And suddenly it looked like, instead of being twenty bucks, a valid MT license was going to cost hundreds. As you can imagine, people freaked out and started looking at the available alternatives. There were lots, but none of them were particularly impressive at that stage. But some prominent people (see http://web.archive.org/web/20060410125402/http://diveintomark.org/archives/2004/05/14/freedom-0 http://web.archive.org/web/20060410125402/http://diveintomar...) noticed a little project called WordPress that, while pretty feature-poor compared to MT, had two big things going for it: it was 100% GPL (so no worries about future price changes), and it used a dynamic publishing model where pages were generated live on request through database queries instead of being baked into static files. "Hooray!" everyone yelled. "No more rebuilding!" And so the rush to WordPress began. It was pretty obvious at the time that using live, dynamic pages would be a scaling headache. But most blogs never get enough traffic for scaling to become an issue, and dynamic pages meant not having to wait for rebuilding anymore. Never underestimate the power of appealing to impatience. The irony, of course, is that once WordPress got popular, people started noticing how inefficient dynamic pages could be, and the trend of the "static site generator" was born. SSGs are now very hot and buzzed-around, at least among programmers (they're generally too nerd-optimized to appeal to the general market). But there's nothing really new about the idea; they're just the pendulum swinging back to where Movable Type was in 2001. Perhaps someday an SSG will dethrone WordPress. If that happens, mark your calendar; you'll want to be prepared when "instant updates!" become a selling point again, five years or so later.
pixie_ 13y ago
Wordpress kind of makes me realize how incredible PHP is. It's a 'program of pages.' Which can basically hot swap update itself as it runs. You can download plug-ins in the control panel and extend the functionality of your site instantly. I still get mixed up trying to wrap my head around it even after 15 years of using it. It's a program that exists in different files that can be updated individually while the program itself is used. Most of the 'state' is temporary or stored in dedicated memcaches or databases. It makes you wonder what if other applications were written in a PHP style structure - what would that be like?
- jacques_chester 13y ago
  PHP succeeded in part over mod_perl because of the very statelessness you describe. State is delegated to a database or a memory caching service; each request is answered from scratch. mod_perl allowed programmers to sink their application code deep into the guts of Apache. In retrospect, this turned out to be a dead-end because it is difficult to install, difficult to update and difficult to scale up. PHP's lack of power and low cost of operation ("here's your FTP account, have fun") made it much more attractive to shared hosts. In short, operational advantages outweighed programming advantages.
- teddyh 13y ago
  That would be a Lisp Machine or a Smalltalk environment.
CurtMonash 13y ago
Features like category lists are implemented via database queries. There's little wrong with that; what's wrong is that small result sets can't easily be pinned in memory. As with website software, cache performance probably has a great deal to do with how WordPress performs.

by tibbon 13y ago

jacques_chester 13y ago

> Why even have database queries at all- aside from perhaps comments? Comments are non-negotiable features for many bloggers. 3rd party javascript is not the same in the eyes of many (including my little network).

deleted 13y ago
[deleted]
xaritas 13y ago
Unless things have changed dramatically recently, Hacker News uses the file system to store comments (https://news.ycombinator.com/item?id=5229522 https://news.ycombinator.com/item?id=5229522) and it seems to work well. I can imagine a few cases where having your comments in a DBMS would be a requirement, but none of them seem applicable to situations where WordPress where one might want to use WordPress anyway. Of course, this is guess from a priori reasoning. I would be genuinely interested if somebody tried file-based comments for a WordPress (or a similar platform) based site and it fell over or failed in some way.
- jacques_chester 13y ago
  Reasonable point, insofar as it's unlike that comments will ever be joined with unrelated articles. The main reason you'd get failures in a flat file system would be the normal reasons you get a relational database in the first place. Either you get update anomalies, or you want to compact the data into single atoms, or you want ad hoc querying rather than a specific pattern of access set in advance. It doesn't help that until 5.6 MySQL's query planner joined on disk if you tried to join any two tables in which one or both has a TEXT field, regardless of engine and regardless of the actual fields selected. As you can imagine, this slows things down a bit, considering that's the core join performed for single post generation. It also makes every kind of "recent comments" plugin a performance nightmare (quite aside from their cache-busting power).
tibbon 13y ago
Then all you should need at most is a single query to a single table- which could be loaded asynchronously via Ajax for swift loading time and less load on the server (no slow view renders) and that response could be cached with Nginx/Varnish, and further cached on the backend with Redis. I've seen WP sites that I had to debug/fix with 200+ queries per page. That should never happen.
- jacques_chester 13y ago
  > Then all you should need at most is a single query to a single table Wordpress divides posts + comments into 4 tables. > which could be loaded asynchronously via Ajax for swift loading time and less load on the server (no slow view renders) Wordpress can't assume any particular PHP + webserver configuration. Sometimes they're configured to send while rendering, sometimes the whole process blocks until it runs to completion. Ajax doesn't magically change that. I also dislike the idea that I should need to download what is a small application to assemble a document ... when I could just download the document straight up. When page loads get too long, I switch on pagination. > cached with Nginx/Varnish, and further cached on the backend with Redis My own setup is memcache for working objects. WP-supercache is configured to write gzip'd pages to disk and nginx is configured to serve .gz versions of page directly from disk whenever they're found. The OS file cache does the rest. > I've seen WP sites that I had to debug/fix with 200+ queries per page. When you see hundreds of queries per page, check your plugins and widgets. Wordpress is not my favourite software by any measure, but out of the box it's not that dumb.
unknownian 13y ago
This is exactly it for me as well. I'd use Jekyll/Pelican again if comments were somehow built-in. Disqus is still a good service for those who need it.

minimaxir 13y ago

Database features are important for the CMS aspects of Wordpress, not just the blogging aspects. Although Wordpress crashing under load is what caused me to switch to Octopress.

RKearney 13y ago
>Although Wordpress crashing under load is what caused me to switch to Octopress. It's all about the caching plugins. I had my blog hit 300k visits in the span of 8 hours, peaking at about 100 visits per second on a shared web host. Utilizing cashing plugins means every single person that visited my blog was served static content that refreshed itself every 10 minutes.
- minimaxir 13y ago
  My Wordpress blog on Dreamhost shared hosting w/ WP Super Cache and Cloudflare could easily handle 100 visits/second. 200 visits/second caused blank pages half the time. 300 visits/second? Boom.
  - RKearney 13y ago
    I've never used WP Super Cache so I can't comment there, but having used W3 Total Cache I can say that the performance I get with W3 Total Cache is exactly the same as when I tried out Jekyll/Octopress since both solutions are only serving static HTML files and nothing else. Then again, 300 visits per second, estimating 250KB for the entire page (css, js, images, etc) is about 75Mbps which is going to be a lot to ask from any shared hosting provider.
    - ryandonsullivan 13y ago
      Another thing to look at is using nginx as a web server. It's a lot more lightweight than Apache. Paired with a solid SSD drive and varnish caching, and you can get some really great performance out of WordPress.

smacktoward 13y ago

You need to understand a little history to understand why WordPress is where it is today. The first blog software to get really popular was Movable Type (http://en.wikipedia.org/wiki/Movable_Type http://en.wikipedia.org/wiki/Movable_Type). MT was basically a set of Perl scripts that provided a nice GUI backend to manage content stored in a database. But the actual public-facing sites it created were all static files. When you hit "Publish," it would scan over the database and grind out a new set of static HTML files. This made scaling a Movable Type site really easy, because even back then Apache could serve static files like a monster on practically any hardware. But it meant that when you changed something on your site, that change didn't show up there immediately; you had to wait for a rebuild operation to complete and a new set of static files to be ground out to see it. The larger your site got, the longer that rebuild operation could take. "Rebuilding" became to Movable Type what "buffering" used to be to RealPlayer (remember that?). Then, after a few years of MT being the leading product in the then-rapidly growing blog market, the developers behind it decided to jack up the pricing structure for it (http://scott.yang.id.au/2004/05/license-changes-with-movable-type-3/ http://scott.yang.id.au/2004/05/license-changes-with-movable...). MT was not open source, it was proprietary, for-pay software, so if you wanted to use it you had to pay what they asked for it. And suddenly it looked like, instead of being twenty bucks, a valid MT license was going to cost hundreds. As you can imagine, people freaked out and started looking at the available alternatives. There were lots, but none of them were particularly impressive at that stage. But some prominent people (see http://web.archive.org/web/20060410125402/http://diveintomark.org/archives/2004/05/14/freedom-0 http://web.archive.org/web/20060410125402/http://diveintomar...) noticed a little project called WordPress that, while pretty feature-poor compared to MT, had two big things going for it: it was 100% GPL (so no worries about future price changes), and it used a dynamic publishing model where pages were generated live on request through database queries instead of being baked into static files. "Hooray!" everyone yelled. "No more rebuilding!" And so the rush to WordPress began. It was pretty obvious at the time that using live, dynamic pages would be a scaling headache. But most blogs never get enough traffic for scaling to become an issue, and dynamic pages meant not having to wait for rebuilding anymore. Never underestimate the power of appealing to impatience. The irony, of course, is that once WordPress got popular, people started noticing how inefficient dynamic pages could be, and the trend of the "static site generator" was born. SSGs are now very hot and buzzed-around, at least among programmers (they're generally too nerd-optimized to appeal to the general market). But there's nothing really new about the idea; they're just the pendulum swinging back to where Movable Type was in 2001. Perhaps someday an SSG will dethrone WordPress. If that happens, mark your calendar; you'll want to be prepared when "instant updates!" become a selling point again, five years or so later.

pixie_ 13y ago

Wordpress kind of makes me realize how incredible PHP is. It's a 'program of pages.' Which can basically hot swap update itself as it runs. You can download plug-ins in the control panel and extend the functionality of your site instantly. I still get mixed up trying to wrap my head around it even after 15 years of using it. It's a program that exists in different files that can be updated individually while the program itself is used. Most of the 'state' is temporary or stored in dedicated memcaches or databases. It makes you wonder what if other applications were written in a PHP style structure - what would that be like?

jacques_chester 13y ago
PHP succeeded in part over mod_perl because of the very statelessness you describe. State is delegated to a database or a memory caching service; each request is answered from scratch. mod_perl allowed programmers to sink their application code deep into the guts of Apache. In retrospect, this turned out to be a dead-end because it is difficult to install, difficult to update and difficult to scale up. PHP's lack of power and low cost of operation ("here's your FTP account, have fun") made it much more attractive to shared hosts. In short, operational advantages outweighed programming advantages.
teddyh 13y ago
That would be a Lisp Machine or a Smalltalk environment.

CurtMonash 13y ago

Features like category lists are implemented via database queries. There's little wrong with that; what's wrong is that small result sets can't easily be pinned in memory. As with website software, cache performance probably has a great deal to do with how WordPress performs.