6 ms·

Use Jekyll To Keep Your Blog From Folding Under Traffic

48 points by kapilkale 14y ago

mshang 14y ago
Then why bother using Heroku at all? GitHub Pages supports Jekyll.
- kapilkale 14y ago
  Good point. I hadn't seen / used github pages before, and it looks like they let you set up a custom domain so might be a comparable solution.
  - nickknw 14y ago
    I use github pages with jekyll and I'm pretty satisfied with it. I have an additional wrinkle - I have a jekyll plugin I wrote that I want to use, so I have to run jekyll on my own machine and then commit the plain html output to a separate repository. Once it was set up though it doesn't look much different though, just one extra step (running ./push_site_live.sh) after I'm done.
- lachyg 14y ago
  Because you can't use plugins (RSS, Read More, etc)
- ryanlchan 14y ago
  Another alternative is also just hosting directly off S3, which, combined with cloudfront, should eliminate any possibility of folding.
koenbok 14y ago
Shameless plug: if you like your static site generator simpler, in python/django and with amazon s3 deployment built in check my project Cactus: http://github.com/koenbok/Cactus http://github.com/koenbok/Cactus
elithrar 14y ago
I use Octopress (http://octopress.org/ http://octopress.org/) for my own blog[1], which is a Jekyll "framework". Styles, templates, RSS, etc. is all ready to go and you can tweak those without needing to build it from scratch. There's still certainly more development time than a pre-packaged blog, the "new post" workflow requires a couple of lines on the CLI (rake new_post[title]; $EDITOR path/to/new/post/title.md), and there's no clear documentation on creating your own theme (I ported Bootstrap over to Octopress as a self-contained theme), but it can be a nice option for those who want some flexibility. I just host it all on Heroku out of laziness, although I would likely see some speed improvements if I hosted it on a VPS w/ nginx. FWIW, I used to host my blog on Tumblr, until they added fixed Open Graph metadata to the site headers that resulted in me unable to show/change a photo thumbnail when sharing to Facebook (a specific issue, and really not their problem to think about). Also, as others have said, GitHub Pages is a very low-friction alternative to DIY'ing your own Jekyll site and you can always fork a popular users' site to get you up-and-running quickly. [1]: https://github.com/elithrar/octopress https://github.com/elithrar/octopress
- nickknw 14y ago
  I've been starting to see more and more octopress sites out there. I'm glad of it - the default template is a really nice looking, attractive one for blogs. My site uses Jekyll, but I set it up before Octopress was made. If I were doing it now I'd probably use Octopress to get me started.
- bbx 14y ago
  Octopress is a great framework, although the default theme has one major issue: the default fonts. It uses both PT Sans and PT Serif, which comprise 6 .woff files that amount to 300kb and take almost 3 seconds to load. One of the main purposes of a static website is to increase its apparent speed. But when I visit an Octopress website, I always have to wait at least 1 second for anything to appear. On iPad it's even worse, as it takes about 5 seconds before any text appears. I guess Octopress doesn't need those 2 fonts, and can just stick with Georgia and Arial (or Verdana). Apart from being lazy to switch, it's one of the reasons why I'm still using Jekyll.
TomGullen 14y ago
This doesn't seem to be solving the root cause of the problem. Seriously, if your website can't handle bursts of traffic like HN would give - upgrade your server. Pay more. Do it now and don't wait for it to fail. Jekyll sounds clever, but I would be interested to know the benefits of using it over learning how to cache content properly in your server side language, or just getting a better server.
- nickknw 14y ago
  Benefits of using static site generators from my point of view: * Don't have to worry about traffic at all. * Don't have to pay for hosting (it's free using Heroku or Github, probably other places as well). * Less moving parts, less to go wrong. For my simple personal site I don't need any server-side code and don't want it. * Still super easy (for me) to use to publish content.
  - btilly 14y ago
    Free hosting is only free if your time has no value. See http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-blog-anymore/ http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-b... for more.
    - nickknw 14y ago
      Nope, my time has plenty of value and I'm pretty sure my hosting is still free. Patio11's article is a good one but just not relevant to my situation. I've spent 0 minutes managing hosting details since I first put my website live. That's just the way I like it. That's part of the reason I chose a static solution: less moving parts, less fuss maintaining it.
  - harshreality 14y ago
    * No possibility of SQL injection * No possibility of XSS (generally, although perhaps unusual content modification via javascript could open up XSS avenues?) * No possibility of obscure dynamic language runtime bugs
- 14y ago
inghoff 14y ago
I've grown more and more into thinking having a database is maybe overkill for only a blog. I do use Octopress too and it really makes you think "do you really need a DB for that?" If you compile your posts into HTML, use Javascript for more dynamic content (like comments by Disqus or a Twitter feed, for example), HTML files take almost no resources to serve, so you can handle a huge load without many resources.
dannygarcia 14y ago
Jekyll's really great. I found that it makes publishing articles super easy (especially with GitHub Pages) while giving you full control of your front-end. If you use grunt.js, you might like this grunt-jekyll plugin I wrote: https://github.com/dannygarcia/grunt-jekyll https://github.com/dannygarcia/grunt-jekyll
marcrosoft 14y ago
I got sick of Wordpress security issues and updates so I moved my blog to Octopress. What is even better is that you can host those files without a server environment with Amazon S3 and cloudfront.
pessimism 14y ago
On that note, does anyone know a working S3 policy script to prevent hotlinking outside your domain/bucket? I can't get any to work.
kmfrk 14y ago
Maybe someone should create a Use WP-Cache website similar to the famous Use bcrypt.
Axsuul 14y ago
Static sites are great except they limit what sort of content you can put on your site. Varnish is a great way to make your blog "web scale".
pdonis 14y ago
I've looked at Jekyll, but even it was a bit too complex for my needs (plus, I'd rather use Python than Ruby...). So I rolled my own: https://github.com/pdonis/simpleblog https://github.com/pdonis/simpleblog
deleted 14y ago
[deleted]
davidbanham 14y ago
You need to go deeper. Take all those static files you've generated and just put them right on Akamai. Don't even worry about scaling your infrastructure on Heroku or EC2, because it's all on Akamai. It's dirt cheap and really easy to keep updated if you do it right. I presented this at SydJS a little while ago, my slides are here: http://revealjs.jit.su/#/5 http://revealjs.jit.su/#/5
MatthewPhillips 14y ago
No reason to make things so complicated. HTML5 tags makes writing your blog in plain HTML/CSS a breeze. On nginx with a cheap vps you'll handle any load.
JamesLeonis 14y ago
I recently ported my blog to Jekyll/Github pages, and I couldn't be happier. It becomes an interesting challenge to figure out how to implement common blog features, like RSS, tags, and comments without a back-end to call to. Building a custom theme is likewise a small challenge, but is IMHO easier than building one for a blog system like Wordpress or Tumblr. I also like that it's easy to back up. I have a git clone on two computers as well as Github [1]. Right now I'm experimenting with a tags in javascript that mimics the functionality used in other blog implementations. [1]: https://github.com/jamesleonis/jamesleonis.github.com https://github.com/jamesleonis/jamesleonis.github.com
flexterra 14y ago
I tried octopress and it turned into a nightmare, because I had to migrate hundreds of posts and wanted a custom theme. Making a custom theme proved to be harder than what I was able to tolerate. Now I use Wordpress with WP Super Cache plugin which effectively turns my site into a bunch of static files. My site has gotten a lot of traffic from HN and reddit without ever going offline and I made a custom theme in a few hours. http://elweb.co http://elweb.co
laktek 14y ago
I host my blog at Amazon S3, not only it takes away the worry of high traffic, it almost cost nothing (I paid $1 last month for 8GB of transfers). I wrote about the setup sometime back - http://laktek.com/2011/11/17/why-and-how-i-revamped-my-blog http://laktek.com/2011/11/17/why-and-how-i-revamped-my-blog While Jekyll is a nice fit for creating and maintaining a blog, it's an overkill for running other kinds of static sites. I felt there's a need for a simple and more flexible static site generator, so I wrote Punch (http://github.com/laktek/punch http://github.com/laktek/punch). These days, I'm working on a Punch based extension to generate blogs as well, which solves couple of pains I have with Jekyll (such as slow generation, non-existent error handling, lack of support for partials and rigidness in templating, etc.)
- masnick 14y ago
  +1 for S3 and static sites. I wrote about this topic last winter after a rash of links on HN went down. http://www.maxmasnick.com/articles/bulletproof_your_blog/ http://www.maxmasnick.com/articles/bulletproof_your_blog/ I recently switched to github pages after their stability improved, but if you don't want to pay for a github account or you really want 100% uptime, S3 is where it's at.
- bad_user 14y ago
  The problem with S3 is that I hate www. in front of my domain name.
yesimahuman 14y ago
I use Jekyll+GitHub Pages for my personal blog. It works fine, but I recently started using Wordpress for my company blogs, and I prefer it quite a bit. I use WP Engine for my blog hosting now and my blogs rarely have problems under load now.
cmwelsh 14y ago
I find it's easier to use a fully hosted platform like Tumblr with a custom domain, but if you like Jekyll then GitHub pages is pretty awesome. If you're using WordPress, install a full page caching plugin.
smacktoward 14y ago
The irony of these "static blogs are The Future" stories is that the first blogging software to get really popular, Movable Type, worked in exactly this way -- grinding out static files. Which led to everyone moaning and groaning about what a pain it was to have to wait for the site to rebuild when you made a change. Which led to WordPress, which used live queries against the database to avoid the need for rebuilding, being hailed as The Future. Which led to everyone moaning and groaning about how their blog would fall over anytime five people even thought about visiting it. Which led to tools like Jekyll, which grind out static pages, being hailed as The Future. Sometimes tech feels less like a story of progress and more like a merry-go-round...
- jonny_eh 14y ago
  Which leads to moaning about moaning. Meta-moaning?
- cheald 14y ago
  Which leads us to "Use whatever you want and put Varnish in front of it", which is actually the future.
  - bad_user 14y ago
    Which leads us to "why in the world would I need beefy servers for Varnish+Wordpress, when I could get away with a micro EC2 instance or a free Heroku dyno".
willholloway 14y ago
Static sites hosted on S3 have become my specialty lately. Not just for blogs. You can do a lot with html,css and javascript. I have created and maintained dozens upon dozens of Drupal sites over the years, and the frequent core and module updates are quite annoying. I've heard it said that sites for Churches have more malware than porn sites, probably because so many run out of date wordpress installations.
- pattern 14y ago
  Would you mind elaborating on what you use for static generation, and how you get it into S3? Any links to your work? I feel this solution has a lot of merit, and I would like to see how others are doing it :)
helpbygrace 14y ago
I'm using Hakyll which derived from Jekyll with Git server. It is quite appropriate for personal simple website like mine. To edit page for website, just open terminal and edit and git commit & push. That's all. After git server accept the push request, it rebuilds and publishes the website automatically like Github with Jekyll.
hpaavola 14y ago
One great thing about these static file generators is that they make it easier to preserve history. You can't just leave Wordpress based blog to linger on your server after you have quit blogging. It will occasionally eat all your resources and every once in a while new gaping security holes are found which need to be patched. And if you decide to switch hosting provider, it's a hassle. With static files all these problems go away and maybe our kids get to see what daddy wrote 30 years ago.
- ibotty 14y ago
  "With static files all these problems go away and maybe our kids get to see what daddy wrote 30 years ago." is this a good thing?
  - pattern 14y ago
    My first reading of this interpreted it as a tongue-in-cheek way of saying "Static websites are resilient enough to last for years, for better or worse!" I do agree with the premise in that most dynamic solutions will in relatively short order (~1-2 years?) show signs of their age - be it memory leaks, security breaches, or just lack of flexibility. "Me Too Aside": A few months ago I finally started up my personal blog[1] which uses Hyde[2] (the Python brother of Jekyll) and is served via an S3 bucket. I found it extremely refreshing not to have to worry about yet another web infrastructure to maintain. It's exceedingly easy to update and maintain. [1]: http://www.verdantrefuge.com/ http://www.verdantrefuge.com/ [2]: https://github.com/hyde/hyde https://github.com/hyde/hyde
kayman 14y ago
+1 for Hyde. +2 for Emacs Org mode to xhtml

48 points by kapilkale 14y ago

mshang 14y ago

Then why bother using Heroku at all? GitHub Pages supports Jekyll.

kapilkale 14y ago
Good point. I hadn't seen / used github pages before, and it looks like they let you set up a custom domain so might be a comparable solution.
- nickknw 14y ago
  I use github pages with jekyll and I'm pretty satisfied with it. I have an additional wrinkle - I have a jekyll plugin I wrote that I want to use, so I have to run jekyll on my own machine and then commit the plain html output to a separate repository. Once it was set up though it doesn't look much different though, just one extra step (running ./push_site_live.sh) after I'm done.
lachyg 14y ago
Because you can't use plugins (RSS, Read More, etc)
ryanlchan 14y ago
Another alternative is also just hosting directly off S3, which, combined with cloudfront, should eliminate any possibility of folding.

koenbok 14y ago

Shameless plug: if you like your static site generator simpler, in python/django and with amazon s3 deployment built in check my project Cactus: http://github.com/koenbok/Cactus http://github.com/koenbok/Cactus

elithrar 14y ago

I use Octopress (http://octopress.org/ http://octopress.org/) for my own blog[1], which is a Jekyll "framework". Styles, templates, RSS, etc. is all ready to go and you can tweak those without needing to build it from scratch. There's still certainly more development time than a pre-packaged blog, the "new post" workflow requires a couple of lines on the CLI (rake new_post[title]; $EDITOR path/to/new/post/title.md), and there's no clear documentation on creating your own theme (I ported Bootstrap over to Octopress as a self-contained theme), but it can be a nice option for those who want some flexibility. I just host it all on Heroku out of laziness, although I would likely see some speed improvements if I hosted it on a VPS w/ nginx. FWIW, I used to host my blog on Tumblr, until they added fixed Open Graph metadata to the site headers that resulted in me unable to show/change a photo thumbnail when sharing to Facebook (a specific issue, and really not their problem to think about). Also, as others have said, GitHub Pages is a very low-friction alternative to DIY'ing your own Jekyll site and you can always fork a popular users' site to get you up-and-running quickly. [1]: https://github.com/elithrar/octopress https://github.com/elithrar/octopress

nickknw 14y ago
I've been starting to see more and more octopress sites out there. I'm glad of it - the default template is a really nice looking, attractive one for blogs. My site uses Jekyll, but I set it up before Octopress was made. If I were doing it now I'd probably use Octopress to get me started.
bbx 14y ago
Octopress is a great framework, although the default theme has one major issue: the default fonts. It uses both PT Sans and PT Serif, which comprise 6 .woff files that amount to 300kb and take almost 3 seconds to load. One of the main purposes of a static website is to increase its apparent speed. But when I visit an Octopress website, I always have to wait at least 1 second for anything to appear. On iPad it's even worse, as it takes about 5 seconds before any text appears. I guess Octopress doesn't need those 2 fonts, and can just stick with Georgia and Arial (or Verdana). Apart from being lazy to switch, it's one of the reasons why I'm still using Jekyll.

TomGullen 14y ago

This doesn't seem to be solving the root cause of the problem. Seriously, if your website can't handle bursts of traffic like HN would give - upgrade your server. Pay more. Do it now and don't wait for it to fail. Jekyll sounds clever, but I would be interested to know the benefits of using it over learning how to cache content properly in your server side language, or just getting a better server.

nickknw 14y ago
Benefits of using static site generators from my point of view: * Don't have to worry about traffic at all. * Don't have to pay for hosting (it's free using Heroku or Github, probably other places as well). * Less moving parts, less to go wrong. For my simple personal site I don't need any server-side code and don't want it. * Still super easy (for me) to use to publish content.
- btilly 14y ago
  Free hosting is only free if your time has no value. See http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-blog-anymore/ http://www.kalzumeus.com/2012/02/09/why-i-dont-host-my-own-b... for more.
  - nickknw 14y ago
    Nope, my time has plenty of value and I'm pretty sure my hosting is still free. Patio11's article is a good one but just not relevant to my situation. I've spent 0 minutes managing hosting details since I first put my website live. That's just the way I like it. That's part of the reason I chose a static solution: less moving parts, less fuss maintaining it.
- harshreality 14y ago
  * No possibility of SQL injection * No possibility of XSS (generally, although perhaps unusual content modification via javascript could open up XSS avenues?) * No possibility of obscure dynamic language runtime bugs
14y ago

inghoff 14y ago

I've grown more and more into thinking having a database is maybe overkill for only a blog. I do use Octopress too and it really makes you think "do you really need a DB for that?" If you compile your posts into HTML, use Javascript for more dynamic content (like comments by Disqus or a Twitter feed, for example), HTML files take almost no resources to serve, so you can handle a huge load without many resources.

dannygarcia 14y ago

Jekyll's really great. I found that it makes publishing articles super easy (especially with GitHub Pages) while giving you full control of your front-end. If you use grunt.js, you might like this grunt-jekyll plugin I wrote: https://github.com/dannygarcia/grunt-jekyll https://github.com/dannygarcia/grunt-jekyll

marcrosoft 14y ago

I got sick of Wordpress security issues and updates so I moved my blog to Octopress. What is even better is that you can host those files without a server environment with Amazon S3 and cloudfront.

pessimism 14y ago

On that note, does anyone know a working S3 policy script to prevent hotlinking outside your domain/bucket? I can't get any to work.

kmfrk 14y ago

Maybe someone should create a Use WP-Cache website similar to the famous Use bcrypt.

Axsuul 14y ago

Static sites are great except they limit what sort of content you can put on your site. Varnish is a great way to make your blog "web scale".

pdonis 14y ago

I've looked at Jekyll, but even it was a bit too complex for my needs (plus, I'd rather use Python than Ruby...). So I rolled my own: https://github.com/pdonis/simpleblog https://github.com/pdonis/simpleblog

deleted 14y ago

[deleted]

davidbanham 14y ago

You need to go deeper. Take all those static files you've generated and just put them right on Akamai. Don't even worry about scaling your infrastructure on Heroku or EC2, because it's all on Akamai. It's dirt cheap and really easy to keep updated if you do it right. I presented this at SydJS a little while ago, my slides are here: http://revealjs.jit.su/#/5 http://revealjs.jit.su/#/5

MatthewPhillips 14y ago

No reason to make things so complicated. HTML5 tags makes writing your blog in plain HTML/CSS a breeze. On nginx with a cheap vps you'll handle any load.

JamesLeonis 14y ago

I recently ported my blog to Jekyll/Github pages, and I couldn't be happier. It becomes an interesting challenge to figure out how to implement common blog features, like RSS, tags, and comments without a back-end to call to. Building a custom theme is likewise a small challenge, but is IMHO easier than building one for a blog system like Wordpress or Tumblr. I also like that it's easy to back up. I have a git clone on two computers as well as Github [1]. Right now I'm experimenting with a tags in javascript that mimics the functionality used in other blog implementations. [1]: https://github.com/jamesleonis/jamesleonis.github.com https://github.com/jamesleonis/jamesleonis.github.com

flexterra 14y ago

I tried octopress and it turned into a nightmare, because I had to migrate hundreds of posts and wanted a custom theme. Making a custom theme proved to be harder than what I was able to tolerate. Now I use Wordpress with WP Super Cache plugin which effectively turns my site into a bunch of static files. My site has gotten a lot of traffic from HN and reddit without ever going offline and I made a custom theme in a few hours. http://elweb.co http://elweb.co

laktek 14y ago

I host my blog at Amazon S3, not only it takes away the worry of high traffic, it almost cost nothing (I paid $1 last month for 8GB of transfers). I wrote about the setup sometime back - http://laktek.com/2011/11/17/why-and-how-i-revamped-my-blog http://laktek.com/2011/11/17/why-and-how-i-revamped-my-blog While Jekyll is a nice fit for creating and maintaining a blog, it's an overkill for running other kinds of static sites. I felt there's a need for a simple and more flexible static site generator, so I wrote Punch (http://github.com/laktek/punch http://github.com/laktek/punch). These days, I'm working on a Punch based extension to generate blogs as well, which solves couple of pains I have with Jekyll (such as slow generation, non-existent error handling, lack of support for partials and rigidness in templating, etc.)

masnick 14y ago
+1 for S3 and static sites. I wrote about this topic last winter after a rash of links on HN went down. http://www.maxmasnick.com/articles/bulletproof_your_blog/ http://www.maxmasnick.com/articles/bulletproof_your_blog/ I recently switched to github pages after their stability improved, but if you don't want to pay for a github account or you really want 100% uptime, S3 is where it's at.
bad_user 14y ago
The problem with S3 is that I hate www. in front of my domain name.

yesimahuman 14y ago

I use Jekyll+GitHub Pages for my personal blog. It works fine, but I recently started using Wordpress for my company blogs, and I prefer it quite a bit. I use WP Engine for my blog hosting now and my blogs rarely have problems under load now.

cmwelsh 14y ago

I find it's easier to use a fully hosted platform like Tumblr with a custom domain, but if you like Jekyll then GitHub pages is pretty awesome. If you're using WordPress, install a full page caching plugin.

smacktoward 14y ago

The irony of these "static blogs are The Future" stories is that the first blogging software to get really popular, Movable Type, worked in exactly this way -- grinding out static files. Which led to everyone moaning and groaning about what a pain it was to have to wait for the site to rebuild when you made a change. Which led to WordPress, which used live queries against the database to avoid the need for rebuilding, being hailed as The Future. Which led to everyone moaning and groaning about how their blog would fall over anytime five people even thought about visiting it. Which led to tools like Jekyll, which grind out static pages, being hailed as The Future. Sometimes tech feels less like a story of progress and more like a merry-go-round...

jonny_eh 14y ago
Which leads to moaning about moaning. Meta-moaning?
cheald 14y ago
Which leads us to "Use whatever you want and put Varnish in front of it", which is actually the future.
- bad_user 14y ago
  Which leads us to "why in the world would I need beefy servers for Varnish+Wordpress, when I could get away with a micro EC2 instance or a free Heroku dyno".

willholloway 14y ago

Static sites hosted on S3 have become my specialty lately. Not just for blogs. You can do a lot with html,css and javascript. I have created and maintained dozens upon dozens of Drupal sites over the years, and the frequent core and module updates are quite annoying. I've heard it said that sites for Churches have more malware than porn sites, probably because so many run out of date wordpress installations.

pattern 14y ago
Would you mind elaborating on what you use for static generation, and how you get it into S3? Any links to your work? I feel this solution has a lot of merit, and I would like to see how others are doing it :)

helpbygrace 14y ago

I'm using Hakyll which derived from Jekyll with Git server. It is quite appropriate for personal simple website like mine. To edit page for website, just open terminal and edit and git commit & push. That's all. After git server accept the push request, it rebuilds and publishes the website automatically like Github with Jekyll.

hpaavola 14y ago

One great thing about these static file generators is that they make it easier to preserve history. You can't just leave Wordpress based blog to linger on your server after you have quit blogging. It will occasionally eat all your resources and every once in a while new gaping security holes are found which need to be patched. And if you decide to switch hosting provider, it's a hassle. With static files all these problems go away and maybe our kids get to see what daddy wrote 30 years ago.

ibotty 14y ago
"With static files all these problems go away and maybe our kids get to see what daddy wrote 30 years ago." is this a good thing?
- pattern 14y ago
  My first reading of this interpreted it as a tongue-in-cheek way of saying "Static websites are resilient enough to last for years, for better or worse!" I do agree with the premise in that most dynamic solutions will in relatively short order (~1-2 years?) show signs of their age - be it memory leaks, security breaches, or just lack of flexibility. "Me Too Aside": A few months ago I finally started up my personal blog[1] which uses Hyde[2] (the Python brother of Jekyll) and is served via an S3 bucket. I found it extremely refreshing not to have to worry about yet another web infrastructure to maintain. It's exceedingly easy to update and maintain. [1]: http://www.verdantrefuge.com/ http://www.verdantrefuge.com/ [2]: https://github.com/hyde/hyde https://github.com/hyde/hyde

kayman 14y ago

+1 for Hyde. +2 for Emacs Org mode to xhtml