3 ms·

I have written a script in Python. This generates all html/css/images from a bunch of markdown files, and I host them on Heroku (which gives quite nice deployme

I have written a script in Python. This generates all html/css/images from a bunch of markdown files, and I host them on Heroku (which gives quite nice deployment procedure with minimal effort, and costs almost nothing - $7 per month). And due to my strange requirements, I couldn't use the widely used static page generators. I have a blog about programming, and sometimes I have some source files, data files, some programs - all used in a blog post, which is stored in a markdown file. I wanted to keep them in the same directory as the text file. I also wanted to publish only part of those files. To get there I implemented a couple of things like custom markdown tags, which are converted to proper links to the image files etc. The result is quite nice, I can shuffle the old blog posts to other directories, and the generated output is still the same. I can keep the images, data files next to the blog post, and they don't interfere with each other. Previously I had all my blogs on Wordpress. That was terrible. Endless upgrades which sometimes destroyed my blog design. Security holes. Endless tries of hacking the page, or guessing the password. Now I have no password, no hacking attempts (there is nothing to hack, and if someone does - I will redeploy the files from my computer). All posts are in git versioned files. I can edit them locally in Vim. Deployment is as simple as `git push heroku master`.

returnbuyer 10y ago
Can we see? Sounds awesome
- pleasecalllater 10y ago
  Well, that's quite a private project, the code is tangled a little bit, and needs refactoring, which will be a problem due to time shortage.
stevekemp 10y ago
That does sound like a fun project! Though it has to be said that most of the popular static-site creators will allow custom things to be done with plugins. I wrote my own static-blog compiler, which reads a tree of markdown files, then spits out posts, archives, tags, etc. As an intermediary form it inserts all the posts into a local SQLite database (which can be trashed and rebuilt when required), which means I can efficiently output tags, etc. Realistically my generator isn't so different from others, but it does integrate well for my personal use and supports comments - something a lot of static blogs miss. (I literally have a CGI script which writes out comments to flat-files, then these are picked up in the build-process, being fetched via rsync first). Source: https://github.com/skx/chronicle2/ https://github.com/skx/chronicle2/ Demo: https://blog.steve.fi/ https://blog.steve.fi/
- pleasecalllater 10y ago
  I have something similar. Although I don't use any database in the middle. I just read the files. I have a cache layer, which is totally useless since I got an SSD drive. The only problem I have now is generating `similar posts` under each posts, which takes just too much time. And I have no comments. I used disqus, but didn't like that.
  - stevekemp 10y ago
    I used to just read all the posts into a big hash, but when you have 1200+ the RAM usage started to get too high. That's when I started sticking them into an SQLite database. Even if you remove the database post-build, such that each post must be re-parsed and re-added, the reduction of RAM meant things were still very fast on my desktop.
    - pleasecalllater 10y ago
      I have only 76 posts, and all of them cached on disk take 3MB, so don't have this problem yet. However I'm thinking about something else: I'm going to write a blog software with a nice api. It will be storing posts in a database on the server side, and will store them on the local too (as cache) - however the source files will be on disk, versioned by git. And it will send just new versions to server using a local command line utility. I really love publishing with a one liner. This way I could have fast search, and comments on the server side.
      - stevekemp 10y ago
        There are definitely different trade-offs to be made on storage, but it would seem that using both a database and a flat-file structure would be the worst of both worlds. If the database is used to serve the content it must be online, and so you lose one of the big appeals of static-sites: That they are static and can be served trivially. If you're writing in text-files you need some kind of update-process which might become tricky if you edit past-entries, etc. I suspect that publishing with one command doesn't really matter too much on technology; in my own case I just use a Makefile: * Download the comments from the remote server, via rsync. * Rebuild the blog. * Upload the generated HTML-site, via rsync.

returnbuyer 10y ago

Can we see? Sounds awesome

pleasecalllater 10y ago
Well, that's quite a private project, the code is tangled a little bit, and needs refactoring, which will be a problem due to time shortage.

stevekemp 10y ago

That does sound like a fun project! Though it has to be said that most of the popular static-site creators will allow custom things to be done with plugins. I wrote my own static-blog compiler, which reads a tree of markdown files, then spits out posts, archives, tags, etc. As an intermediary form it inserts all the posts into a local SQLite database (which can be trashed and rebuilt when required), which means I can efficiently output tags, etc. Realistically my generator isn't so different from others, but it does integrate well for my personal use and supports comments - something a lot of static blogs miss. (I literally have a CGI script which writes out comments to flat-files, then these are picked up in the build-process, being fetched via rsync first). Source: https://github.com/skx/chronicle2/ https://github.com/skx/chronicle2/ Demo: https://blog.steve.fi/ https://blog.steve.fi/

pleasecalllater 10y ago
I have something similar. Although I don't use any database in the middle. I just read the files. I have a cache layer, which is totally useless since I got an SSD drive. The only problem I have now is generating `similar posts` under each posts, which takes just too much time. And I have no comments. I used disqus, but didn't like that.
- stevekemp 10y ago
  I used to just read all the posts into a big hash, but when you have 1200+ the RAM usage started to get too high. That's when I started sticking them into an SQLite database. Even if you remove the database post-build, such that each post must be re-parsed and re-added, the reduction of RAM meant things were still very fast on my desktop.
  - pleasecalllater 10y ago
    I have only 76 posts, and all of them cached on disk take 3MB, so don't have this problem yet. However I'm thinking about something else: I'm going to write a blog software with a nice api. It will be storing posts in a database on the server side, and will store them on the local too (as cache) - however the source files will be on disk, versioned by git. And it will send just new versions to server using a local command line utility. I really love publishing with a one liner. This way I could have fast search, and comments on the server side.
    - stevekemp 10y ago
      There are definitely different trade-offs to be made on storage, but it would seem that using both a database and a flat-file structure would be the worst of both worlds. If the database is used to serve the content it must be online, and so you lose one of the big appeals of static-sites: That they are static and can be served trivially. If you're writing in text-files you need some kind of update-process which might become tricky if you edit past-entries, etc. I suspect that publishing with one command doesn't really matter too much on technology; in my own case I just use a Makefile: * Download the comments from the remote server, via rsync. * Rebuild the blog. * Upload the generated HTML-site, via rsync.