4 ms·

I just don't see the point to this kind of stuff.. =/ For high traffic apps, Varnish is the answer as you don't hit the application layer. If you think that's

by ashray 14y ago

I just don't see the point to this kind of stuff.. =/ For high traffic apps, Varnish is the answer as you don't hit the application layer. If you think that's too complicated, try nginx-memcached - also an excellent solution. If not that, try django's template caching with memcached - also extremely fast but will hit the application layer. If you're in some shared hosting environment (you probably are too small still to warrant this kind of aggressive caching on static assets - but hey, efficiency never hurt anybody :P) without access to memcached, use django's cache backend with a file based cache. It's almost 100% as what this does and you don't have any additional overhead. Beats me why people are re-inventing the wheel - or am I missing something ?

yuvadam 14y ago
Yeah, deployment scenarios where you have no application layer, and can only deploy static assets (S3, GitHub pages and the likes).
- ashray 14y ago
  That's interesting. So basically it's for rendering pages and then pushing them on to S3/Other-static-storage and serving them from there ? I don't see any particular gains however since high traffic pages (static or otherwise..) served off S3 will end up costing way more than on a shared host/dedicated server. What would be an exact use case where this would really be important and help out ? I'm just trying to understand what need it solves and under what circumstances hosting static pages on S3 is beneficial in cost/performance.
  - Bockit 14y ago
    Our last 2 clients have asked for static solutions because they don't have the capacity to maintain django apps down the line and don't want to hire new staff or continue paying us for maintenance. We ended up developing both sites in django, which made it easy for them to add copy during development and see how it would actually look on the site. Once finished, we got the HTML output and sent it over for them to put up on their servers. We were pretty happy with how it turned out, and I think django-medusa being able to automate the whole rendering to html files would be nice if we have to do it again.
    - ashray 14y ago
      I see, that makes perfect sense! So it's an interesting thing for website development businesses. Especially the part where you allow the client to modify 'static' content. Well, in that case I can see how this tool could be really handy! Thanks for the input :)
      - Bockit 14y ago
        I'm imagining a setup now where we host their staging environment as it would effectively just be disk space, and when they want to publish changes it could render using this and publish to wherever the static files are hosted (S3, their own service, provide them as a zipped download, etc.).
  - njharman 14y ago
    Small site for which app server is overkill and s3 charges will be minimal.
eragnew 14y ago
for people like me who haven't spent the time to learn the ins and outs of django but still like hcking around in python, things like this are very enticing. so in a way, the author 'invented the wheel' for me, so i don't have to reinvent it. if that makes sense.
natrius 14y ago
"There are only two hard things in Computer Science: cache invalidation and naming things." Static files on S3 are far simpler than setting up Varnish and coming up with a reasonable strategy for cache invalidation. Varnish is great, but I don't know anyone who would describe setting it up as "simple." This solution, on the other hand, looks quite simple.
mtigas 14y ago
It was an itch I wanted to scratch. I’d been tempted to convert my entire (Django-based) blog over to something like Hyde[1], but wanted a bit more flexibility than the framework provided. In most applications I work on, I lovingly use and abuse Memcached, Redis, and Varnish. If you’re working on an application that warrants using a live website and the whole application server shebang, then yes, I’d agree with you. But for something like my blog and other non-dynamic websites which don’t update very much at all, I’m not sure if I see a pressing need for an application server. My blog previously ran varnish-nginx-uwsgi-django, but I was moving off of a VPS and it was the last thing left on that server. I got curious. In the case of something like the L.A. Times’ Data Desk[2] projects (who use their own django-bakery app), if some views are very expensive/slow to generate, you can offload the work from the application server and do it in advance. (This makes sense if you want to just render everything out on a fast workstation or if you have a local database of several hundred gigabytes that you don’t want a live server querying to crunch the data.) It’s not out of the question to pregenerate HTML pages, JSON for visualizations, and simple image files (generated in PIL). In any case: it’s not so much a question of “high traffic apps” as much as the tradeoff between (computation cost + server maintenance cost) and (app that is server-side dynamic or updates frequently). Most people don’t want to configure and maintain an app server (with cache layers and all) for a simple app and those that don’t seem to have uptime issues the moment they get any legitimate traffic: see [3]. So: * I decided I didn’t want to maintain an app server for my blog, and my historical average for updates is about once every four weeks (or even more infrequent). * People seemed to be big fans of Jekyll/Hyde, Movable Type’s static publishing mode[4], WP Super Cache, etc. * I felt a Django-friendly analogue to those would be cool. * Like any developer tinkering with their own blog, there didn’t have to be a point. [1]: http://pypi.python.org/pypi/hyde/ http://pypi.python.org/pypi/hyde/ [2]: http://datadesk.latimes.com/ http://datadesk.latimes.com/ [3]: http://inessential.com/2011/03/16/a_plea_for_baked_weblogs http://inessential.com/2011/03/16/a_plea_for_baked_weblogs [4]: http://daringfireball.net/linked/2011/03/18/brent-baked http://daringfireball.net/linked/2011/03/18/brent-baked
- ericingram 14y ago
  I'm wondering, what kind of dynamic features were you interested in for a blog that you update once per month? Why not just edit static files on S3?

by ashray 14y ago

yuvadam 14y ago

Yeah, deployment scenarios where you have no application layer, and can only deploy static assets (S3, GitHub pages and the likes).

ashray 14y ago
That's interesting. So basically it's for rendering pages and then pushing them on to S3/Other-static-storage and serving them from there ? I don't see any particular gains however since high traffic pages (static or otherwise..) served off S3 will end up costing way more than on a shared host/dedicated server. What would be an exact use case where this would really be important and help out ? I'm just trying to understand what need it solves and under what circumstances hosting static pages on S3 is beneficial in cost/performance.
- Bockit 14y ago
  Our last 2 clients have asked for static solutions because they don't have the capacity to maintain django apps down the line and don't want to hire new staff or continue paying us for maintenance. We ended up developing both sites in django, which made it easy for them to add copy during development and see how it would actually look on the site. Once finished, we got the HTML output and sent it over for them to put up on their servers. We were pretty happy with how it turned out, and I think django-medusa being able to automate the whole rendering to html files would be nice if we have to do it again.
  - ashray 14y ago
    I see, that makes perfect sense! So it's an interesting thing for website development businesses. Especially the part where you allow the client to modify 'static' content. Well, in that case I can see how this tool could be really handy! Thanks for the input :)
    - Bockit 14y ago
      I'm imagining a setup now where we host their staging environment as it would effectively just be disk space, and when they want to publish changes it could render using this and publish to wherever the static files are hosted (S3, their own service, provide them as a zipped download, etc.).
- njharman 14y ago
  Small site for which app server is overkill and s3 charges will be minimal.

eragnew 14y ago

for people like me who haven't spent the time to learn the ins and outs of django but still like hcking around in python, things like this are very enticing. so in a way, the author 'invented the wheel' for me, so i don't have to reinvent it. if that makes sense.

natrius 14y ago

"There are only two hard things in Computer Science: cache invalidation and naming things." Static files on S3 are far simpler than setting up Varnish and coming up with a reasonable strategy for cache invalidation. Varnish is great, but I don't know anyone who would describe setting it up as "simple." This solution, on the other hand, looks quite simple.

mtigas 14y ago

It was an itch I wanted to scratch. I’d been tempted to convert my entire (Django-based) blog over to something like Hyde[1], but wanted a bit more flexibility than the framework provided. In most applications I work on, I lovingly use and abuse Memcached, Redis, and Varnish. If you’re working on an application that warrants using a live website and the whole application server shebang, then yes, I’d agree with you. But for something like my blog and other non-dynamic websites which don’t update very much at all, I’m not sure if I see a pressing need for an application server. My blog previously ran varnish-nginx-uwsgi-django, but I was moving off of a VPS and it was the last thing left on that server. I got curious. In the case of something like the L.A. Times’ Data Desk[2] projects (who use their own django-bakery app), if some views are very expensive/slow to generate, you can offload the work from the application server and do it in advance. (This makes sense if you want to just render everything out on a fast workstation or if you have a local database of several hundred gigabytes that you don’t want a live server querying to crunch the data.) It’s not out of the question to pregenerate HTML pages, JSON for visualizations, and simple image files (generated in PIL). In any case: it’s not so much a question of “high traffic apps” as much as the tradeoff between (computation cost + server maintenance cost) and (app that is server-side dynamic or updates frequently). Most people don’t want to configure and maintain an app server (with cache layers and all) for a simple app and those that don’t seem to have uptime issues the moment they get any legitimate traffic: see [3]. So: * I decided I didn’t want to maintain an app server for my blog, and my historical average for updates is about once every four weeks (or even more infrequent). * People seemed to be big fans of Jekyll/Hyde, Movable Type’s static publishing mode[4], WP Super Cache, etc. * I felt a Django-friendly analogue to those would be cool. * Like any developer tinkering with their own blog, there didn’t have to be a point. [1]: http://pypi.python.org/pypi/hyde/ http://pypi.python.org/pypi/hyde/ [2]: http://datadesk.latimes.com/ http://datadesk.latimes.com/ [3]: http://inessential.com/2011/03/16/a_plea_for_baked_weblogs http://inessential.com/2011/03/16/a_plea_for_baked_weblogs [4]: http://daringfireball.net/linked/2011/03/18/brent-baked http://daringfireball.net/linked/2011/03/18/brent-baked

ericingram 14y ago
I'm wondering, what kind of dynamic features were you interested in for a blog that you update once per month? Why not just edit static files on S3?