9 ms·

How Basecamp Next got to be so damn fast without using much client-side UI

428 points by pors 15y ago

ayanb 15y ago
Cross posted from a comment by DHH on this post >> We use nodejs for local development via Sam’s Pow server, but there’s no nodejs in Basecamp Next itself. It’s all Ruby on Rails. We’re running the default stack using Rails 3.2+, MySQL 5.5, Memcached, a tad of Redis (mostly for Resque), ERB , test/unit, CoffeeScript, Sass, and whatever else is in the default box you get with Rails.
mattmanser 15y ago
Hmm, degraded development experience of doing everything on the client? Bit of a bizarre claim there given the state of templating these days. Sounds more of a prejudice than an actual problem. Also json tends to be much lighter than HTML and if you're taking an optimistic view of updates you can assume the action's been a success and even immediately update the display without waiting for the server to create some simple HTML, so it's arguably a worse user experience due to the lag.
- dhh 15y ago
  We've found that the additional overhead of sending HTML vs JSON is negligible in most cases. The additional speed gained by just sending JSON across is not worth the programming enjoyment hit of doing everything client-side. When you get below 50-100ms per action, things are generally fast enough that this is not a key problem any more. The user cares greatly if you can go from 800ms to 100ms, but not so much if you can go from 100ms to 70ms.
  - MatthewPhillips 15y ago
    Programming enjoyment is, of course, subjective, but for your team the enjoyment is ruby, understood :) What browsers are you getting the 50-100ms numbers? Similar results on mobile (with a 3G connection)?
    - dhh 15y ago
      50-100ms is the time it takes to generate the response on the server. On top of that you have to add the ping time between you and server and whatever other network overhead there is. That's why it's great to be closer to 50ms so you'll stay under 100ms even with the network overhead.
  - ayanb 15y ago
    How does this impact if you are aiming to build a JSON API too?
    - MatthewPhillips 15y ago
      Double work.
      - brunoc 15y ago
        Not with Rails, afaik. Assuming the routes are consistent, you simply add a 'respond' for JSON and send the relevant chunk of the model. Easy as pie.
        nkohari 15y ago
        That assumes, of course, that the viewmodel used to render the server-side view is identical to the model you'd return serialized into JSON. That's mostly true, but not always.
  - zaidf 15y ago
    I have to agree here. A major problem with client-side json is the need for additional documentation and likelihood of things breaking because of simple html edits. For example, if you are populating a bunch of divs inside a parent div with the data returned using json, you would describe this using something like jquery selectors. Now if someone else comes and moves around the div during a redesign, he must go through whole bunch of js to make sure his shuffling of the divs around won't break the jquery selectors populating the data from json. Now think about having a parent div that is simply populated with returned HTML from server-side and it seems way easier than using json to populate the data.
    - jonknee 15y ago
      That's nonsense, the same issue would apply on the server side templates. If you go around messing with DIVs without an idea of the consequences, you will get in trouble regardless of your templates being on the server or client. If you try to have the designer do a redesign without testing the application afterwards, you're always in for trouble.
      - zaidf 15y ago
        you will get in trouble regardless of your templates being on the server or client My view is that you will get in significantly less trouble because manipulating an HTML template that is rendered on server side is much easier to manage and adapt to a new design than finding nitpick jquery selectors across the app that depend on a specific div structure. Btw, my post doesn't imply anywhere that a designer should redesign an app without testing. Therefore, your argument about testing is garbage.
        agilebyte 15y ago
        Rather than "finding nitpick jquery selectors" I use a client side framework (backbone.js) with a lot of Views which makes my development faster vs using server side templating, so I cannot share your opinion.
  - cicloid 15y ago
    How about HTML versus MessagePack ( http://msgpack.org/ http://msgpack.org/ )?
- condiment 15y ago
  Your comment ignores the next section in the article, detailing how they are caching templates with a high level of granularity, and actively removing template logic that impedes efficient cacheing. The article goes on to state that delivery times for cached segments of HTML are under 50ms in most cases. While I agree that it would be possible for them to improve the user's perception of the app performance even further via JSON or client-side updates, with the level of performance they described that type of improvement is unnecessary and introduces additional vectors for bugs to creep into production code.
- MatthewPhillips 15y ago
  Stacker sounds like AJAX as it was done in 2006, except trigger by pushState. Maybe I'm misunderstanding it though. I'd be curious what the performance penalty is on mobile.
  - latch 15y ago
    Well, it's really stacker + pushState + MAX cache. But why would there be a performance penalty? Doesn't this result in the client doing _less_ work all around? Less work than the multi-page approach...and less work from the client-side templating approach, no?
  - asaeidi 15y ago
    Seems it's a highly refined version of what was done in the "old" days. I remember doing something similar for an old project in ASP.NET. (Well more similar to pjax) As to performance on mobile, i think it might all depend on how large the html fragments are.
- jd 15y ago
  When you have a persistent page (that you update with HTML5 push-state) you suddenly have to worry about issues like memory fragmentation. If you do lots of small updates to build a page from a template + JSON every time the user navigates it's easy to hit some situation where everything starts lagging for no clear reason. If you simply swap out a large part of the page with some HTML right out of an Ajax request you don't have to worry about any of that.
  - arturadib 15y ago
    Let's be fair - poorly designed code causes memory bloat whether you're on the client or on the server-side. Also, garbage collection efficiency in browsers is a metric of fierce competition among browsers. It's superb already, and it's getting awesomer by the minute. It's a safe choice for the future.
  - Joeri 15y ago
    And when you do everything client-side, you have to worry about memory leaks. Where the state is, is where the memory pain is.
- arturadib 15y ago
  Exactly what I thought. But you have to remember that 37 is the quintessential Rails shop - client-side logic is not their sweet spot. So it's only natural that for their flagship product they'd rather keep pushing the limits of their comfort zone rather than go all out client-side, no matter how natural that choice might be for a "web app".
- jondot 15y ago
  With all of the backbone and node.js hype going around, people tend to forget that client-side development is developing on the client's side. Sooner than later you will have to deal with * nondeterministic testing, * interactive testing (leaving machines on to run interactive testing at night), * most of the bugs will live on the client side, * you'll have to support each and every machine's quirks (you can't pretend everyone are on at least core i5 and chrome, people WILL have machines that do javascript client side template rendering SLOWLY). * you'll have to find ways to diagnose problems at the client's site, and you will maintain a fleet of various computer configurations to run sanity on. * which will also cause you to start investing in pairwise testing * i can go on DHH is COMPLETELY right. Just because the universe is not ready enough for client side "SPA"s (universe=browsers,internet) I'm laying my arguments out of experience - I've been a long time Ruby/Rails/.Net/JVM server side dev as well as enterprise software dev (enterprise desktop/workstation application suites, as complex as visual studio and the sorts). I have several node.js projects in production and I'm also doing a couple of projects with Backbone.
  - boucher 15y ago
    Your arguments are a checklist of responsibilities for anyone running any web software, whether the actual DOM elements are being rendered on the server or the client.
    - jondot 15y ago
      I agree that you do most of these arguments when most of your content render on the servers too. However, you DO have limited resources (time, money, people), and these points become more and more painful as you push towards client side. Still, I insist that some of these arguments are client side specific. Once instance is javascript template rendering speed. Another is, that you would prefer getting statistics on your speed of rendering at the server's side where it is easily controllable and deterministic, rather than shuffle around to close that loop and build a usage service to report back usage and statistics to from the client's side. I give my arguments from pain and experience in both sides, and I still do both sides despite of the arguments (since it is not always my own choice).
deleted 15y ago
[deleted]
- CWIZO 15y ago
  That is not even close to what is being described in the article.
  - deleted 15y ago
    [deleted]
    - robododo 15y ago
      Richard, MVC JS is great when there’s no way around it. Nice "..." there. You trolling? If so, this is the most bizarre topic for trolling I've seen in a while.
espeed 15y ago
Is this similar to Quora's livenode/webnode2 (http://www.quora.com/Quora-Infrastructure/What-is-Webnode2 http://www.quora.com/Quora-Infrastructure/What-is-Webnode2, http://www.quora.com/Quora-Infrastructure/What-limitations-has-Quora-encountered-due-to-Webnode2-LiveNode http://www.quora.com/Quora-Infrastructure/What-limitations-h..., http://www.bigfastblog.com/quoras-technology-examined http://www.bigfastblog.com/quoras-technology-examined, http://www.quora.com/Quora-Infrastructure/How-does-LiveNode-work http://www.quora.com/Quora-Infrastructure/How-does-LiveNode-...)?
_johnny 15y ago
Interesting. We've just had a discussion in our team about moving as much as possible to the client becasue... developers enjoy writing CoffeeScript!
fleaflicker 15y ago
This is very similar to what Google+ does with Closure Templates.
latch 15y ago
I have to say, reading this made my day. I've been feeling such a push towards client-side templates lately, that I felt like it was just me that didn't get it. I really don't understand what's so compelling - unless those exact same actions are actually (YAGNI) used as endpoints of a web service. render :partial => '..' is such an sweet, simple and natural paradigm to web programming.
- dos1 15y ago
  In my head it's much easier to imagine manipulating a UI based on objects and their state, rather than partial snippets of HTML. It seems like it would get complicated to track the pieces of html you needed to request in order to redraw only portions of the UI. Wouldn't there still be a lot of javascript involved with that logic?
  - rgoddard 15y ago
    Normally that basic cycle would go: 1. I need to view some data, if I do not already have it, request it. 2. Generate html from data 3. Take generated html and place in appropriate place on page. This skips step 2 by requesting the required html rather then the raw data. Step 2 is the part that would require the most code.
    - dos1 15y ago
      I disagree. I work on an app that is entirely client side rendered. We use the JSON data approach. Step 2 is trivial. There are many JS templating engines out there. It takes almost no code. The part that's hard is step 1, figuring out if you have the right data or whether your data is stale. And then, to figure out which part of the UI needs to be redrawn. This is where frameworks like Backbone can come in handy.
- roc 15y ago
  > "I really don't understand what's so compelling - unless those exact same actions are actually (YAGNI) used as endpoints of a web service" Well of course it's not going to appear all that compelling after you rule out the most compelling use case.
jshen 15y ago
This ia a bit of a tangent, but Stacker feels like breadcrumbs that require more real estate than necessary. Maybe the physical metaphor is important, but my first thought was 'this is just bread crumbs'.
jashkenas 15y ago
This is all fascinating ... but does it feel like a sustainable approach going forward, or more like a turbo-charged horse and buggy? Developer happiness aside -- there are plenty of folks who like JavaScript just as much as Ruby -- and assuming that we're talking about an non-public-facing app (like Basecamp Next), is there still an argument that can be made in favor of doing your UI on the server side? Even just in principle, it can't possibly be close to as fast (client-side can render optimistically, where possible), and it can't be close to as flexible (you don't have a model of state on the client to perform logic with). I think that the truth of this is already admitted in that the really fancy bits of UI are being implemented in JS: http://37signals.com/svn/posts/3094-code-statistics-for-basecamp-next http://37signals.com/svn/posts/3094-code-statistics-for-base...
- latch 15y ago
  I don't understand why "it can't possibly be close to as fast"? Is the idea that with client-side templating, you've distributed the rendering? But returning a cached html fragment is the exact same amount of work on the server as returning a cached-json value. Plus, it's less work for the client to drop the already-rendered html into a container than to have to bind the model to the template. I feel like I'm being super dumb, but to me this approach is going to result in faster rendering.
  - MatthewPhillips 15y ago
    Don't overlook the bandwidth overhead of html fragments vs. json snippets. That issue is magnified on mobile. Additionally, if you are doing small updates the DOM API is faster than injecting .innerHTML. I think this has changed in recent years and .innerHTML might have caught up for larger nodes. I personally still use the DOM API because it encourages more simplistic layouts.
    - danmaz74 15y ago
      IIRC, innerHTML is actually faster than the DOM API, if you have to make the same changes...
robinduckett 15y ago
Aren't they are using a lot of client side UI code in order to replace parts of the UI with updated HTML from the server? They have caching on the server side but that can only account for some of this. Although in my couple of years of using Basecamp in my previous job, I never found it particularly "snappy" but then I was one user out of a team of five, out of how ever many users they have.
rkalla 15y ago
Are you guys taking care to send as-plain-as-possible markup from the server to the client and then style it up to keep most the UI development in CSS or are you just sending through raw, styled HTML that gets inserted as-sent directly into the page? This reminds me of the early days of Java Servlets when people would stream back HTML from inside their Java Servlet code to JSP pages to build it out... made it impossibly hard to edit the templates. Then the Java world moved to templating engines, but it was still the same idea. Just curious how you are managing this pain point (I am not a Ruby dev so maybe there is a well-known 'best practice' templating approach?).
- dhh 15y ago
  I'm not sure I understand the pain point you're describing. What's "styled HTML"? We're sending HTML across the wire that looks exactly like the HTML we used to render the initial page. It has classes, ids, and custom data descriptors. We have not experienced any pain doing this.
  - rkalla 15y ago
    You answered my poorly-worded question. Thanks.
- losethos 15y ago
  LoseThos has a document format "Ltf". The format supports local file links and graphics and trees and colors and page settings. It's kinda like an HTML language. This file converts to HTML. http://www.losethos.com/LTHtml/Adam/ToHtml.html http://www.losethos.com/LTHtml/Adam/ToHtml.html Your small minds cannot imagine something besides html or what? Short circuit? Don't get me started on arrogant Indian airheads. I tell them I wrote a compiler and they start talking down to me clearly having no clue they should be talking up to me.
hello_moto 15y ago
I find GWT to be the best platform to do full-blown fat client-side paradigm due to several reasons: 1) UI Templating (like XAML) 2) MVP Pattern + JUnit = test automation as part of your build with minimum investment to infrastructure (most JS based solution requires investment on infrastructure) 3) Resource Bundle (similar to Asset Pipelining) cut down HTTP requests 4) Resource splitting (follow up from #3) to reduce the size of the responses when it comes to resources 5) JS binding (like C binding, C++ binding, JNI, etc) to support popular JS library 6) Top Notch Compiler (arguable... but it's there) that can also prune dead code 7) JS switching based on browser's request (IE browser will get IE-specific JS) 8) CSS variable substitution (like Sass) 9) History framework. 10) Flexibility: end point can either return JSON, XML, or use the built-in XML-RPC mechanism (Java only). The disadvantages are: it's Java and requires compilation. I like GWT the most so far because I don't have to find libraries or tools that support all of the above (Sass, mustache, various js unit-testing tools, library to merge and compile css/js, etc). YMMV.
- bradleyjg 15y ago
  I like GWT as well, but its development history makes me a bit nervous about its future. It is open source, but doesn't have a developer ecosystem outside of google employees (in fact they outright state that no one else can be a committer). That in itself wouldn't be the end of the world, but unfortunately google has a) reduced resources devoted to GWT dramatically in the last year or so (many team members were reassigned to the dart project), and b) has a history rapidly changing 'paradigms' and semi-abandoning the code and documentation for the old way of doing things.
  - hello_moto 15y ago
    I think the biggest danger of using GWT is when it breaks on newer browsers and this has happened once in the past with the release of Safari 4 (or 5?) there was a minor hiccup in which they quickly fix it the next day. That is definitely a major glare. Other than that, if the community decided to fork it (and have the time and capability to do so) it'll be a safe choice for a long time. GWT as of today is definitely very polished and well maintained and fulfil the needs of Single Page App smoother than having to hunt different tools and libraries and to set them individually to be a part of the build tool.
joshontheweb 15y ago
I can't help but feel that you are sacrificing the flexibility of your UI in order to stay in your 'safe place'. IMO its easier to implement complex UI if you can deal with it all in the front end. I have used a similar approach before and felt like my hands were tied many times by it.
abdelm 15y ago
or, you know, you can also stop using Rails.
deleted 15y ago
[deleted]
gabyar 15y ago
Is Stacker a proprietary 37 Signals thing? It doesn't seem to pop up after several search attempts.
- latch 15y ago
  yes, the blog post was quite clear that it's built in-house, with no plans to open source it since it isn't a generic solution (it's purpose-built with Basecamp in mind).
adeelk 15y ago
I think in the whole client-side rendering debate, a lot of people miss the fact that there is a one-to-one correspondence between JSON and a certain subset of HTML, that can be illustrated by the following example: {"user": {"first_name": "David", "last_name": "Hansson"}} <div class="user"> <div class="first_name">David</div> <div class="last_name">Hansson</div> </div> If, as 37signals basically seem to be doing, you restrict yourself to this subset of HTML and use CSS to do _all_ presentational styling, then honestly it’s all the same either way.
edw519 15y ago
I love posts like this, not just for what OP has done, but for what you guys have to say about it. I'm continually asking myself 3 questions: 1. Should I do it on the client or the server? 2. What should be travelling between the client and the server? 3. Based on the answer to #2, what else do I need on the client? Even though I've read all your great discussion points, I'm not sure I'm that much smarter. But it's nice to know I don't suffer alone. :-)
- alinajaf 15y ago
  +1, I'm also none the wiser having read the OP and the comments here. I'm quite used to a fully server side model so I'm currently experimenting with a few side projects to see just how much I can put into the client without going anywhere near the server. I think once I've pushed it to its limits I can start working my way back to the server and achieve a good balance. None of this would of course be bearable without coffeescript and a nice js framework like backbone.
- irahul 15y ago
  > 1. Should I do it on the client or the server? I think most applications use both client and server components. If your question is more on the lines of which one plays a major role, it depends mostly on the app. GMail won't be half as good without all the JS magic, but again, I assume it also has a heavy server component. > 2. What should be travelling between the client and the server? If you need any client side processing or are using a framework which works on raw data(backbone.js), then json/xml/...; or else if your goal is to run the app without reloading unnecessary parts, html is a better idea. > 3. Based on the answer to #2, what else do I need on the client? I am not sure what you are looking for here, but if your goal is to make a responsive app, there are some basic things to take care of. Have bookmarkable links and don't break the back button. Doing that is breaking the base paradigm, and plain ajax does that. Use something that provides bookmarkable links, and doesn't break the back/forwrad button; and at the same time, doesn't reload the whole page when only a small portion of the page needs reloading. pjax is one such solution - the 37signals guys investigated it, then went with their home grown solution, but I think pjax will work just fine for majority of use cases. The trick to pjax is serving content without layout if it is a pjax reques; with layout otherwise. There is nothing you can't handroll, but 1) why would you want to? 2) pjax uses pushState to preserve links and back button behavior. Here is the pjax code: https://github.com/defunkt/jquery-pjax https://github.com/defunkt/jquery-pjax And here is a gist in Flask/Python which shows conditionally including the layout: https://gist.github.com/830fc28c680e77759b4b https://gist.github.com/830fc28c680e77759b4b
tapertaper 15y ago
What's the point of putting the tl;dr at the bottom?
jarnix 15y ago
advertising
va_coder 15y ago
I don't think push state is supported in IE (except ie 10): http://caniuse.com/#search=pushstate http://caniuse.com/#search=pushstate This makes it a non-starter for me.
- akavlie 15y ago
  The app still works in IE, it just requires page refreshes on every page load.
iamleppert 15y ago
Awesome, very interesting article that shows you don't always have to go full client side to get a snappy app. However, I do agree with another poster about this being a turbo charged horse and buggy. You can't, for example, deploy the app to a static CDN, or serve templates from a CDN. Templates can't be cached on the client, so you've developed what is essentially an elaborate partials caching infrastructure that is likely very brittle and must be watched over closely to maintain speed benefits (keeping all those things in mind when content changes -- what to throw out and what to keep, etc). Also, by delivering HTML you're limiting your presentation to browser-only devices, at least with this app. I know basecamp has a JSON API, and at that point, why didn't you just do a traditional client side app I wonder? I think the answer rests in the fact that you preferred the "niceness" of server side development, which is arguably more mature than client side at the present time. If that's the only benefit, in a few years, do you think that advantage will hold true?
bjeanes 15y ago
> Alexander, I think the Law of Demeter is shit and never follow it. Classic DHH
dreamdu5t 15y ago
Summary: Use window.pushState() to reduce HTTP requests and cache every thing.
kristianp 15y ago
I'm showing my ignorance of rails here, but is it hard to do the 'russian doll' style of fragment caching that DHH describes? Are there any gems that do this?
deleted 15y ago
[deleted]
deepkut 15y ago
I just realized how "next last year" is an oxymoron... Or is it? A reference to the present per chance?

428 points by pors 15y ago

ayanb 15y ago

Cross posted from a comment by DHH on this post >> We use nodejs for local development via Sam’s Pow server, but there’s no nodejs in Basecamp Next itself. It’s all Ruby on Rails. We’re running the default stack using Rails 3.2+, MySQL 5.5, Memcached, a tad of Redis (mostly for Resque), ERB , test/unit, CoffeeScript, Sass, and whatever else is in the default box you get with Rails.

mattmanser 15y ago

Hmm, degraded development experience of doing everything on the client? Bit of a bizarre claim there given the state of templating these days. Sounds more of a prejudice than an actual problem. Also json tends to be much lighter than HTML and if you're taking an optimistic view of updates you can assume the action's been a success and even immediately update the display without waiting for the server to create some simple HTML, so it's arguably a worse user experience due to the lag.

dhh 15y ago
We've found that the additional overhead of sending HTML vs JSON is negligible in most cases. The additional speed gained by just sending JSON across is not worth the programming enjoyment hit of doing everything client-side. When you get below 50-100ms per action, things are generally fast enough that this is not a key problem any more. The user cares greatly if you can go from 800ms to 100ms, but not so much if you can go from 100ms to 70ms.
- MatthewPhillips 15y ago
  Programming enjoyment is, of course, subjective, but for your team the enjoyment is ruby, understood :) What browsers are you getting the 50-100ms numbers? Similar results on mobile (with a 3G connection)?
  - dhh 15y ago
    50-100ms is the time it takes to generate the response on the server. On top of that you have to add the ping time between you and server and whatever other network overhead there is. That's why it's great to be closer to 50ms so you'll stay under 100ms even with the network overhead.
- ayanb 15y ago
  How does this impact if you are aiming to build a JSON API too?
  - MatthewPhillips 15y ago
    Double work.
    - brunoc 15y ago
      Not with Rails, afaik. Assuming the routes are consistent, you simply add a 'respond' for JSON and send the relevant chunk of the model. Easy as pie.
      - nkohari 15y ago
        That assumes, of course, that the viewmodel used to render the server-side view is identical to the model you'd return serialized into JSON. That's mostly true, but not always.
- zaidf 15y ago
  I have to agree here. A major problem with client-side json is the need for additional documentation and likelihood of things breaking because of simple html edits. For example, if you are populating a bunch of divs inside a parent div with the data returned using json, you would describe this using something like jquery selectors. Now if someone else comes and moves around the div during a redesign, he must go through whole bunch of js to make sure his shuffling of the divs around won't break the jquery selectors populating the data from json. Now think about having a parent div that is simply populated with returned HTML from server-side and it seems way easier than using json to populate the data.
  - jonknee 15y ago
    That's nonsense, the same issue would apply on the server side templates. If you go around messing with DIVs without an idea of the consequences, you will get in trouble regardless of your templates being on the server or client. If you try to have the designer do a redesign without testing the application afterwards, you're always in for trouble.
    - zaidf 15y ago
      you will get in trouble regardless of your templates being on the server or client My view is that you will get in significantly less trouble because manipulating an HTML template that is rendered on server side is much easier to manage and adapt to a new design than finding nitpick jquery selectors across the app that depend on a specific div structure. Btw, my post doesn't imply anywhere that a designer should redesign an app without testing. Therefore, your argument about testing is garbage.
      - agilebyte 15y ago
        Rather than "finding nitpick jquery selectors" I use a client side framework (backbone.js) with a lot of Views which makes my development faster vs using server side templating, so I cannot share your opinion.
- cicloid 15y ago
  How about HTML versus MessagePack ( http://msgpack.org/ http://msgpack.org/ )?
condiment 15y ago
Your comment ignores the next section in the article, detailing how they are caching templates with a high level of granularity, and actively removing template logic that impedes efficient cacheing. The article goes on to state that delivery times for cached segments of HTML are under 50ms in most cases. While I agree that it would be possible for them to improve the user's perception of the app performance even further via JSON or client-side updates, with the level of performance they described that type of improvement is unnecessary and introduces additional vectors for bugs to creep into production code.
MatthewPhillips 15y ago
Stacker sounds like AJAX as it was done in 2006, except trigger by pushState. Maybe I'm misunderstanding it though. I'd be curious what the performance penalty is on mobile.
- latch 15y ago
  Well, it's really stacker + pushState + MAX cache. But why would there be a performance penalty? Doesn't this result in the client doing _less_ work all around? Less work than the multi-page approach...and less work from the client-side templating approach, no?
- asaeidi 15y ago
  Seems it's a highly refined version of what was done in the "old" days. I remember doing something similar for an old project in ASP.NET. (Well more similar to pjax) As to performance on mobile, i think it might all depend on how large the html fragments are.
jd 15y ago
When you have a persistent page (that you update with HTML5 push-state) you suddenly have to worry about issues like memory fragmentation. If you do lots of small updates to build a page from a template + JSON every time the user navigates it's easy to hit some situation where everything starts lagging for no clear reason. If you simply swap out a large part of the page with some HTML right out of an Ajax request you don't have to worry about any of that.
- arturadib 15y ago
  Let's be fair - poorly designed code causes memory bloat whether you're on the client or on the server-side. Also, garbage collection efficiency in browsers is a metric of fierce competition among browsers. It's superb already, and it's getting awesomer by the minute. It's a safe choice for the future.
- Joeri 15y ago
  And when you do everything client-side, you have to worry about memory leaks. Where the state is, is where the memory pain is.
arturadib 15y ago
Exactly what I thought. But you have to remember that 37 is the quintessential Rails shop - client-side logic is not their sweet spot. So it's only natural that for their flagship product they'd rather keep pushing the limits of their comfort zone rather than go all out client-side, no matter how natural that choice might be for a "web app".
jondot 15y ago
With all of the backbone and node.js hype going around, people tend to forget that client-side development is developing on the client's side. Sooner than later you will have to deal with * nondeterministic testing, * interactive testing (leaving machines on to run interactive testing at night), * most of the bugs will live on the client side, * you'll have to support each and every machine's quirks (you can't pretend everyone are on at least core i5 and chrome, people WILL have machines that do javascript client side template rendering SLOWLY). * you'll have to find ways to diagnose problems at the client's site, and you will maintain a fleet of various computer configurations to run sanity on. * which will also cause you to start investing in pairwise testing * i can go on DHH is COMPLETELY right. Just because the universe is not ready enough for client side "SPA"s (universe=browsers,internet) I'm laying my arguments out of experience - I've been a long time Ruby/Rails/.Net/JVM server side dev as well as enterprise software dev (enterprise desktop/workstation application suites, as complex as visual studio and the sorts). I have several node.js projects in production and I'm also doing a couple of projects with Backbone.
- boucher 15y ago
  Your arguments are a checklist of responsibilities for anyone running any web software, whether the actual DOM elements are being rendered on the server or the client.
  - jondot 15y ago
    I agree that you do most of these arguments when most of your content render on the servers too. However, you DO have limited resources (time, money, people), and these points become more and more painful as you push towards client side. Still, I insist that some of these arguments are client side specific. Once instance is javascript template rendering speed. Another is, that you would prefer getting statistics on your speed of rendering at the server's side where it is easily controllable and deterministic, rather than shuffle around to close that loop and build a usage service to report back usage and statistics to from the client's side. I give my arguments from pain and experience in both sides, and I still do both sides despite of the arguments (since it is not always my own choice).

deleted 15y ago

[deleted]

CWIZO 15y ago
That is not even close to what is being described in the article.
- deleted 15y ago
  [deleted]
  - robododo 15y ago
    Richard, MVC JS is great when there’s no way around it. Nice "..." there. You trolling? If so, this is the most bizarre topic for trolling I've seen in a while.

espeed 15y ago

Is this similar to Quora's livenode/webnode2 (http://www.quora.com/Quora-Infrastructure/What-is-Webnode2 http://www.quora.com/Quora-Infrastructure/What-is-Webnode2, http://www.quora.com/Quora-Infrastructure/What-limitations-has-Quora-encountered-due-to-Webnode2-LiveNode http://www.quora.com/Quora-Infrastructure/What-limitations-h..., http://www.bigfastblog.com/quoras-technology-examined http://www.bigfastblog.com/quoras-technology-examined, http://www.quora.com/Quora-Infrastructure/How-does-LiveNode-work http://www.quora.com/Quora-Infrastructure/How-does-LiveNode-...)?

_johnny 15y ago

Interesting. We've just had a discussion in our team about moving as much as possible to the client becasue... developers enjoy writing CoffeeScript!

fleaflicker 15y ago

This is very similar to what Google+ does with Closure Templates.

latch 15y ago

I have to say, reading this made my day. I've been feeling such a push towards client-side templates lately, that I felt like it was just me that didn't get it. I really don't understand what's so compelling - unless those exact same actions are actually (YAGNI) used as endpoints of a web service. render :partial => '..' is such an sweet, simple and natural paradigm to web programming.

dos1 15y ago
In my head it's much easier to imagine manipulating a UI based on objects and their state, rather than partial snippets of HTML. It seems like it would get complicated to track the pieces of html you needed to request in order to redraw only portions of the UI. Wouldn't there still be a lot of javascript involved with that logic?
- rgoddard 15y ago
  Normally that basic cycle would go: 1. I need to view some data, if I do not already have it, request it. 2. Generate html from data 3. Take generated html and place in appropriate place on page. This skips step 2 by requesting the required html rather then the raw data. Step 2 is the part that would require the most code.
  - dos1 15y ago
    I disagree. I work on an app that is entirely client side rendered. We use the JSON data approach. Step 2 is trivial. There are many JS templating engines out there. It takes almost no code. The part that's hard is step 1, figuring out if you have the right data or whether your data is stale. And then, to figure out which part of the UI needs to be redrawn. This is where frameworks like Backbone can come in handy.
roc 15y ago
> "I really don't understand what's so compelling - unless those exact same actions are actually (YAGNI) used as endpoints of a web service" Well of course it's not going to appear all that compelling after you rule out the most compelling use case.

jshen 15y ago

This ia a bit of a tangent, but Stacker feels like breadcrumbs that require more real estate than necessary. Maybe the physical metaphor is important, but my first thought was 'this is just bread crumbs'.

jashkenas 15y ago

This is all fascinating ... but does it feel like a sustainable approach going forward, or more like a turbo-charged horse and buggy? Developer happiness aside -- there are plenty of folks who like JavaScript just as much as Ruby -- and assuming that we're talking about an non-public-facing app (like Basecamp Next), is there still an argument that can be made in favor of doing your UI on the server side? Even just in principle, it can't possibly be close to as fast (client-side can render optimistically, where possible), and it can't be close to as flexible (you don't have a model of state on the client to perform logic with). I think that the truth of this is already admitted in that the really fancy bits of UI are being implemented in JS: http://37signals.com/svn/posts/3094-code-statistics-for-basecamp-next http://37signals.com/svn/posts/3094-code-statistics-for-base...

latch 15y ago
I don't understand why "it can't possibly be close to as fast"? Is the idea that with client-side templating, you've distributed the rendering? But returning a cached html fragment is the exact same amount of work on the server as returning a cached-json value. Plus, it's less work for the client to drop the already-rendered html into a container than to have to bind the model to the template. I feel like I'm being super dumb, but to me this approach is going to result in faster rendering.
- MatthewPhillips 15y ago
  Don't overlook the bandwidth overhead of html fragments vs. json snippets. That issue is magnified on mobile. Additionally, if you are doing small updates the DOM API is faster than injecting .innerHTML. I think this has changed in recent years and .innerHTML might have caught up for larger nodes. I personally still use the DOM API because it encourages more simplistic layouts.
  - danmaz74 15y ago
    IIRC, innerHTML is actually faster than the DOM API, if you have to make the same changes...

robinduckett 15y ago

Aren't they are using a lot of client side UI code in order to replace parts of the UI with updated HTML from the server? They have caching on the server side but that can only account for some of this. Although in my couple of years of using Basecamp in my previous job, I never found it particularly "snappy" but then I was one user out of a team of five, out of how ever many users they have.

rkalla 15y ago

Are you guys taking care to send as-plain-as-possible markup from the server to the client and then style it up to keep most the UI development in CSS or are you just sending through raw, styled HTML that gets inserted as-sent directly into the page? This reminds me of the early days of Java Servlets when people would stream back HTML from inside their Java Servlet code to JSP pages to build it out... made it impossibly hard to edit the templates. Then the Java world moved to templating engines, but it was still the same idea. Just curious how you are managing this pain point (I am not a Ruby dev so maybe there is a well-known 'best practice' templating approach?).

dhh 15y ago
I'm not sure I understand the pain point you're describing. What's "styled HTML"? We're sending HTML across the wire that looks exactly like the HTML we used to render the initial page. It has classes, ids, and custom data descriptors. We have not experienced any pain doing this.
- rkalla 15y ago
  You answered my poorly-worded question. Thanks.
losethos 15y ago
LoseThos has a document format "Ltf". The format supports local file links and graphics and trees and colors and page settings. It's kinda like an HTML language. This file converts to HTML. http://www.losethos.com/LTHtml/Adam/ToHtml.html http://www.losethos.com/LTHtml/Adam/ToHtml.html Your small minds cannot imagine something besides html or what? Short circuit? Don't get me started on arrogant Indian airheads. I tell them I wrote a compiler and they start talking down to me clearly having no clue they should be talking up to me.

hello_moto 15y ago

I find GWT to be the best platform to do full-blown fat client-side paradigm due to several reasons: 1) UI Templating (like XAML) 2) MVP Pattern + JUnit = test automation as part of your build with minimum investment to infrastructure (most JS based solution requires investment on infrastructure) 3) Resource Bundle (similar to Asset Pipelining) cut down HTTP requests 4) Resource splitting (follow up from #3) to reduce the size of the responses when it comes to resources 5) JS binding (like C binding, C++ binding, JNI, etc) to support popular JS library 6) Top Notch Compiler (arguable... but it's there) that can also prune dead code 7) JS switching based on browser's request (IE browser will get IE-specific JS) 8) CSS variable substitution (like Sass) 9) History framework. 10) Flexibility: end point can either return JSON, XML, or use the built-in XML-RPC mechanism (Java only). The disadvantages are: it's Java and requires compilation. I like GWT the most so far because I don't have to find libraries or tools that support all of the above (Sass, mustache, various js unit-testing tools, library to merge and compile css/js, etc). YMMV.

bradleyjg 15y ago
I like GWT as well, but its development history makes me a bit nervous about its future. It is open source, but doesn't have a developer ecosystem outside of google employees (in fact they outright state that no one else can be a committer). That in itself wouldn't be the end of the world, but unfortunately google has a) reduced resources devoted to GWT dramatically in the last year or so (many team members were reassigned to the dart project), and b) has a history rapidly changing 'paradigms' and semi-abandoning the code and documentation for the old way of doing things.
- hello_moto 15y ago
  I think the biggest danger of using GWT is when it breaks on newer browsers and this has happened once in the past with the release of Safari 4 (or 5?) there was a minor hiccup in which they quickly fix it the next day. That is definitely a major glare. Other than that, if the community decided to fork it (and have the time and capability to do so) it'll be a safe choice for a long time. GWT as of today is definitely very polished and well maintained and fulfil the needs of Single Page App smoother than having to hunt different tools and libraries and to set them individually to be a part of the build tool.

joshontheweb 15y ago

I can't help but feel that you are sacrificing the flexibility of your UI in order to stay in your 'safe place'. IMO its easier to implement complex UI if you can deal with it all in the front end. I have used a similar approach before and felt like my hands were tied many times by it.

abdelm 15y ago

or, you know, you can also stop using Rails.

deleted 15y ago

[deleted]

gabyar 15y ago

Is Stacker a proprietary 37 Signals thing? It doesn't seem to pop up after several search attempts.

latch 15y ago
yes, the blog post was quite clear that it's built in-house, with no plans to open source it since it isn't a generic solution (it's purpose-built with Basecamp in mind).

adeelk 15y ago

I think in the whole client-side rendering debate, a lot of people miss the fact that there is a one-to-one correspondence between JSON and a certain subset of HTML, that can be illustrated by the following example: {"user": {"first_name": "David", "last_name": "Hansson"}} <div class="user"> <div class="first_name">David</div> <div class="last_name">Hansson</div> </div> If, as 37signals basically seem to be doing, you restrict yourself to this subset of HTML and use CSS to do _all_ presentational styling, then honestly it’s all the same either way.

edw519 15y ago

I love posts like this, not just for what OP has done, but for what you guys have to say about it. I'm continually asking myself 3 questions: 1. Should I do it on the client or the server? 2. What should be travelling between the client and the server? 3. Based on the answer to #2, what else do I need on the client? Even though I've read all your great discussion points, I'm not sure I'm that much smarter. But it's nice to know I don't suffer alone. :-)

alinajaf 15y ago
+1, I'm also none the wiser having read the OP and the comments here. I'm quite used to a fully server side model so I'm currently experimenting with a few side projects to see just how much I can put into the client without going anywhere near the server. I think once I've pushed it to its limits I can start working my way back to the server and achieve a good balance. None of this would of course be bearable without coffeescript and a nice js framework like backbone.
irahul 15y ago
> 1. Should I do it on the client or the server? I think most applications use both client and server components. If your question is more on the lines of which one plays a major role, it depends mostly on the app. GMail won't be half as good without all the JS magic, but again, I assume it also has a heavy server component. > 2. What should be travelling between the client and the server? If you need any client side processing or are using a framework which works on raw data(backbone.js), then json/xml/...; or else if your goal is to run the app without reloading unnecessary parts, html is a better idea. > 3. Based on the answer to #2, what else do I need on the client? I am not sure what you are looking for here, but if your goal is to make a responsive app, there are some basic things to take care of. Have bookmarkable links and don't break the back button. Doing that is breaking the base paradigm, and plain ajax does that. Use something that provides bookmarkable links, and doesn't break the back/forwrad button; and at the same time, doesn't reload the whole page when only a small portion of the page needs reloading. pjax is one such solution - the 37signals guys investigated it, then went with their home grown solution, but I think pjax will work just fine for majority of use cases. The trick to pjax is serving content without layout if it is a pjax reques; with layout otherwise. There is nothing you can't handroll, but 1) why would you want to? 2) pjax uses pushState to preserve links and back button behavior. Here is the pjax code: https://github.com/defunkt/jquery-pjax https://github.com/defunkt/jquery-pjax And here is a gist in Flask/Python which shows conditionally including the layout: https://gist.github.com/830fc28c680e77759b4b https://gist.github.com/830fc28c680e77759b4b

tapertaper 15y ago

What's the point of putting the tl;dr at the bottom?

jarnix 15y ago

advertising

va_coder 15y ago

I don't think push state is supported in IE (except ie 10): http://caniuse.com/#search=pushstate http://caniuse.com/#search=pushstate This makes it a non-starter for me.

akavlie 15y ago
The app still works in IE, it just requires page refreshes on every page load.

iamleppert 15y ago

Awesome, very interesting article that shows you don't always have to go full client side to get a snappy app. However, I do agree with another poster about this being a turbo charged horse and buggy. You can't, for example, deploy the app to a static CDN, or serve templates from a CDN. Templates can't be cached on the client, so you've developed what is essentially an elaborate partials caching infrastructure that is likely very brittle and must be watched over closely to maintain speed benefits (keeping all those things in mind when content changes -- what to throw out and what to keep, etc). Also, by delivering HTML you're limiting your presentation to browser-only devices, at least with this app. I know basecamp has a JSON API, and at that point, why didn't you just do a traditional client side app I wonder? I think the answer rests in the fact that you preferred the "niceness" of server side development, which is arguably more mature than client side at the present time. If that's the only benefit, in a few years, do you think that advantage will hold true?

bjeanes 15y ago

> Alexander, I think the Law of Demeter is shit and never follow it. Classic DHH

dreamdu5t 15y ago

Summary: Use window.pushState() to reduce HTTP requests and cache every thing.

kristianp 15y ago

I'm showing my ignorance of rails here, but is it hard to do the 'russian doll' style of fragment caching that DHH describes? Are there any gems that do this?

deleted 15y ago

[deleted]

deepkut 15y ago

I just realized how "next last year" is an oxymoron... Or is it? A reference to the present per chance?