5 ms·
Twitter Rolls Twitter.com Back to a Server-Side Architecture
- sunwooz 14y agoSorry, but what's the difference between using a web application architecture and using a server side architecture? I'm curious!
- bsaunder 14y agoI think the general difference is the location of the controller. On more traditional web frameworks, the controller is typically entirely on the server. The model twitter adopted moves the controller to the browser which just invokes services on the web server via API calls. Not really sure "web application architecture" unambiguously indicates "client side controller", but it does in this dialog when contrasted with "server side architecture".
- wangweij 14y agoBy saying "web app", there is an app inside the web browser, while "server side" means the browser only sees the final content.
- wahnfrieden 14y agoServerside rendering: HTML is rendered on the server and sen to the client to display. Clientside rendering: data is sent to the client, which uses JavaScript to render templates clientside using that data as input, to output HTML for display (or for mutating the DOM directly). That's the essential difference.
- anextio 14y agoAre you familiar with Model-View-Controller? In this case it would be moving some of the controller logic (the parts responsible for rendering views and dispatching events from the UI) to the browser, and using an API to communicate with the model (and the rest of the controller). So, if I'm going to display a page of 10 tweets, in the classic server side architecture, I load the tweets, render a view template based on a context containing those tweets, and then send the HTML to the browser. In an API-based web application architecture like this, I _always_ send the same HTML, which is cached in memory and is almost a no-op to send to the browser, then the client side scripting looks at the URL of the page and makes a call to a JSON or XML API that will result in those 10 tweets. These tweets are then rendered on the client side by the browser. This is beneficial in a few ways: * You can re-use the exact same API and expose it to third parties, who can then write apps with all the functionality of your webapp, and you've one less thing to test. * Rendering a template can be a (relatively) time-consuming operation that may require some I/O. On a single-threaded platform like Node.js, where you simply want to dispatch an event (request) as fast as possible and move on, rendering a template takes a lot of time. * If the time is going to be spent rendering that template anyway, you may as well crowdsource it and let the browser handle it. The browser is probably only processing one page load at a time, whereas your servers could be processing thousands or millions. The cons of using this system are related to the benefits: * The API you use is likely to suffer from Abstraction Inversion[1], or if that's intentionally avoided, a Leaky Abstraction[2]. The reason for this is that there may be some pages to be rendered that require more complex, or less abstract, queries on the data. On the server you could easily issue a JOIN query, but it may not make sense to expose such a thing directly via a RESTful interface. Therefore your application may end up firing numerous API requests and joining the results manually in order to complete a request. If you DO expose this functionality, it will probably end up looking very out of place and very specific. * The extra time spent by the browser may not be suitable in all cases. It poses a number of accessibility and performance problems on certain devices. I may not be able to use a certain website with a screen reader because of this, or I may not be able to view it on an ancient version of Opera running on a shitty old phone over 3G. Even on a modern smartphone, the time taken from seeing a Twitter page's header and background with the loading indicator to actually seing the content was a pain in the ass. * Depending on your application and architecture, you may be able to deliver even better performance by caching rendered templates than by offloading it to the browser. For example, I believe Reddit pretty much prerenders and caches every single page on the site, which is why most stuff loads instantly, but at peak times it takes a while to load the message inbox, or the 80th page of posts. This might be made even worse by extra API calls and client side rendering. [1] http://en.wikipedia.org/wiki/Abstraction_inversion http://en.wikipedia.org/wiki/Abstraction_inversion [2] http://en.wikipedia.org/wiki/Leaky_abstraction http://en.wikipedia.org/wiki/Leaky_abstraction
- mark_story 14y agoThat was a really great explanation :D
- dansingerman 14y agoA lot of that could be described as "lessons I have learned on recent projects". I wish I had read (and grokked) this post 18 months ago.
- anextio 14y agoWell, there's no One True Way for Software Engineering, and certainly this service-based approach can and does work for many, many projects. Obviously it wasn't a complete disaster in Twitter's case. If I may, I'll employ another Anti-Pattern, Cargo Cult Programming[1]. This means that a team sees some method or 'One True Way' and starts implementing it without understanding why. It's almost guaranteed to go wrong, and it tends to apply a lot to things that are hip, like this architecture and things like NoSQL. [1] http://en.wikipedia.org/wiki/Cargo_cult_programming http://en.wikipedia.org/wiki/Cargo_cult_programming
- harichinnan 14y agoI wonder if this could be remedied by using a front end Web App Server consuming the JSON API. So you would have a JSON API server and a front end Web App say in Node using the API as a database. You would have the advantages of building a SOA architecture, letting the API server do the heavy lifting. And you could reuse a well tested API for Mobile Apps. I would think a Meteor.JS like front end would be an ideal solution for low traffic websites(Not twitter!)
- lazugod 14y agoMost websites create the webpage on the server. Once your browser receives it, it parses and displays it once. Compare it to a salad - most restaurants prepare their salads in the kitchen. The waiter takes one trip to deliver your plate, and all you have to do is eat. What Twitter was experimenting with was a thick client. The initial page load would contain little more than the basic Twitter architecture and a URL, and this architecture would send off more requests (individual tweet, profiles, responses, lists) based on the contents of the URL, updating the page to fit what it gets back each time. In a salad bar, you don't get one plate. There's a whole barrage of vegetation to select from, and you have to walk around yourself to get any of it. You also have to spend time deciding what you want. Metaphorically, the salad-eater is your web browser, and that time it takes to send and receive more requests, as well as make decisions and display the results, turns a 140-character message into a slowly loading behemoth.
- kijin 14y agoThere's still a shebang in the URL. If I enter a URL without a shebang, it redirects to one that contains a shebang. Since the browser won't include the fragment in the request, how does the server know which page to send back without using JavaScript to make another request? So it's not really "server side", not even the initial page load.
- garindra 14y agoThey can technically look at the referrer header value and look at the URL path of the referring page.
- ryannielsen 14y agoIf you read Twitter's actual announcement (http://engineering.twitter.com/2012/05/improving-performance-on-twittercom.html http://engineering.twitter.com/2012/05/improving-performance...), they've only rolled out the described changes to the tweet permalink page. Other pages will see these changes over the coming weeks.
- grampajoe 14y agoThe client-side approach would work great if they didn't try to cram in so many features.
- DanI-S 14y agoTwitter is used in a lot of developing countries, through a lot of outdated browsers, on a lot of slow computers. I'd imagine that this plays into their choices somewhat.
- AdrianRossouw 14y agofrom what i can tell, they are just rendering the initial page load on the server now.
- wahnfrieden 14y agoNot at all surprising then except that it shows they're confident enough in their choice of clientside rendering to invest more deeply in it (by optimizing the initial load now.) What a misleading title.
- akkartik 14y agoTo mark this momentous occasion I'm going to go back and reread Steve Yegge on Amazon's SOA. http://plus.google.com/112678702228711889851/posts/eVeouesvaVX http://plus.google.com/112678702228711889851/posts/eVeouesva...
- demetris 14y agoHere is the original post, by the Twitter engineers: http://engineering.twitter.com/2012/05/improving-performance-on-twittercom.html http://engineering.twitter.com/2012/05/improving-performance...
- tnash 14y agoI think this is great. The browser isn't an OS, it's a portal. It's already running on an OS. It didn't work for mobile (remember Apple was all about the web apps back in the day), it shouldn't work for desktop.
- bceagle 14y agoIt is always a little misleading reading articles like this because they don't get into many of the details. At the end of the day, you need to build to meet the requirements of your target audience. It doesn't really make sense to think that just because Twitter does something, then you should do it. The optimizations they are talking about may be so specific that they don't even apply to the common mid-sized web app. I think that the overall client-side architecture has broken new ground and does provide benefits that make sense for a lot of sites. I wouldn't think twice about not using the client side architecture just because of Twitter making a change. Also, as some people already noted, Twitter didn't abandon client side rendering. If you go on the site now, you will still get redirected to the #! page.
- jsprinkles 14y agoThis entry doesn't get into details because it's blog spam, submitted by the blogger himself. You want this: http://engineering.twitter.com/2012/05/improving-performance-on-twittercom.html http://engineering.twitter.com/2012/05/improving-performance...
- apievangelist 14y agoYou guys are the blog detail. Your comments are the reason for short post.
- jsprinkles 14y agoI'm not sure how you expect the discussion to be accurate or informative without details that the article you summarized happily goes into. We already see confusion about where it's rolled out because we're reading you, not Twitter Engineering. What does your post add to theirs? Honestly?
- apievangelist 14y agoI'm not looking to add anything to theirs. Thats why I link to them. I'm looking to get discussion here and on Twitter...while I formulate my own thoughts...and more analysis. That's why I said...your the detail of this post.
- swah 14y agoBut Twitter.com is slow (like a couple seconds to load twitter.com/#!/someone) even if you're using the latest Chrome browser - it can't be just fault of the client side architecture... If the initial JSON was coming bundled with the page, the only reason for the slowdown is parsing JS? I thought that was what browsers were good at those days...
- InclinedPlane 14y agoOn a cold load you'd have to transfer 2 megs of mostly JS over the wire, and only after all that was done could JS start rendering. Even with a fast 10 megabits/s line that will still take upwards of 2 seconds just to get to square 1.
- callumjones 14y agoI wouldn't call it a rollback completely, but an improvement on the first time load. The whole Twitter page is rendered initially server side but then any subsequent request is made by JSON calls and rendered client side -- probably the best approach given the time it took for Twitter.com to load.
- joshfraser 14y agoI'm glad to see this. While it was an interesting experiment for Twitter to undergo, their front end performance really took a hit. I expect they'll see their usage numbers on twitter.com start going up again as they fix things.
- SCdF 14y agoThank the creator. It was a blindingly unnecessary fashion statement that caused more problems than it solves, and I wish it safe passage on its way to the closet of failed experiments.
- j45 14y agoSeeing as the web is more service oriented than object oriented.. Are we surprised to see a step towards SOA (presumably from OO that was also at the service layer)? For me Web / mobile apps that are architected from a primarily OO perspective can turn out very differently (often with unique OO bottlenecks) than apps architected from a service-first perspective, using OO to fulfill services.
- deleted 14y ago[deleted]
- parasubvert 14y agoThis isn't a rollback of the API-oriented architecture, it's a rescue of a bunch of wrong-headed design mistakes, like forcing the rendering of individual status pages through a JavaScript function, using a hash-bang URI. One of the major drivers behind the constraints in a RESTful architecture is user-perceived response time. Routing all content through "code on demand", which is effectively what you're doing when you force a JavaScript function to do all the rendering, instead of the browser itself, is not taking advantage of the fast, incremental rendering nature of plain old HTML generated by the server. One can still design good API-oriented URIs with a server-side approach, you're just providing different serializations of the same resource (a nice HTML one and a JSON one for API access).... so there's nothing fundamental that Twitter has lost or abandoned here. They're just using the web better.
- stdbrouw 14y agoHTML is HTML. Why would it be any easier for the browser to render server-generated HTML incrementally, versus client-generated HTML?
- deleted 14y ago[deleted]
- parasubvert 14y agoNot easier... just faster. Server-generated HTML: the browser GETs it and renders it as it streams through. Client-generated HTML: the server GETs an HTML page with a JavaScript link, then GETs a JavaScript .js, then executes it, the jS GETs the JSON, then translates that into HTML at the DOM level. The latter is arguably something that will lead to a better user experience once the page is in a "steady state", i.e. all dependent representations are loaded into the browser and rendered. But relying on it for "first render" makes for a slow experience when (e.g.) clicking on a link to an individual Twitter status.
- stdbrouw 14y agoGetting that JavaScript is a one-time operation, and a 304 from then on. Also, the HTML can include bootstrapped data, saving the roundtrip for JSON. Also, with client-side rendering you execute more code on the client but less on the server, so in an environment like Twitter where it's not possible to do any sort of heavy caching (everybody sees something else), you're simply trading time on the server for time on the client. Not faster, not slower. Server-side HTML generation is not a magical 0ms process.
- jemeshsu 14y agoTwitter is doing the best approach which is a mix of both client-side and server-side. It is not an either or situation.
- halayli 14y agoI feel they were doing it wrong. If they preloaded the page with bootstrapped data + templates then it should be as fast.
- chintan 14y ago2011-02-11: "We've made a tradeoff, however, in making twitter.com into an application using the hash, which is that it now cannot be both an app and a site at the same time." http://www.adequatelygood.com/2011/2/Thoughts-on-the-Hashbang http://www.adequatelygood.com/2011/2/Thoughts-on-the-Hashban...
- ComputerGuru 14y agoI am so proud of twitter right now, that I feel words will fail me, so I shall keep this short and to the point: It takes serious balls to admit you were wrong after you kicked off an entire avalanche that has been breaking the web ever since. Yes, a lot of us knew then it was a ridiculously bad idea and we all said so; but for them to actually take this advice after going so out of their way to go completely client-side is just fantastic. Kudos, twitter folk!
- marknutter 14y agoFor some web apps, namely ones that don't get linked to often like Twitter or ones that are truly trying to behave like desktop applications, hash-bang urls are really the only option for maintaining state in browsers that don't support HTML5 push states yet. I think the backlash against Twitter's and Gawker's use of them is warranted, but not the hyperbole that hash-bangs, no matter the context, "break the web."
- kamweti 14y agoI welcome this but sticking with the fastest twitter client yet - their mobile site. i use it even on desktop plus they recently made improvements on it http://news.ycombinator.com/item?id=2977580 http://news.ycombinator.com/item?id=2977580