7 ms·
Rebuilding the Shopify Admin: Deleting 28k lines of JavaScript
- coldcode 12y agoWhile it's refreshing to see people talk about the things they did wrong and how they intelligently approached fixing them, building your own javascript framework from scratch (batman) is something I would have said no to from the start. Not for the faint of heart or those who have not suffered such an endeavor before.
- Bahamut 12y agoIf I had to guess, none of the solutions back then sufficed for them. Batman has been around for a while.
- funion 12y agoYou're dead on. We started work on Batman some time ago now - options then were quite limited.
- tericho 12y ago2 years ago I was exploring frameworks for an enterprise project. I tried Batman along with most others, I can easily see why building your own framework would have been an option any earlier than that. Most frameworks weren't even at 1.0 stable yet. I initially wrote a fairly extensive prototype with Ember, but there were breaking changes so frequently I couldn't keep up with the learning curve. In the end I chose Angular because it had the best documentation, Google dogfooding, and testing was an obvious priority. Worked out so well we're on our 4th major Angular app now.
- adrianhoward 12y agoYeah. I've made that kind of decision a few times in the past — do we use NewFramework or do we develop our own. It's a really tough call. I've been wrong (and right ;-) multiple times going down either path. The last time I had to make that sort of choice we went the set-based design / real options path and developed both in parallel for as long as we could. More expensive, but lower risk. I'm glad we did coz the decision we made four months in on which one to drop wasn't the one we would have made at the start.
- carsongross 12y agoMoney quote from the article: The new admin would make a return to more classic architecture with some modernizations. Our approach would be ERB views and server-side rendering, with the use of Turbolinks, and a lightweight custom JavaScript binding system. This allowed us to tackle problems of code duplication and developer productivity in a single blow. As always, the devil is in the details and I'm looking forward to seeing what their JS binding system looks like, but the pendulum appears to be swinging back towards server-side rendering and models, even amongst the cool kids. I have been working on a small library for doing HTML partial AJAX programming, using fairly straight-forward HTML attributes and traditional server side rendering (plus some goodies like custom HTTP header support, timers, etc.): http://intercoolerjs.org http://intercoolerjs.org I'm using it successfully in a few projects and very much enjoy the simplicity of the whole approach when contrasted with full MVC systems.
- tericho 12y agoCan we please drop the "cool kids" cliche? It's a tired generalization. Clearly at this point both architectures work, it's childish to see this as an argument for server-side rendering. Obviously it works, so do client MVC systems. The value in this article is the humble detailing of their mistakes that many of us also experience in our careers.
- matthewmacleod 12y agoI think part of that is a reaction to how many people jumped on to the client-side JS framework bandwagon very early, because it was "the next big thing", but before the tools were ready for production. I've had to deal with more than my fair share of crappy, hacked-together Ember apps that were developed like this. There have been too many breaking changes to upgrade, performance is bad, an they're full of hacks to work around deficiencies that existed back then. Even though we actually have production-ready frameworks now, that sort of thing can leave a bad taste in your mouth. Especially when you see the same mistakes made by people jumping onto the Meteor bandwagon, for example. But it's defintiely something to try and get over :)
- josho 12y agoI started down a similar path with ember.js and was starting to see the same issues. Primarily the business models duplicated on the client. So, it strikes me that we are still figuring out the right path forward on the client/server balance.
- mabbo 12y agoMy teammate and I compete to see not how much code we can write, but who can have the highest sum of lines deleted in all commits. Any old hack can write a lot of code, but to remove code while still adding functionality (or at least, not removing it) is the sign of a good developer, imho.
- iagooar 12y agoWe do that too, in our team. Reminds me of: http://delete-your-code.herokuapp.com http://delete-your-code.herokuapp.com
- fidz 12y agoA little bit OOT, i need to connect to my VPN in US to be able to see the post. It is always redirect to my country's TLD (www.shopify.co.id), in which there is no related link (http://www.shopify.co.id/technology/15646068-rebuilding-the-shopify-admin-improving-developer-productivity-by-deleting-28-000-lines-of-javascript/ http://www.shopify.co.id/technology/15646068-rebuilding-the-...)
- wvanbergen 12y agoAh, this is a bug in our redirect system. I will look into it.
- wvanbergen 12y agofidz: should be fixed now. Thanks for reporting this.
- aikah 12y agoSo AFAIK they send html back to the client right? that's my understanding of turbolinks The problem with this is that it makes client and server tightly coupled.One cannot update the client app without touching the server code. I understand that's a tradeoff but I think a restful architecture serving only json/xml data is better. And you dont have to duplicate logic that much.If any validation needs to happen clientside,create some validation resource for each model and do it entirely serverside for instance. Even SPAs dont need fat clientside models or a complex service layer on the client.
- matthewmacleod 12y agoI don't know that what you've identified there is actually a problem in practice – I'd wager that in a system like Shopify's admin, the client and server are inherently coupled regardless of what architecture you choose. Trying to abstract to a client-side app and JSON API in these situations often results in more complexity for few benefits.
- Joeri 12y agoIf you have multiple front ends on top of a shared backend, like a desktop and mobile UI, as well as a bunch of third party integrations, then it makes a lot of sense to build the API first. The thing is that it requires three different use cases before you can make an API generic enough to cover the majority of the needs, and that makes it difficult to build the API first and UI later. I believe in the approach, but it's not the right way to go for everybody.
- wvanbergen 12y agoThis. We ended up creating models for the exact same stuff in both Ruby and JS. Lots of code duplication, and losing client-side performance due to deserializing JSON all the time.
- colin_jack 12y agoI guess the key bit of the article is: "These frameworks work well for highly interactive apps with complex user interface requirements. We only needed a small piece of what a full framework would offer." As you say its a tradeoff...
- cnp 12y agoI know its difficult to admit mistakes in your own libraries, but I'm curious if anyone at Shopify had considered switching to say Ember or Angular or React before the rebuild?
- AYBABTME 12y agoYes, it's also mentioned in the article.
- cageface 12y agoWith all the buzz around SPA frameworks like Angular or Ember it's a worthwhile reminder that there's a still a lot of room in the world for more conventional server-side rendering approaches. Not every web app needs to be an SPA and old-fashioned Rails apps can be a lot simpler to implement.
- colin_jack 12y agoYup, also a good reminder that you need to constantly question the design decisions as you go (on an almost daily basis).
- jbergens 12y agoOn the other hand I had to develop a small admin framework and decided to do it with server rendering since I thought it would be easier. I soon found all the small details I needed to think about and that there were a number of server side components/files that I had to create to do what I wanted to do. It would probably have been easier to just do it in the SPA framework I am used to (Angular).
- bigpeopleareold 12y agoSometimes I wish I can do more server-side rendering etc. I have been working on single page applications for awhile now; I miss the ease that is afforded by just capturing state through server-side page transitions. Specifically, with SPAs (existing codebases now mostly), I run in bugs a lot that involve Backbone events: when are they firing, what part of the codebase is firing them, who is ignoring the global state and firing anyway. Sometimes, looking through a call stack is not even helpful. Every project that I worked on had these problems. The last big bug I dealt with involved, for me, an unfamiliar codebase and about a week-and-a-half of sitting with the debugger tracing every line. While for new projects, you can work very hard to ensure that firing events is hygienic, existing projects might not be so hygienic. With a server-centric approach, each page load limits the mental load with state. On page-transition, the page state is clean. While I think frameworks like Angular is very interesting, I tend to question my own ambition personally. Something like the approach that Turbolinks takes might actually be more appealing.
- jawngee 12y agoFrom the perspective of someone deeply rooted in UI development for the desktop, seeing stuff like this makes me glad I am deeply rooted in UI development for the desktop. If I had to build a Cocoa app with this messy, complicated shit ... I'd blow my brains out. All over my desk. Maybe I'd leave a note, but I think people who knew what I did for a living would get it and not need me to leave one. It also makes me wonder, how we've all decided that HTML is the way to go for this class of application, how hacking a document display format to ape a native UI is a good idea. I have to think that at some point people think that there has to be a better way to deliver a network transparent, portable UI without worrying about strange things like a Shadow DOM, the hack that is CSS, and all of the other weird shit web developers have to put up with to create something that mimics (usually poorly) a desktop app. Like is there a breaking point where people are like "fuck it" and come up with an alternative browser that is specific to this and similar kinds of applications? I guess not because the gray line between what HTML is good at and what people need to HTML to do despite itself is a pretty big fat line.
- aikah 12y agoThings are not that complicated today,really.It's just that there isnt only one way to do things. > I have to think that at some point people think that there has to be a better way to deliver a network transparent, portable UI without worrying about strange things like a Shadow DOM, the hack that is CSS, and all of the other weird shit web developers have to put up with to create something that mimics (usually poorly) a desktop app It already exists : GWT,or you can use something like C++ and the emscripten toolchain,and whatever C++ gui framework. So if you dislike all the js/html/css you can definetly avoid it.And users dont care,provided it runs in their browser without plugin. Most webdevs wont use these because they already know javascript and there are good frameworks out there. And at the end of the day it is still a webpage,it's not a desktop app.
- sisk 12y agoThis is one of those religious debates that will never die as long as the web exists in a similar capacity as it does, today. The fact of the matter—whether we like it or not—is that these are the tools we have for web development. It's hacks on hacks on hacks, as one might expect in a suite of technologies originally intended for X but arguably powerful enough to perform Y, Z, and Z(2014). Take ECMAScript: the first draft was written up retroactively, quirks and all. Many of the oddities and edgecases were written into the official language spec and vendors were encouraged to implement them as written (thus, intentionally writing engines with quirks). But the fact remains that no single, "fuck it" has been able to take hold. The technologies we have at our disposal have proven themselves "good enough" to keep that from happening to date. At this point, we've invested ~20 years into these technologies. And, for what it's worth, there are ways to cut through the cruft. These things can be done moderately cleanly. It is possible to write a web-based application that doesn't make one want to blow their brains out (I'd make the argument that, in a lot of cases if you're trying to write a "desktop app" in the browser, you're focusing on the wrong thing but that's neither here nor there[0]). It's simply that legacy and twisting the language to do what we want and not what it was intended for has caused that to not be terribly straightforward. I understand you're being hyperbolic but it's been my experience that this mindset ("this is so bad I'm never going to touch it") generally comes from folks who take one look and turn in disgust. That's fine—and I'm not going to argue it's not without it's warts because that would be insanity—but it isn't so bad that you'd need to off yourself, and I think the staying power it has had regardless of quirks, poor design choices, etc., proves that. [0] - with the right team and the right reasons, it can be done. The Slack desktop app is a pretty good example of when it makes sense and is done quite well.
- peeyek 12y agoWondering why Shopify redirected my request on that post URL to http://www.shopify.co.id http://www.shopify.co.id . Ngghhh, i can't read those sweat post :(
- hackerboos 12y agoTry: http://webcache.googleusercontent.com/search?sourceid=chrome-psyapi2&ion=1&ie=UTF-8&q=cache%3Awww.shopify.com%2Ftechnology%2F15646068-rebuilding-the-shopify-admin-improving-developer-productivity-by-deleting-28-000-lines-of-javascript&bav=on.2,or.r_cp.r_qf.&ech=1&psi=t9JQVPqgIYn1ONu8gYgJ.1414582968361.3&ei=t9JQVPqgIYn1ONu8gYgJ&emsg=NCSR&noj=1 http://webcache.googleusercontent.com/search?sourceid=chrome...
- peeyek 12y agoThanks!
- richardkmichael 12y agoCommented above: https://news.ycombinator.com/item?id=8526930 https://news.ycombinator.com/item?id=8526930
- jbergens 12y agoI think it is great that they hade the courage to redo something that wasn't working for them anymore.
- mromanuk 12y agoDuplication and poor documentation made it difficult for developers to make changes to the admin. Once we started questioning Batman, we saw that the evidence strongly indicated it was time to move on to the next chapter. I don't see a SPA vs Server side rendering debate in the article. Building your own framework was the problem, is not a core competence of Shopify to build JS frameworks. Maybe they could choose other route, like going with React or Angular there.
- wvanbergen 12y agoWe actually evaluated other options as well, including Angular, but found them not the best solution for our problem. We documented our experiments and evaluations at the time here: https://gist.github.com/kristianpd/f4c2e0aeb53d09f6def1 https://gist.github.com/kristianpd/f4c2e0aeb53d09f6def1
- drderidder 12y agoComplexity can hurt on the server just as much as the client. The same thing that Shopify experienced with Batman.JS had affected our server-side codebase: uncontrolled complexity stemming from ad hoc design and frameworkophilia. We achieved incredible performance improvements by moving away from server side rendering to a lightweight, in house micro framework [1] and making improvements in our server side API code, going from many tens of thousands SLOC down to one-tenth of that, and from 0% test coverage to ~85%. AFAICT its very rare to find developers with that unique ability to write truly minimalist code. The natural tendency of almost all of the developers I've worked with seems to be to add complexity and to revel in their ability to navigate that complexity. That's why, given the choice, I try to hire or recommend developers who also have a background in music composition and/or visual arts - people who are motivated more by producing an end product that evokes beauty, not by the intricacies of the medium used to produce the work. Batman was led by someone who previously worked on the Cappuccinno web framework, which is based on its own programming language (Objective-J) and an attempt to port most of the Cocoa framework over to the web environment. Batman was different, but it might say something about its tolerance for complexity. Cross-compilation from 3rd generation language X to 3rd generation language Y has always been a bad idea imho, and seems to be based on a desire to mitigate the effort of learning, much like the tendency to reach for a framework with the most bells and whistles instead of understanding the simplest way to solve a problem. Trying to counter that tendency towards complexity has motivated a few talks I've done at our local JavaScript meetup [2][3][4] (apologies for being overly self-referential). [1] https://github.com/darrenderidder/talks/tree/master/jswidgets https://github.com/darrenderidder/talks/tree/master/jswidget... [2] http://ottawajs.org http://ottawajs.org [3] http://51elliot.blogspot.com/2012/08/a-simple-intro-to-mvc-pubsub-and.html http://51elliot.blogspot.com/2012/08/a-simple-intro-to-mvc-p... [4] http://darrenderidder.github.io/talks/ModulePatterns/#/ http://darrenderidder.github.io/talks/ModulePatterns/#/
- Gigablah 12y agoI ported one of Shopify's open source projects (their dashboard tool) to Golang a while back, and in the process I also managed to wrangle the Batman.js frontend portion into a Grunt pipeline. It was a challenge, to say the least :p https://github.com/gigablah/generator-dashing-go/blob/master/app/templates/Gruntfile.js https://github.com/gigablah/generator-dashing-go/blob/master... I've been toying with the idea of replacing the whole frontend with something much more lightweight. Perhaps a good time for me to play with Mithril or React...
- deleted 12y ago[deleted]