19 ms·
Under Deconstruction: The State of Shopify’s Monolith
- meesterdude 6y agoInteresting read. I've seen a component based rails architecture work wonders for cleaning up a codebase and allowing for the benefits of a SOA encapsulation while still keeping everything under a monolithic architecture (and avoiding the networking nasties). Not such a fan of sorbet though, but hopefully something better comes along.
- octernion 6y agowe are actually doing precisely the same thing at instacart (breaking our 1+ million lines of code monolith into discrete components, which we call "domains"), and typing the boundaries and as much of the internals of these domains as possible with sorbet types. this has the benefit of ruby dynamicism (fast development within domains, you can use all the nice railsy tooling, activerecord, and all the libraries we've built over the years), with type safety at the boundaries (we've also put in timeouts, thread separation, and error handling at the boundaries). the additional benefit for using sorbet is that it makes making typed RPC calls (over twirp or graphql) much easier as you can introspect the boundaries trivially. really cool to see other companies evolving similarly given the same starting conditions!
- exterm 6y agoThere are quite a few people talking about this kind of stuff on https://rubymod.slack.com https://rubymod.slack.com. I can send invites, just DM me on twitter https://twitter.com/_exterm https://twitter.com/_exterm
- dragosmocrii 6y agoSlightly off topic, but does anyone know if this "component based" development is what umbrella applications are in Elixir?
- exterm 6y agoIt's certainly related. In very general terms, I would say splitting a Rails app into multiple engines is the same pattern as umbrella applications. However, there are more interesting specifics here about things like all engines sharing a database, but having exclusive ownership of tables, as well as splitting HTTP routing over multiple engines etc.
- Arubis 6y agoI think you'll also find a lot of conceptual overlap with Phoenix Contexts; they'll generally all start as part of the same monolith/app but are sufficiently discrete that you can separate them out more easily than the Rails situation in TFA.
- ravenstine 6y agoAm I the only one who has a distaste for this phrase "component based development"? It just seems like a fancy way of saying object oriented programming without an overarching design pattern.
- aidos 6y agoSounds like the “components” described above are much larger than classes.
- octernion 6y agothat's correct, at least for us a domain encapsulates many response types and dozens of different APIs that wrap various datastores, business logic, etc.
- octernion 6y agowe've actually taken the pattern of making the classes relatively stateless, and explicitly passing around typed state through these explicit apis. it's not really the same design pattern and imo conceptually different.
- IshKebab 6y agoMy god I can't imagine a million lines of untyped code. Must be hell. Presumably you spend all day writing tests?
- octernion 6y agohah, it's not hell but it's not entirely pleasant either. a _lot_ of that is tests, which is essentially how contracts and safety is enforced in ruby (at least prior to types).
- straws 6y agoA number of years ago, I worked on a team (~20 engineers in total) that successfully carved off two relatively independent portions of a large Rails app using engines. I'm happy to see that Shopify is also using that strategy. I'm curious to know more what sorts of challenges they have around managing dependencies across engines — I think what we were doing was fairly vanilla Rails, and we didn't have the opportunity to run into those sorts of issues.
- exterm 6y agoThe answer to that question could probably fill another blog post :D Long story short, Rails and dependency inversion equals lots of friction. The whole framework is built on the assumption that it's OK to access everything from everywhere, and over the years we've built lots of tooling on top of those assumptions. E.g. we heavily use https://github.com/Shopify/identity_cache https://github.com/Shopify/identity_cache with active record associations that cross component boundaries. We also have a GraphQL implementation that is pretty closely coupled to the active record layer and _really_ wants to reach into all the components directly. All of those problems can be overcome, but this is definitely an area where we have to working against "established" Rails culture, and our own assumptions from the past.
- sandGorgon 6y agoWhat's the difference between "componentization+engines" and microservices? From a deployment perspective are your engines deployed and scaled independently?
- exterm 6y agocomponents are - same database - same runtime - same deployment - same repository That said, I don't think this is an either/or. It's a spectrum. you can have components within the same runtime and repository that have separate databases, or components that are using the same database but live in separate repos, etc. From one monolithic app towards fully separated microservices is a spectrum, and I think developers should be enabled to move freely around that spectrum.
- sandGorgon 6y agoThis is a brilliant brilliant article. Does anyone know how Shopify created it's Architecture Guild and grew it ? The author talks about "should have done it earlier"
- exterm 6y agoAs the author, I would know :) Thank you for the praise. Ours kind of organically grew over time, but as I've been keeping it alive for the last few years I have a pretty good idea of how I would start it fresh. You probably have some people in the company who either know much more about architecture than others, or are working on projects that are more interesting in terms of architecture. Find one of them, convince them to give a 15 min talk. Announce the talk widely within the company, tell people to come to the new "architecture guild" slack channel you created to get the details / invites. Schedule an hour to give plenty of time for discussions after the talk. Repeat biweekly.
- sandGorgon 6y agoThanks for replying. How would you do it in a remote-first world? A zoom talk ? How does this go beyond that one talk - would you incorporate aspects of this into official rewards/recognition ? Or is gratification good enough. Getting a zoom audience is gonna be hard.
- exterm 6y agoShopify has been a fully remote company for a few months now. https://financialpost.com/technology/shopify-is-joining-twitter-in-permanent-work-from-home https://financialpost.com/technology/shopify-is-joining-twit... We're not using zoom, but google meet - but yes, these happen completely online now. I find that people that are doing interesting stuff often _want_ to talk about it. However, a big part of Shopify culture is "do things, tell people" - it is definitely encouraged to spend time spreading context. It's not directly part of any rewards framework, but one metric that goes into promotions is the area of impact. By giving a talk to the guild, you can have impact on a group that's larger than your team, potentially the whole organization. It counts. But another reward is the positive feedback, interesting discussions and new connections that you make through this.
- mochii 6y agoVery interesting read! Thank you for sharing.
- gregkerzhner 6y agoInteresting article. We use a similar approach for our mobile apps to allow multiple teams to develop their own modules independently. Can anyone speak to what the advantages and disadvantages to such an approach are as opposed to going full Kubernetes / Microservices? Is it that deploys are riskier and you can't scale separate pieces independently?
- joelbluminator 6y ago2.8 million lines , 100 billion business. Rails can scale.
- mandelbrotwurst 6y ago100 billion? That seems like a lot of businesses per capita!
- tgarv 6y agoI think "100 billion business" means that the business (Shopify) is valued at $100 billion. (I'm not sure if that's true, that's just how I interpreted it)
- shwoopdiwoop 6y agoFairly certain GP referred to the market cap, not the number of businesses on Shopify's platform.
- khendron 6y agoHe might be referring to Shopify GMV (Gross Merchandise Volume —the value of commerce facilitated by the platform), which is probably approaching $100B per year.
- csomar 6y agoNo shopify market cap is $100bn; which is much higher than I expected. So I looked up their revenue, which is $1.6bn and they have an income deficit. so...
- mandelbrotwurst 6y agoAh, yeah I thought it said businesses plural.
- jtsiskin 6y ago“business”, not “businesses”
- ryanmarsh 6y agoThere’s so much truth in this. It’s full of lessons I tell clients at the outset of similar endeavors yet they often do not heed until they experience the pain first hand.
- kawsper 6y agoDoes anyone know if the Storefront rendering described here[0] is running Rails or something else? [0] https://engineering.shopify.com/blogs/engineering/how-shopify-reduced-storefront-response-times-rewrite https://engineering.shopify.com/blogs/engineering/how-shopif...
- rafaelfranca 6y agoThe application is a Rack application reusing some of the components of Rails, but it is not a conventional Rails application given it doesn't need most of the framework.
- treis 6y agoHave y'all seen any issues around autoloading of classes/modules in development? I've been working on a rails app composed of a handful of engines and I've noticed that every so often classes aren't loaded. 6 seems to be a lot better about it than 5 was.
- mhoad 6y agoRails 6 has a totally new code loader that was built specifically to address those issues called Zeitwerk. Some details here if you're interested https://blog.bigbinary.com/2019/10/08/rails-6-introduces-new-code-loader-called-zeitwerk.html https://blog.bigbinary.com/2019/10/08/rails-6-introduces-new...
- leafboi 6y agoI think it's kind of bad that we have this trend to use hardware to enforce modularity. If it's a performance issue, sure break it up into more hardware. If it's just code modularity than by shifting to microservices you are adding additional complexity of maintaining multiple services on top of modularizing the system. In short it's overkill. This whole thing about using hardware to enforce "developer behavior" is stupid. You can use software to enforce developer behavior. Your operating system, your programming language is already "enforcing" developer behavior. Additionally, your microservices are hard lines of modularization. It is very hard to change a module once it's been materialized because it's hardware. If you think about it, almost all lack of modularity comes from shared mutable variables. Segregate mutability away from the core logic of your system and the smallest function in your architecture will become as modular as a microservice. Really, any function that is stateless can be moved anywhere at anytime and used anywhere without fear of it being creating a permanent foothold in the architectural complexity of the system. So if the code is getting to structured where you become afraid of moving things... do this rather than go to microservices. >We can more easily onboard new developers to just the parts immediately relevant to them, instead of the whole monolith. Correct me if I'm wrong but don't folders and files and repos do this? Does this make sense to you that it has to be broken down into hardware? >Instead of running the test suite on the whole application, we can run it on the smaller subset of components affected by a change, making the test suite faster and more stable. Right because software could never do this in the first place. In order to test a quarter of my program in an isolated environment I have to move that quarter of my program onto a whole new computer. Makes sense. >Instead of worrying about the impact on parts of the system we know less well, we can change a component freely as long as we’re keeping its existing contracts intact, cutting down on feature implementation time. Makes sense because software contracts only exist as http json/graphql/grpc apis. The below code isn't a software contract it's only how old people do things: int add(x: int, y: int) Remember as long as that add function doesn't mutate shared state you know it has zero impact on any part of the system other than it's output... you can replace it or copy it or use it anywhere.... this is really all you need to do to improve modularity of your system. Editing it on the other hand could have some issues. There are other ways to deal with this and simply copying the function, renaming and editing it is still a good solution. But for some reason people think the only way to deal with these problems is to put an entire computer around it as a wall. So whenever I need some utility function that's located on another system I have to basically copy it over (along with a million other dependencies) onto my system and rename it... wait a minute can't I do that anyway (without copying dependencies) if it was located in the same system? >Again and again we pondered: How should components call each other? I think this is what's tripping most people up. They think DI IOC and OOP patterns are how you improve modularity. It's not. Immutable functions are what improves modularity of your program. The more immutable functions you have and the smaller they are the more modular your program will be. Segregate IO and mutations into tiny auxiliary functions away from your core logic which is composed of pure immutable functions. That's really the only pattern you need to follow and some languages can enforce this pattern without the need of "hardware." >Circular dependencies are situations where for example component A depends on component B but component B also depends on component A. I've never seen circular dependencies happen with pure functions. It's rare in practice. I think it occurs with objects because when you want one method of an object you have to instantiate that object which has a bunch of other methods and in turn dependencies that could be circular to the current object you're trying to call it from. In essence this kind of thing tends to happen with exclusively with objects. Don't group one function with the instantiation of other functions and you'll be fine. Still I've seen this issue occur with namespacing when you import files. Hardware isn't going to segregate this from happening. You need to structure your dependencies as a tree.
- lmarcos 6y agoGreat article. Main takeaway: microservices is not the only option when managing big codebases. In a parallel universe I imagine that the coolest trend in software development right now is a tool for monoliths: all code in a single repo, independent deployable components, contracts in the boundaries and mockable dependant components where needed. As opposed to our universe in which building microservices is the non-official way to go.
- hliyan 6y agoI often find myself saying "never do at runtime what could be done at compile time".
- exterm 6y agoyou don't work with Ruby eh? :D
- WJW 6y agoThe lack of compile time in the ruby world really makes it difficult to do a lot of work there. :P There's a nice Ruby trick btw where you put significant precalculations in constants, since the value of a constant gets computed during program startup it still allow you to do work "up front" instead of during a web request.
- mhoad 6y agoI never knew that but that IS a cool trick.
- owyn 6y agoYep, that's a good trick. At a previous PHP shop we had a large amount of static XML configuration (well it was generated, but not that often). Converting it all to PHP arrays and including it was significantly faster than parsing the XML on each request, and then PHP would cache that result too. Re-running the XML->PHP tool just caused it to re-include/cache these giant arrays of static config. It worked great. I mean, arguments about whether that was a good design or not aside... (edit to reply since I can't reply to a reply to a reply) Yep, it is very common in lisp/smalltalk environments to dump the state of the world to disk and re-load it later. This is one of those tricks that gets relearned every generation. :) For bonus credit apply this analogy to docker images. :)
- bori 6y agoI like that they completely dodged the term "microservice" in the whole post.
- exterm 6y agoyou should read the first post in the series if you want to read about microservices. https://engineering.shopify.com/blogs/engineering/deconstructing-monolith-designing-software-maximizes-developer-productivity https://engineering.shopify.com/blogs/engineering/deconstruc...
- throwaway691999 6y agoI think it's kind of bad that we have this trend to use "walls" to enforce modularity. This whole thing about using "walls" to enforce "developer behavior" is, in my humble opinion, the wrong direction. If you think about it, almost all lack of modularity comes from shared mutable variables. Segregate mutability away from the core logic of your system and the smallest function in your architecture will become as modular as a microservice. Really, any function that is stateless can be moved anywhere at anytime and used anywhere without fear of it being creating a permanent foothold in the architectural complexity of the system. So if the code is getting to structured where you become afraid of moving things... do this rather than build classes and walls around all your subroutines. Remember as long as that add function doesn't mutate shared state you know it has zero impact on any part of the system other than it's output... you can replace it or copy it or use it anywhere.... this is really all you need to do to improve modularity of your system. >Again and again we pondered: How should components call each other? I think this is what's tripping most people up. They think DI IOC and OOP patterns are how you improve modularity. It's not. Immutable functions are what improves modularity of your program. The more immutable functions you have and the smaller they are the more modular your program will be. Segregate IO and mutations into tiny auxiliary functions away from your core logic which is composed of pure immutable functions. >Circular dependencies are situations where for example component A depends on component B but component B also depends on component A. I've never seen circular dependencies happen with pure functions. It's rare in practice. I think it occurs with objects because when you want one method of an object you have to instantiate that object which has a bunch of other methods and in turn dependencies that could be circular to the current object you're trying to call it from. In essence this kind of thing tends to happen because when you call a method you're actually calling a group of methods and state within a class and upon all those dependencies as well increasing the chances of a circular dependency. Still I've seen this issue occur with namespacing when you import files. Walls aren't going to segregate this from happening. You need to structure your dependencies as a tree.
- lmm 6y ago> Really, any function that is stateless can be moved anywhere at anytime and used anywhere without fear of it being creating a permanent foothold in the architectural complexity of the system. That's not really true. A pure function can still be coupled to a particular internal data representation. It can still assume particular invariants that you may not want to maintain. Namespacing functions together with the data structures they operate on is still a good idea, and helps with keeping a coherent model at each level - e.g. if your business logic is calling a function that's about the specific mechanics of encoding data for Redis, you're probably using the wrong abstraction. Pushing mutability to the edges is good and useful but it's not the be-all and end-all of decoupling. Enforced walls are a much better idea than spending your discipline budget on maintaining decoupling by hand. A lot of the time a pure function can actually be decoupled completely from the datatypes it's operating on by using parametricity (and maybe a standard typeclass that the datatype it operates on conforms to), but you may not notice that unless you've got some module boundaries that nudge you to think about that kind of thing.
- ToJans 6y agoI can imagine that this has been a huge effort, and kudos to the team, but this is a solved problem; there are ample methodologies to resolve the big ball of mud. IMHO the shopify team could have saved a lot of time by getting some schooling about strategic DDD, and consulting one or more DDD experts to draw a first version of their context map.
- yeswecatan 6y agoDo you know any DDD experts that offer consulting services?
- banq 6y agoDDD aggrgates: loose coupling with high cohesion!
- banq 6y agoin Shopify, they actually applied DDD bounded context and aggregate ,but they maybe don't know DDD!