14 ms·

Great article. Main takeaway: microservices is not the only option when managing big codebases. In a parallel universe I imagine that the coolest trend in softw

Great article. Main takeaway: microservices is not the only option when managing big codebases. In a parallel universe I imagine that the coolest trend in software development right now is a tool for monoliths: all code in a single repo, independent deployable components, contracts in the boundaries and mockable dependant components where needed. As opposed to our universe in which building microservices is the non-official way to go.

hliyan 6y ago
I often find myself saying "never do at runtime what could be done at compile time".
- exterm 6y ago
  you don't work with Ruby eh? :D
  - WJW 6y ago
    The lack of compile time in the ruby world really makes it difficult to do a lot of work there. :P There's a nice Ruby trick btw where you put significant precalculations in constants, since the value of a constant gets computed during program startup it still allow you to do work "up front" instead of during a web request.
    - mhoad 6y ago
      I never knew that but that IS a cool trick.
      - owyn 6y ago
        Yep, that's a good trick. At a previous PHP shop we had a large amount of static XML configuration (well it was generated, but not that often). Converting it all to PHP arrays and including it was significantly faster than parsing the XML on each request, and then PHP would cache that result too. Re-running the XML->PHP tool just caused it to re-include/cache these giant arrays of static config. It worked great. I mean, arguments about whether that was a good design or not aside... (edit to reply since I can't reply to a reply to a reply) Yep, it is very common in lisp/smalltalk environments to dump the state of the world to disk and re-load it later. This is one of those tricks that gets relearned every generation. :) For bonus credit apply this analogy to docker images. :)
        im3w1l 6y ago
        I heard emacs did that but went one step further, they dump the memory of the interpreter post-init* and just load it into memory when starting. * Some early step in the init process. May things are still interpreted at init.
        im3w1l 6y ago
        I heard emacs did that but went one step further, they dump the memory of the interpreter post-init* and just load it into memory when starting. * Some early step in the init process. Many things are still interpreted at init.
        andreareina 6y ago
        "emacs unexec" is the search term if you're looking for it. https://news.ycombinator.com/item?id=21394916 https://news.ycombinator.com/item?id=21394916 https://lwn.net/Articles/707615/ https://lwn.net/Articles/707615/ "The Emacs dumper dispute" https://lwn.net/Articles/673724/ https://lwn.net/Articles/673724/ "Removing support for Emacs unexec from Glibc"
        JohnBooty 6y ago
        I did this in PHP once as well. I had to code a coupon lookup site where people entered coupon codes and they were verified against a database. I forget how many coupons there were... pretty sure it was less than 100,000. Anyway, I coded it up in my local dev environment. Unfortunately, it turned out that I'd been mislead and the actual deployment environment didn't have a database server available. In desperation and facing a deadline, I dumped all the lookup values into an array in a PHP file. As you said, it was really quite performant. The first request after starting the server was a bit slow (but not too bad... still < 10 seconds I think) and after that things were golden. I felt a bit dirty, but things worked and we got paid.
      - skipants 6y ago
        I just want to caveat this as it is not a Ruby construct, it's part of Ruby web servers. Because they are long-living Ruby processes they are only loading files once (I suppose it's similar to compiling). This means it runs all globally-scoped code, which class definitions and constants (generally) are. That's actually what Ruby bootloaders like Spring and Zeus are doing on your dev machine to speed up the load time when you use Rails commands. They cache all that globally run stuff in their own process. It's also why they run into a bunch of issues when you have logic in your constant definitions.
      - aantix 6y ago
        In Ruby, the class definition is code as well.
        imhoguy 6y ago
        Where they are not code? I think you meant that Ruby class is defined at runtime with sequential imperative or functional code. Ha! You can build class from Ruby code in a string too. A lot of choice.
      - shawnz 6y ago
        What? Don't global variables exist in pretty much every language on the planet? This isn't a "trick", it is a bad practice which should be avoided in any language. Imagine describing globals in C, Python, or JavaScript, or static fields in Java or C# as a "neat trick"...
JamesSwift 6y ago
And I will call that parallel universe: monorepo-verse
ChiefOBrien 6y ago
Reminds me of OSGi. Great idea, poor adoption, mostly due to the complexity of the problem domain. Microservices are a lot worse in that regard, yet remain a whole lot more popular, sadly.
etaioinshrdlu 6y ago
This is basically what I do and it's great :)
jakobmartz3 6y ago
interesting thought!
didibus 6y ago
Ignoring some of the deployments and dependencies related aspects of microservices Vs monolith, one aspect that has me convinced against my own ideals is that a micro-service has a "strong" boundary in that it is actually difficult and effortful for a developer to cross it. This in turn has a positive effect in maintaining proper boundary and putting the right amount of thought about the interfaces and responssability of each component.
- exterm 6y ago
  It's all tradeoffs. You get a stronger boundary, but you also get a distributed system. Also, the first try of drawing boundaries will always be varying degrees of wrong. If you have very strong boundaries at this stage, iterating on them, moving responsibilities around, can be harder. Also, with the right tooling it's definitely possible to harden monolith internal boundaries to a comparable level. I can see though how many smaller companies would not be in a position to build that tooling. Anyway... there is no either / or here, as I've explained in another comment. What if you have components within a monolith, but each component has its own database, for example? What if test suites are completely isolated, so that tests for component A can not access code in component B? You can get pretty strong boundaries with a few comparably simple tricks.
- lmm 6y ago
  A decent module system can achieve the same thing without all of the network overhead. Even just something like maven multi-module projects makes it hard enough to accidentally cross the boundaries.
- yeswecatan 6y ago
  Agreed 100%.
gen220 6y ago
What you describe is microservices developed in a monorepo, and a lot of companies (including the one I currently work at) have gone this route. Some people might disagree, but imo the cult of microservices does not require 1 repo per microservice. The tools you describe are build-graph management tools (bazel pants buck etc) and rpc tools (gRPC + protobufs, cap'n proto) and they are indeed pretty cool, albeit to a niche crowd.
- djohnston 6y ago
  I agree 100%. It gives you the boundaries but also the whole world maps to a single revision in VCS
- jurre 6y ago
  I think the key difference here is that there is no network in between components in a componentized monolith, each component runs the entire “monorepo”
  - dodobirdlord 6y ago
    Whether there’s actually network between components is something a platform team can handle based on their best judgement. Having collections of containers that always run together is a common pattern.
    - pbourke 6y ago
      Certainly, but such a system is not a monolith. A core trait of the monolith is that there are no network calls between components.
      - deleted 6y ago
        [deleted]
      - inopinatus 6y ago
        This negative-space definition of "monolith" is unhelpful to the point of irrelevance. It's unreasonable, in the sense that adopting it gives us nothing to reason about, as with the comment above. By such a standard the last monolithic in-service system was a Burroughs mainframe ca. 1975. I've got statically linked binaries that would fail this definition. Even the plainest Rails application depends on network traffic, including to communicate with parts of itself. It cannot function without an operating system, which is also talking to parts of itself via network protocols, and this runs on a server whose internal buses are themselves a distributed system. It's networks, all the way down, and a heads-in-the-sand attitude doesn't help us reason about performance, reliability, scalability, maintainability et cetera. Put this in a "Falsehoods programmers believe ...": calling a stateless function in a stack-based language to compute an immutable result won't lead to a network call. Monolithic applications are defined by something they are, not something they don't do, and what they are is a single unit of code for development and deployment purposes that includes everything necessary to fulfil an entire system's purpose. The issue of intentionally crossing a network boundary, and when, and why, is an dependent topic in comparative systems architecture, but it's analytically orthogonal.
jackbravo 6y ago
That's what tools like https://github.com/nrwl/nx https://github.com/nrwl/nx try to facilitate. Mainly facilitating separating the code in boundaries like backend, frontend, and common libraries. I haven't really used nx myself :-p. But I would love to use something similar for other frameworks or languages.

hliyan 6y ago

I often find myself saying "never do at runtime what could be done at compile time".

exterm 6y ago
you don't work with Ruby eh? :D
- WJW 6y ago
  The lack of compile time in the ruby world really makes it difficult to do a lot of work there. :P There's a nice Ruby trick btw where you put significant precalculations in constants, since the value of a constant gets computed during program startup it still allow you to do work "up front" instead of during a web request.
  - mhoad 6y ago
    I never knew that but that IS a cool trick.
    - owyn 6y ago
      Yep, that's a good trick. At a previous PHP shop we had a large amount of static XML configuration (well it was generated, but not that often). Converting it all to PHP arrays and including it was significantly faster than parsing the XML on each request, and then PHP would cache that result too. Re-running the XML->PHP tool just caused it to re-include/cache these giant arrays of static config. It worked great. I mean, arguments about whether that was a good design or not aside... (edit to reply since I can't reply to a reply to a reply) Yep, it is very common in lisp/smalltalk environments to dump the state of the world to disk and re-load it later. This is one of those tricks that gets relearned every generation. :) For bonus credit apply this analogy to docker images. :)
      - im3w1l 6y ago
        I heard emacs did that but went one step further, they dump the memory of the interpreter post-init* and just load it into memory when starting. * Some early step in the init process. May things are still interpreted at init.
      - im3w1l 6y ago
        I heard emacs did that but went one step further, they dump the memory of the interpreter post-init* and just load it into memory when starting. * Some early step in the init process. Many things are still interpreted at init.
        andreareina 6y ago
        "emacs unexec" is the search term if you're looking for it. https://news.ycombinator.com/item?id=21394916 https://news.ycombinator.com/item?id=21394916 https://lwn.net/Articles/707615/ https://lwn.net/Articles/707615/ "The Emacs dumper dispute" https://lwn.net/Articles/673724/ https://lwn.net/Articles/673724/ "Removing support for Emacs unexec from Glibc"
      - JohnBooty 6y ago
        I did this in PHP once as well. I had to code a coupon lookup site where people entered coupon codes and they were verified against a database. I forget how many coupons there were... pretty sure it was less than 100,000. Anyway, I coded it up in my local dev environment. Unfortunately, it turned out that I'd been mislead and the actual deployment environment didn't have a database server available. In desperation and facing a deadline, I dumped all the lookup values into an array in a PHP file. As you said, it was really quite performant. The first request after starting the server was a bit slow (but not too bad... still < 10 seconds I think) and after that things were golden. I felt a bit dirty, but things worked and we got paid.
    - skipants 6y ago
      I just want to caveat this as it is not a Ruby construct, it's part of Ruby web servers. Because they are long-living Ruby processes they are only loading files once (I suppose it's similar to compiling). This means it runs all globally-scoped code, which class definitions and constants (generally) are. That's actually what Ruby bootloaders like Spring and Zeus are doing on your dev machine to speed up the load time when you use Rails commands. They cache all that globally run stuff in their own process. It's also why they run into a bunch of issues when you have logic in your constant definitions.
    - aantix 6y ago
      In Ruby, the class definition is code as well.
      - imhoguy 6y ago
        Where they are not code? I think you meant that Ruby class is defined at runtime with sequential imperative or functional code. Ha! You can build class from Ruby code in a string too. A lot of choice.
    - shawnz 6y ago
      What? Don't global variables exist in pretty much every language on the planet? This isn't a "trick", it is a bad practice which should be avoided in any language. Imagine describing globals in C, Python, or JavaScript, or static fields in Java or C# as a "neat trick"...

JamesSwift 6y ago

And I will call that parallel universe: monorepo-verse

ChiefOBrien 6y ago

Reminds me of OSGi. Great idea, poor adoption, mostly due to the complexity of the problem domain. Microservices are a lot worse in that regard, yet remain a whole lot more popular, sadly.

etaioinshrdlu 6y ago

This is basically what I do and it's great :)

jakobmartz3 6y ago

interesting thought!

didibus 6y ago

Ignoring some of the deployments and dependencies related aspects of microservices Vs monolith, one aspect that has me convinced against my own ideals is that a micro-service has a "strong" boundary in that it is actually difficult and effortful for a developer to cross it. This in turn has a positive effect in maintaining proper boundary and putting the right amount of thought about the interfaces and responssability of each component.

exterm 6y ago
It's all tradeoffs. You get a stronger boundary, but you also get a distributed system. Also, the first try of drawing boundaries will always be varying degrees of wrong. If you have very strong boundaries at this stage, iterating on them, moving responsibilities around, can be harder. Also, with the right tooling it's definitely possible to harden monolith internal boundaries to a comparable level. I can see though how many smaller companies would not be in a position to build that tooling. Anyway... there is no either / or here, as I've explained in another comment. What if you have components within a monolith, but each component has its own database, for example? What if test suites are completely isolated, so that tests for component A can not access code in component B? You can get pretty strong boundaries with a few comparably simple tricks.
lmm 6y ago
A decent module system can achieve the same thing without all of the network overhead. Even just something like maven multi-module projects makes it hard enough to accidentally cross the boundaries.
yeswecatan 6y ago
Agreed 100%.

gen220 6y ago

What you describe is microservices developed in a monorepo, and a lot of companies (including the one I currently work at) have gone this route. Some people might disagree, but imo the cult of microservices does not require 1 repo per microservice. The tools you describe are build-graph management tools (bazel pants buck etc) and rpc tools (gRPC + protobufs, cap'n proto) and they are indeed pretty cool, albeit to a niche crowd.

djohnston 6y ago
I agree 100%. It gives you the boundaries but also the whole world maps to a single revision in VCS
jurre 6y ago
I think the key difference here is that there is no network in between components in a componentized monolith, each component runs the entire “monorepo”
- dodobirdlord 6y ago
  Whether there’s actually network between components is something a platform team can handle based on their best judgement. Having collections of containers that always run together is a common pattern.
  - pbourke 6y ago
    Certainly, but such a system is not a monolith. A core trait of the monolith is that there are no network calls between components.
    - deleted 6y ago
      [deleted]
    - inopinatus 6y ago
      This negative-space definition of "monolith" is unhelpful to the point of irrelevance. It's unreasonable, in the sense that adopting it gives us nothing to reason about, as with the comment above. By such a standard the last monolithic in-service system was a Burroughs mainframe ca. 1975. I've got statically linked binaries that would fail this definition. Even the plainest Rails application depends on network traffic, including to communicate with parts of itself. It cannot function without an operating system, which is also talking to parts of itself via network protocols, and this runs on a server whose internal buses are themselves a distributed system. It's networks, all the way down, and a heads-in-the-sand attitude doesn't help us reason about performance, reliability, scalability, maintainability et cetera. Put this in a "Falsehoods programmers believe ...": calling a stateless function in a stack-based language to compute an immutable result won't lead to a network call. Monolithic applications are defined by something they are, not something they don't do, and what they are is a single unit of code for development and deployment purposes that includes everything necessary to fulfil an entire system's purpose. The issue of intentionally crossing a network boundary, and when, and why, is an dependent topic in comparative systems architecture, but it's analytically orthogonal.

jackbravo 6y ago

That's what tools like https://github.com/nrwl/nx https://github.com/nrwl/nx try to facilitate. Mainly facilitating separating the code in boundaries like backend, frontend, and common libraries. I haven't really used nx myself :-p. But I would love to use something similar for other frameworks or languages.