10 ms·

The New Three-Tier Application

81 points by qianli_cs 2y ago

mmastrac 2y ago
Company hawking an orchestrating backend server says you should use an orchestrating backend server? You still have four layers, it's just that one is hidden with annotations.
- tomhallett 2y ago
  When they get to what their implementation is, I’m not even sure what it is. Like how is the following different from a library, like acts-as_state_machine (https://github.com/aasm/aasm https://github.com/aasm/aasm). Are they auto running the retries - in a background job which they handle (like “serverless” background job)? “Implementing orchestration in a library connected to a database means you can eliminate the orchestration tier, pushing its functionality into the application tier (the library instruments your program) and the database tier (your workflow state is persisted to Postgres).“
  - jedberg 2y ago
    > Are they auto running the retries - in a background job which they handle (like “serverless” background job)? Yes. The library makes sure that your app retries failed workflows. Or if you use the commercial products, takes care of that for you.
recursivedoubts 2y ago
Every day we stray further from God’s light.
- baq 2y ago
  To err is human
  - chrisweekly 2y ago
    To forgive, divine
  - floathub 2y ago
    To iterate is human.
    - bryanrasmussen 2y ago
      to recurse, devilish.
      - falcor84 2y ago
        What about triple mutual recursion? That must have a spark of the divine?
        bryanrasmussen 2y ago
        if you follow the doctrine of the trinity - sure.
  - mtillman 2y ago
    This takes me back. http://errtheblog.com/ http://errtheblog.com/
  - falcor84 2y ago
    My favorite take on this is: To err is human; to bring down the whole cluster with one bad command is DevOps.
- falcor84 2y ago
  Pray tell, what software architecture does God's light shine brightest on?
  - infinitezest 2y ago
    The one you're not currently using, of course.
  - pphysch 2y ago
    The GP created HTMX, so possibly he would argue for an even simpler 2-tier or 2.5-tier architecture that addresses frontend complexity
    - dventimi 2y ago
      That's 1 to 1.5 tiers too many: https://docs.postgrest.org/en/v12/how-tos/providing-html-content-using-htmx.html https://docs.postgrest.org/en/v12/how-tos/providing-html-con...
dartos 2y ago
How often do the trades ask “what’s next?”
Trasmatta 2y ago
> At the technical and organizational scale of modern enterprises, the complexity of orchestrating distributed systems is unavoidable. *citation needed We continue to make things much more complex than they need to be. Even better when NON "enterprise" applications also buy into the insane complexity because they feel like they have to (but they have nowhere near the resources to manage that complexity).
- nlitened 2y ago
  > *citation needed You are already citing the person, do you really need that person to cite another person in order to form an opinion?
  - Trasmatta 2y ago
    ? What I'm asking for is evidence that this complexity is "unavoidable".
- viraptor 2y ago
  That would be difficult (impossible?) to prove. But if the claim is not true, a much easier thing would be to show an example of a large enterprise which did not introduce distributed processing. Is there one? I've not heard of that. Even basics like auth eventually require SSO if you want to preserve your sanity, and that's a distributed system.
politelemon 2y ago
I haven't noticed the same trend or evolution of application tiers, perhaps we live in different echo chambers. Teams using microsevices need to evaluate whether it's still a good fit considering the inherent overhead it brings. Applying a bandaid solution on top of it, if it isn't a good fit, only makes the problem worse.
- nkmnz 2y ago
  I think the term "microservice" is useless here. It doesn't matter if you run your backend logic in a monolith or in some complex microservice architecture, because both will depend on external runtime dependencies. Especially the smallest of startups will rely heavily on external APIs to connect their monolith (which is, in enterprise terms, probably a single microservice) to external services such as stripe, to some product analytics tool, to a CRM, to openAI etc. etc. pp. I don't know anything about this post's solution, but if it delivers on the idea to not having to worry that much about failed calls to 3rd parties (or even my own database!), I'd like it a lot. Why would you call that a bandaid solution?
turnsout 2y ago
So basically front end, back end and data.
itsthecourier 2y ago
bought a ten years old company, a division of a public company, some million dollars. got an overly complex, over 30 micro services architecture, over usd20k in monthly cloud fees. rewrote the thing into a monolith in 6 months. reduced development team in half, costs of servers by 80-90%, latency by over 60% newer is not better. each micro service must be born from a real necessity out of usage stats, server stats, cost analisis. not by default following tutorials.
- brookst 2y ago
  It’s telling that you revised both application architecture and org structure to be simpler and more efficient. Microservices are sometimes a reflection of the org; the separation of concerns is about ensuring everyone knows who’s working on what, and enforcing that in the tech. (Not defending that, it’s often inefficient and can be a straight jacket that constrains the product and org)
  - smallnix 2y ago
    +1 matches my experience, https://en.m.wikipedia.org/wiki/Conway%27s_law https://en.m.wikipedia.org/wiki/Conway%27s_law
    - sunrunner 2y ago
      Also known as the only unbreakable law (https://www.youtube.com/watch?v=5IUj1EZwpJY https://www.youtube.com/watch?v=5IUj1EZwpJY), a fact which feels deeply depressing when you dig into it.
- wavemode 2y ago
  and then you renamed the company to X?
- KronisLV 2y ago
  I've seen the opposite: single monolithic codebase, where the different bits of functionality eventually end up tightly coupled, so it's actually pretty difficult to extract bits into a separate service later, so a different type of architecture isn't possible even if you wanted to split it up. Why do that? Well, when a big Excel file is uploaded to import a bunch of data, or when some reports are generated, or when a crapton of emails is being sent, or when batch processes are sending data over to another system, both the API and the UI become slow for everyone. Scale it vertically, would be the first thought - for a plethora of reasons, that doesn't work. There are bottlenecks in the DB thread pool solution, there are bottlenecks in the HTTP request processing, there are bottlenecks all over the place that can be resolved (for example, replacing HikariCP with DBCP2, oddly enough) but each fix takes a bunch of time and it's anyone's guess whether something will break. Suddenly updating the dependencies of the monolith is also a mess, something like bumping the runtime version also leads to all sorts of things breaking, sometimes at compile time, other times at runtime (which leads to out of date packages needing to be used). Definitionally, a big ball of mud. Can you just "build better software" or "write code without bugs"? Well, yes, but no. I've seen plenty of cases of microservices also becoming a chatty mess, but what strikes me as odd is that people don't attempt to go for something like the following: * keep the business functionality, whatever that may be, in one central service as much as possible * instead of chopping up the domain model, extract functionality that pertains to specific types of mechanisms or workloads (e.g. batch processing, file uploads or processing, data import etc.) into separate services, even if some of it might still use the main DB Yet, usually it's either a mess due to shoving too many things into one codebase, or a mess due to creating too much complexity by attempting to have a service for every small set of entities in your project ("user service", "order service", ...).
recroad 2y ago
I like building one-tier applications in Elixir.
deleted 2y ago
[deleted]
dventimi 2y ago
"In the beginning (that is, the 90’s), developers created the three-tier application. Per Martin Fowler, these tiers were the data source tier, managing persistent data, the domain tier, implementing the application’s primary business logic, and the presentation tier, handling the interaction between the user and the software. The motivation for this separation is as relevant today as it was then: to improve modularity and allow different components of the system to be developed relatively independently." Immediately, I see problems. Martin Fowler's "Patterns of Enterprise Application Architecture" was first published in 2002, a year that I think most people will agree was not in "the 90's." Also, was that the motivation? Are we sure? Who had that motivation? Were there any other motivations at play?
- edoceo 2y ago
  Well, Martin's book came out after we were doing these patterns in the 90s. My teams had that motivation - data worked with logic; logic worked with UI teams. Separation of concerns and division of labour are, generally, good ideas. ETA: one of the groups that was motivated was MS: use SQL Server + SP ; then COM in the Logic layer and then ASP in the UI.
  - jmull 2y ago
    Yes. I was happy when Fowler came out because we could all start using the same terminologies for the same things, and work from common concepts when solving the same problem. (It didn't work out that way, though. It seemed like most people used Fowler as some kind of bible or ending point, when it should have been a starting point/ source of inspiration. Somehow it seemed to turn people's brains off, making them dumber and less insightful about the systems they were building.)
  - tomnipotent 2y ago
    > one of the groups that was motivated was MS I remember Microsoft being a huge marketing proponent of the 3-tier architecture in the late 90's, particularly after the release of ASP. The model was promoted everywhere - MSDN, books, blogs, conferences. At this point COM was out of the picture and ASP served as both front-end (serving HTML) and back-end (handling server responses).
ape4 2y ago
Just mentioning MVC (Model-View-Controller) https://developer.mozilla.org/en-US/docs/Glossary/MVC https://developer.mozilla.org/en-US/docs/Glossary/MVC
xyst 2y ago
_Maybe_ something like this is needed at GOOG or NFLX. But for most companies, this added complexity of an “orchestration tier” is unnecessary.
davedx 2y ago
I think people in this industry make using complicated, powerful paradigms part of their identity. They don’t feel like they’re important unless they’re reaching for N-tier architecture or exotic databases or lambdas or whatever else it is. Most apps I’ve worked on could have been a monolith on postgres but they never ever are as soon as I’m not the sole engineer.
- ebiester 2y ago
  The architecture is the function of number of people in the system. How do you manage 100 people in a monolith? 250? What if one group gets to a broken state but another group needs to release to escape a broken state? Architecture is often solving a human problem. That said, too many teams break out way too early.
  - lucianbr 2y ago
    Often the way they attepmt to manage 100 people is to split the monolith into a distributed monolith. Now you have all the same problems plus some new ones, but hey, we're "managing" the human problem. And considering they somehow muddle along, with one person sometimes breaking everything for the other 99, and all the other problems, I think they could very well muddle along with a monolith. With 100 or however many programmers. Yes, the distributed system with well thought out splits into services would be an improvement. But it's clearly not a necessity. So it remains that some places at least, use it out of some other reason - fad, cargo culting, whatever. Architecture should be solving the human or other problems, definitely. But how often it does... I guess each with their own experience.
  - davedx 2y ago
    I’ve heard this and at some large engineering orgs, I’ve even occasionally seen it applied sensibly, but IME irrational hype driven architecture dominates: 15 microservices at companies with 5 engineers. Event sourcing where it made no sense. Apache Spark used on a project where all the data and processing fit on a laptop. Orchestration layers; frontends for backends; queues everywhere; the list is endless.
mrkeen 2y ago
> By persisting execution state to a database, a lightweight library can fulfill the primary goal of an orchestration system: guaranteeing code executes correctly despite failures. If a program fails, the library can look up its state in Postgres to figure out what step to take next, retrying transient issues and recovering interrupted executions from their last completed step. program = email all customers failure = throttled by mailchimp
- jedberg 2y ago
  Your program would gracefully handle this though, because the workflow would fail and it would retry. This requires you to write your program in a way that does not trigger mailchimp's throttling -- a problem that would happen no matter how you write your app and a problem you have to deal with no matter how run your app.
geophile 2y ago
Orchestration tier. Oy. So something goes wrong, and you need to back out an update to one of your microservices. But that back-out attempt goes wrong. Or happens after real-world actions have been based on that update you need to back out. Or the problem that caused a backout was transient, everything turns out to be fine, but now your backout is making its way across the microservices. Backout the backout? What if that goes wrong? The "or"s never end. Just use a centralized relational database, use transactions, and be done with it. People not understanding what can go wrong, and how RDB transactions can deal with a vast subset of those problems -- that's like the 21st century version of how to safely use memory in C. Yes, of course, centralized RDBs with transactions are sometimes the wrong answer, due to scale, or genuinely non-atomic update requirements, or transactions spanning multiple existing systems. But I have the sense that they are often rejected for nonsensical reasons, or not even considered at all.
- shermantanktop 2y ago
  I mostly agree. But I work at a place where scale precludes that, and as a result relational concepts are sneered at. It turns out that pulling atomicity concerns into a pile of Java code leads to consistency problems…
  - geophile 2y ago
    > relational concepts are sneered at That is remarkably dumb. Don't use an RDB if it doesn't fit your requirements. But "relational concepts" covers a lot of valuable ground, and rejecting them out of hand is a combination of ignorant and dumb. Listen kids: Codd's 1970 paper on the relational model of data was absolutely revolutionary. Data processing involved low-level navigation of records. Separation of logical concerns from physical concerns (like 80-column cards) was regarded as crazy. Codd's message was this: a set-oriented data model, manipulated by a high-level, non-procedural language was the right way to process data. And yes, a lot of practical details need to be worked out, but it's still right. And then followed decades of research into the implementation of relational database systems, which gave us working systems that fully realized that set of ideas. It was an incredibly bold research program, which delivered, and gave us incredibly valuable technology such as: - SQL (love it or hate it, it's a mostly non-procedural high-level language that does a great job of data processing). - Query optimization. - Transactions, with levels of isolation and many different implementation approaches, most of which have proven practical at one time or another. Also, early work on distributed algorithms, e.g. two-phase commit. Ignore these "relational concepts", and you will be reinventing it, badly, and at great expense.
ptx 2y ago
> In the beginning (that is, the 90’s), developers created the three-tier application. [...] Of course, application architecture has evolved greatly since the 90's. [...] This complexity has created a new problem for application developers: how to coordinate operations in a distributed backend? For example: How to atomically perform a set of operations in multiple services, so that all happen or none do? This doesn't seem like a correct description of events. Distributed systems existed in the 90s and there was e.g. Microsoft Transaction Server [0] which was intended to do exactly this. It's not a new problem. And the article concludes: > This manages the complexity of a distributed world, bringing the complexity of a microservice RPC call or third-party API call closer to that of a regular function call. Ah, just like DCOM [1] then, just like in the 90s. [0] https://en.wikipedia.org/wiki/Microsoft_Transaction_Server https://en.wikipedia.org/wiki/Microsoft_Transaction_Server [1] https://en.wikipedia.org/wiki/Distributed_Component_Object_Model https://en.wikipedia.org/wiki/Distributed_Component_Object_M...
- smithkl42 2y ago
  I only ever played with DCOM and Transaction Server, and never in production, but I do wonder what about that tech stack made it so absolutely unworkable, and such a technological dead-end? Did anyone ever manage to make it work?
  - dventimi 2y ago
    What I remember is that there were social reasons, market reasons, and technical reasons that MTS didn't pan out. First, Microsoft was out-of-fashion in startup culture. Second, the exploding internet boom had little demand for distributed transactions. Third, COM was a proprietary technology that relied on C++ at a time when developers were flocking to easier memory-managed languages like Java, which was or at least was perceived to be more "open." I'm sure there were other reasons, but that's what looms in my mind.
deleted 2y ago
[deleted]
tnvmadhav 2y ago
i have something positive to say. I really liked the website UX on mobile
localghost3000 2y ago
It took me a bit to realize the author is selling me something. I guess good job there sir. I’ve built a bunch of distributed architectures. In every case I did, we would have been better served with a monolith architecture and a single relational DB like Postgres. In fact I’ve only worked on one system that had the kind of scale that would justify the additional complexity of a distributed architecture. Ironically that system was a monolith with Postgres.
- ambicapter 2y ago
  > In fact I’ve only worked on one system that had the kind of scale that would justify the additional complexity of a distributed architecture. Ironically that system was a monolith with Postgres. This...doesn't seem to support your case at all? Maybe if you'd turned all those distributed architectures into monoliths you would've then thought the distributed architecture was justified (since you have a 1 of 1 case where that was the case). I'm guessing the truth is somewhere in the middle, but unfortunately it's not very useful to the reader to say "well, some systems are better distributed, some systems are better as monolith". The interesting question is which is which.
  - localghost3000 2y ago
    > This...doesn't seem to support your case at all? Hm ok well I am not sure what you mean but its the internet so... <shrug> What I am saying here is that you would be shocked at how far you can get with a simpler architecture. Distributed systems have massive trade offs and are the kind of thing you shouldn't do unless you are FORCED to.
- szundi 2y ago
  Your experience suggest that monoliths on Postgres might be key to win on the market.
- anonzzzies 2y ago
  They are basically advocating that (postgres replaces everything else normally used) however they need to add in enterprisey stuff.
bazizbaziz 2y ago
Workflows/orchestration/reconciliation-loops are basically table stakes for any service that is solving significant problems for customers. You might think you don't need this, but when you start needing to run async jobs in response to customer requests, you will always eventually implement one of the above solutions. IMO the next big improvement in this space is improving the authoring experience. In short, when it comes to workflows, we are basically still writing assembly code. Writing workflows today is done in either a totally separate language (StepFunctions), function-level annotations (Temporal, DBOS, etc), or event/reconciliation loops that read state from the DB/queue. In all cases, devs must manually determine when state should be written back to the persistence layer. This adds a level of complexity most devs aren't used to and shouldn't have to reason about. Personally, I think the ideal here is writing code in any structure the language supports, and having the language runtime automatically persist program state at appropriate times. The runtime should understand when persistence is needed (i.e. which API calls are idempotent and for how long) and commit the intermediate state accordingly.
- junto 2y ago
  There seems to be a lot of negativity about this opinion, but I heartily agree with you. Anytime you’re dealing with large systems that have a multitude of external integrations you’re going to need some kind of orchestration. Anytime you perform a write operation, you cannot safely and idempotently do another IO operation in the same process, without risking a non-retryable exception event of the entire process. Most people when faced with problem will look at some kind of queuing abstraction. The message fails, and you try it automatically later. If you’re a masochist you’ll let it go in a dead letter queue and deal with it manually later. Sagas is one way to orchestrate this kind of system design. Routing slips is another that has the benefit of no central orchestrator and state is just carried in the routing slip. Both are adequate but in the end you’ll end up with a lot of infrastructure and architecture to make it work. Systems like Temporal take a lot of that pain away, allowing developers to focus on writing business code and not infrastructural and architectural code. So I am fully in on this new pattern for the horrible integrations I’m forced do deal with. Web services that are badly written RPC claiming to be REST, or poorly designed SOAP services. REST services that choose to make me to a GET request for the object I just created, because REST purists don’t return objects on creation, only location headers. Flaky web services that are routed over 2 VPN’s because that’s the way the infrastructure team decided to manage it. The worst cast I ever had to deal with was having to process XML instructions over email. And not as an attachment, I mean XML as text in the body of the email. Some people are just cruel. Give someone a greenfield and I’d agree, simplicity rules. But when you’re playing in someone else’s dirty sandpit, you’re always designing for the worst case failure. And for the readers that are still wondering why this matters, I recommend this video from 7 years ago called “six little lines of fail”. https://youtu.be/LGG3IIHUG_w https://youtu.be/LGG3IIHUG_w
Toine 2y ago
Was this written by an infamous 26 yo "senior" developer with 4 years of experience ?
- anonzzzies 2y ago
  Sure mate. https://en.wikipedia.org/wiki/Michael_Stonebraker https://en.wikipedia.org/wiki/Michael_Stonebraker
gizzlon 2y ago
This is pretty weak, makes very bold statements ("this is the way it has to be now") with no evidence. Reads like the set up for a sales pitch, which came at the end
whilenot-dev 2y ago
Following the Getting Started[0] section it seems like DBOS requires the configuration of a Postgres-compatible database[1] (NOTE: DBOS currently only supports Postgres-compatible databases.). Then, after decorating your application functions as workflow steps[2], you'll basically run those workflows by spawning a bunch of worker threads[3] next to your application process. Isn't that a bit... unoptimized? The orchestrator domain doesn't seem to be demanding on compute, so why aren't they making proper use of asyncio here in the first place? And why aren't they outsourcing their runtime to an independent process? EDIT: So "To manage this complexity, we believe that any good solution to the orchestration problem should combine the orchestration and application tiers." (from the article) means that your application runtime will also become the orchestrator for its own workflow steps. Is that a good solution? EDIT2: Are they effectively just shifting any uptime responsibility (delivery guarantees included) to the application process? [0]: https://github.com/dbos-inc/dbos-transact-py/tree/a3bb7cb6dd53ec58ef4d96a4c9314a16391a0aa5#getting-started https://github.com/dbos-inc/dbos-transact-py/tree/a3bb7cb6dd... [1]: https://docs.dbos.dev/python/reference/configuration#database https://docs.dbos.dev/python/reference/configuration#databas... [2]: https://github.com/dbos-inc/dbos-transact-py/blob/a3bb7cb6dd53ec58ef4d96a4c9314a16391a0aa5/dbos/_core.py#L846 https://github.com/dbos-inc/dbos-transact-py/blob/a3bb7cb6dd... [3]: https://github.com/dbos-inc/dbos-transact-py/blob/a3bb7cb6dd53ec58ef4d96a4c9314a16391a0aa5/dbos/_dbos.py#L784 https://github.com/dbos-inc/dbos-transact-py/blob/a3bb7cb6dd...
- jedberg 2y ago
  The point is that your application already has uptime responsibilities, so why not build the orchestration right into it instead of adding another service that will have its own uptime responsibilities?
  - whilenot-dev 2y ago
    Well, my application servers are usually designed stateless to provide sub-second responses, whereas orchestration workflows can take up to hours. I ususally scale my workers differently than my REST APIs, as their failure scenario looks quite different: an unresponse orchestration engine might just delay its jobs (inconsistent, outdated data), whereas an unavailable API won't provide any responses at all (no data). How'd that work in a microservice architecture anyway? Does each service has some part of the orchestration logic defined? Or will I end up writing a separate orchestration engine as one service anyway? Wouldn't that then contradict the promise of the article?

81 points by qianli_cs 2y ago

Company hawking an orchestrating backend server says you should use an orchestrating backend server? You still have four layers, it's just that one is hidden with annotations.

tomhallett 2y ago
When they get to what their implementation is, I’m not even sure what it is. Like how is the following different from a library, like acts-as_state_machine (https://github.com/aasm/aasm https://github.com/aasm/aasm). Are they auto running the retries - in a background job which they handle (like “serverless” background job)? “Implementing orchestration in a library connected to a database means you can eliminate the orchestration tier, pushing its functionality into the application tier (the library instruments your program) and the database tier (your workflow state is persisted to Postgres).“
- jedberg 2y ago
  > Are they auto running the retries - in a background job which they handle (like “serverless” background job)? Yes. The library makes sure that your app retries failed workflows. Or if you use the commercial products, takes care of that for you.

recursivedoubts 2y ago

Every day we stray further from God’s light.

baq 2y ago
To err is human
- chrisweekly 2y ago
  To forgive, divine
- floathub 2y ago
  To iterate is human.
  - bryanrasmussen 2y ago
    to recurse, devilish.
    - falcor84 2y ago
      What about triple mutual recursion? That must have a spark of the divine?
      - bryanrasmussen 2y ago
        if you follow the doctrine of the trinity - sure.
- mtillman 2y ago
  This takes me back. http://errtheblog.com/ http://errtheblog.com/
- falcor84 2y ago
  My favorite take on this is: To err is human; to bring down the whole cluster with one bad command is DevOps.
falcor84 2y ago
Pray tell, what software architecture does God's light shine brightest on?
- infinitezest 2y ago
  The one you're not currently using, of course.
- pphysch 2y ago
  The GP created HTMX, so possibly he would argue for an even simpler 2-tier or 2.5-tier architecture that addresses frontend complexity
  - dventimi 2y ago
    That's 1 to 1.5 tiers too many: https://docs.postgrest.org/en/v12/how-tos/providing-html-content-using-htmx.html https://docs.postgrest.org/en/v12/how-tos/providing-html-con...

dartos 2y ago

How often do the trades ask “what’s next?”

Trasmatta 2y ago

> At the technical and organizational scale of modern enterprises, the complexity of orchestrating distributed systems is unavoidable. *citation needed We continue to make things much more complex than they need to be. Even better when NON "enterprise" applications also buy into the insane complexity because they feel like they have to (but they have nowhere near the resources to manage that complexity).

nlitened 2y ago
> *citation needed You are already citing the person, do you really need that person to cite another person in order to form an opinion?
- Trasmatta 2y ago
  ? What I'm asking for is evidence that this complexity is "unavoidable".
viraptor 2y ago
That would be difficult (impossible?) to prove. But if the claim is not true, a much easier thing would be to show an example of a large enterprise which did not introduce distributed processing. Is there one? I've not heard of that. Even basics like auth eventually require SSO if you want to preserve your sanity, and that's a distributed system.

politelemon 2y ago

I haven't noticed the same trend or evolution of application tiers, perhaps we live in different echo chambers. Teams using microsevices need to evaluate whether it's still a good fit considering the inherent overhead it brings. Applying a bandaid solution on top of it, if it isn't a good fit, only makes the problem worse.

nkmnz 2y ago
I think the term "microservice" is useless here. It doesn't matter if you run your backend logic in a monolith or in some complex microservice architecture, because both will depend on external runtime dependencies. Especially the smallest of startups will rely heavily on external APIs to connect their monolith (which is, in enterprise terms, probably a single microservice) to external services such as stripe, to some product analytics tool, to a CRM, to openAI etc. etc. pp. I don't know anything about this post's solution, but if it delivers on the idea to not having to worry that much about failed calls to 3rd parties (or even my own database!), I'd like it a lot. Why would you call that a bandaid solution?

turnsout 2y ago

So basically front end, back end and data.

itsthecourier 2y ago

bought a ten years old company, a division of a public company, some million dollars. got an overly complex, over 30 micro services architecture, over usd20k in monthly cloud fees. rewrote the thing into a monolith in 6 months. reduced development team in half, costs of servers by 80-90%, latency by over 60% newer is not better. each micro service must be born from a real necessity out of usage stats, server stats, cost analisis. not by default following tutorials.

brookst 2y ago
It’s telling that you revised both application architecture and org structure to be simpler and more efficient. Microservices are sometimes a reflection of the org; the separation of concerns is about ensuring everyone knows who’s working on what, and enforcing that in the tech. (Not defending that, it’s often inefficient and can be a straight jacket that constrains the product and org)
- smallnix 2y ago
  +1 matches my experience, https://en.m.wikipedia.org/wiki/Conway%27s_law https://en.m.wikipedia.org/wiki/Conway%27s_law
  - sunrunner 2y ago
    Also known as the only unbreakable law (https://www.youtube.com/watch?v=5IUj1EZwpJY https://www.youtube.com/watch?v=5IUj1EZwpJY), a fact which feels deeply depressing when you dig into it.
wavemode 2y ago
and then you renamed the company to X?
KronisLV 2y ago
I've seen the opposite: single monolithic codebase, where the different bits of functionality eventually end up tightly coupled, so it's actually pretty difficult to extract bits into a separate service later, so a different type of architecture isn't possible even if you wanted to split it up. Why do that? Well, when a big Excel file is uploaded to import a bunch of data, or when some reports are generated, or when a crapton of emails is being sent, or when batch processes are sending data over to another system, both the API and the UI become slow for everyone. Scale it vertically, would be the first thought - for a plethora of reasons, that doesn't work. There are bottlenecks in the DB thread pool solution, there are bottlenecks in the HTTP request processing, there are bottlenecks all over the place that can be resolved (for example, replacing HikariCP with DBCP2, oddly enough) but each fix takes a bunch of time and it's anyone's guess whether something will break. Suddenly updating the dependencies of the monolith is also a mess, something like bumping the runtime version also leads to all sorts of things breaking, sometimes at compile time, other times at runtime (which leads to out of date packages needing to be used). Definitionally, a big ball of mud. Can you just "build better software" or "write code without bugs"? Well, yes, but no. I've seen plenty of cases of microservices also becoming a chatty mess, but what strikes me as odd is that people don't attempt to go for something like the following: * keep the business functionality, whatever that may be, in one central service as much as possible * instead of chopping up the domain model, extract functionality that pertains to specific types of mechanisms or workloads (e.g. batch processing, file uploads or processing, data import etc.) into separate services, even if some of it might still use the main DB Yet, usually it's either a mess due to shoving too many things into one codebase, or a mess due to creating too much complexity by attempting to have a service for every small set of entities in your project ("user service", "order service", ...).

recroad 2y ago

I like building one-tier applications in Elixir.

deleted 2y ago

[deleted]

dventimi 2y ago

"In the beginning (that is, the 90’s), developers created the three-tier application. Per Martin Fowler, these tiers were the data source tier, managing persistent data, the domain tier, implementing the application’s primary business logic, and the presentation tier, handling the interaction between the user and the software. The motivation for this separation is as relevant today as it was then: to improve modularity and allow different components of the system to be developed relatively independently." Immediately, I see problems. Martin Fowler's "Patterns of Enterprise Application Architecture" was first published in 2002, a year that I think most people will agree was not in "the 90's." Also, was that the motivation? Are we sure? Who had that motivation? Were there any other motivations at play?

edoceo 2y ago
Well, Martin's book came out after we were doing these patterns in the 90s. My teams had that motivation - data worked with logic; logic worked with UI teams. Separation of concerns and division of labour are, generally, good ideas. ETA: one of the groups that was motivated was MS: use SQL Server + SP ; then COM in the Logic layer and then ASP in the UI.
- jmull 2y ago
  Yes. I was happy when Fowler came out because we could all start using the same terminologies for the same things, and work from common concepts when solving the same problem. (It didn't work out that way, though. It seemed like most people used Fowler as some kind of bible or ending point, when it should have been a starting point/ source of inspiration. Somehow it seemed to turn people's brains off, making them dumber and less insightful about the systems they were building.)
- tomnipotent 2y ago
  > one of the groups that was motivated was MS I remember Microsoft being a huge marketing proponent of the 3-tier architecture in the late 90's, particularly after the release of ASP. The model was promoted everywhere - MSDN, books, blogs, conferences. At this point COM was out of the picture and ASP served as both front-end (serving HTML) and back-end (handling server responses).

ape4 2y ago

Just mentioning MVC (Model-View-Controller) https://developer.mozilla.org/en-US/docs/Glossary/MVC https://developer.mozilla.org/en-US/docs/Glossary/MVC

xyst 2y ago

_Maybe_ something like this is needed at GOOG or NFLX. But for most companies, this added complexity of an “orchestration tier” is unnecessary.

davedx 2y ago

I think people in this industry make using complicated, powerful paradigms part of their identity. They don’t feel like they’re important unless they’re reaching for N-tier architecture or exotic databases or lambdas or whatever else it is. Most apps I’ve worked on could have been a monolith on postgres but they never ever are as soon as I’m not the sole engineer.

ebiester 2y ago
The architecture is the function of number of people in the system. How do you manage 100 people in a monolith? 250? What if one group gets to a broken state but another group needs to release to escape a broken state? Architecture is often solving a human problem. That said, too many teams break out way too early.
- lucianbr 2y ago
  Often the way they attepmt to manage 100 people is to split the monolith into a distributed monolith. Now you have all the same problems plus some new ones, but hey, we're "managing" the human problem. And considering they somehow muddle along, with one person sometimes breaking everything for the other 99, and all the other problems, I think they could very well muddle along with a monolith. With 100 or however many programmers. Yes, the distributed system with well thought out splits into services would be an improvement. But it's clearly not a necessity. So it remains that some places at least, use it out of some other reason - fad, cargo culting, whatever. Architecture should be solving the human or other problems, definitely. But how often it does... I guess each with their own experience.
- davedx 2y ago
  I’ve heard this and at some large engineering orgs, I’ve even occasionally seen it applied sensibly, but IME irrational hype driven architecture dominates: 15 microservices at companies with 5 engineers. Event sourcing where it made no sense. Apache Spark used on a project where all the data and processing fit on a laptop. Orchestration layers; frontends for backends; queues everywhere; the list is endless.

mrkeen 2y ago

> By persisting execution state to a database, a lightweight library can fulfill the primary goal of an orchestration system: guaranteeing code executes correctly despite failures. If a program fails, the library can look up its state in Postgres to figure out what step to take next, retrying transient issues and recovering interrupted executions from their last completed step. program = email all customers failure = throttled by mailchimp

jedberg 2y ago
Your program would gracefully handle this though, because the workflow would fail and it would retry. This requires you to write your program in a way that does not trigger mailchimp's throttling -- a problem that would happen no matter how you write your app and a problem you have to deal with no matter how run your app.

geophile 2y ago

Orchestration tier. Oy. So something goes wrong, and you need to back out an update to one of your microservices. But that back-out attempt goes wrong. Or happens after real-world actions have been based on that update you need to back out. Or the problem that caused a backout was transient, everything turns out to be fine, but now your backout is making its way across the microservices. Backout the backout? What if that goes wrong? The "or"s never end. Just use a centralized relational database, use transactions, and be done with it. People not understanding what can go wrong, and how RDB transactions can deal with a vast subset of those problems -- that's like the 21st century version of how to safely use memory in C. Yes, of course, centralized RDBs with transactions are sometimes the wrong answer, due to scale, or genuinely non-atomic update requirements, or transactions spanning multiple existing systems. But I have the sense that they are often rejected for nonsensical reasons, or not even considered at all.

shermantanktop 2y ago
I mostly agree. But I work at a place where scale precludes that, and as a result relational concepts are sneered at. It turns out that pulling atomicity concerns into a pile of Java code leads to consistency problems…
- geophile 2y ago
  > relational concepts are sneered at That is remarkably dumb. Don't use an RDB if it doesn't fit your requirements. But "relational concepts" covers a lot of valuable ground, and rejecting them out of hand is a combination of ignorant and dumb. Listen kids: Codd's 1970 paper on the relational model of data was absolutely revolutionary. Data processing involved low-level navigation of records. Separation of logical concerns from physical concerns (like 80-column cards) was regarded as crazy. Codd's message was this: a set-oriented data model, manipulated by a high-level, non-procedural language was the right way to process data. And yes, a lot of practical details need to be worked out, but it's still right. And then followed decades of research into the implementation of relational database systems, which gave us working systems that fully realized that set of ideas. It was an incredibly bold research program, which delivered, and gave us incredibly valuable technology such as: - SQL (love it or hate it, it's a mostly non-procedural high-level language that does a great job of data processing). - Query optimization. - Transactions, with levels of isolation and many different implementation approaches, most of which have proven practical at one time or another. Also, early work on distributed algorithms, e.g. two-phase commit. Ignore these "relational concepts", and you will be reinventing it, badly, and at great expense.

ptx 2y ago

> In the beginning (that is, the 90’s), developers created the three-tier application. [...] Of course, application architecture has evolved greatly since the 90's. [...] This complexity has created a new problem for application developers: how to coordinate operations in a distributed backend? For example: How to atomically perform a set of operations in multiple services, so that all happen or none do? This doesn't seem like a correct description of events. Distributed systems existed in the 90s and there was e.g. Microsoft Transaction Server [0] which was intended to do exactly this. It's not a new problem. And the article concludes: > This manages the complexity of a distributed world, bringing the complexity of a microservice RPC call or third-party API call closer to that of a regular function call. Ah, just like DCOM [1] then, just like in the 90s. [0] https://en.wikipedia.org/wiki/Microsoft_Transaction_Server https://en.wikipedia.org/wiki/Microsoft_Transaction_Server [1] https://en.wikipedia.org/wiki/Distributed_Component_Object_Model https://en.wikipedia.org/wiki/Distributed_Component_Object_M...

smithkl42 2y ago
I only ever played with DCOM and Transaction Server, and never in production, but I do wonder what about that tech stack made it so absolutely unworkable, and such a technological dead-end? Did anyone ever manage to make it work?
- dventimi 2y ago
  What I remember is that there were social reasons, market reasons, and technical reasons that MTS didn't pan out. First, Microsoft was out-of-fashion in startup culture. Second, the exploding internet boom had little demand for distributed transactions. Third, COM was a proprietary technology that relied on C++ at a time when developers were flocking to easier memory-managed languages like Java, which was or at least was perceived to be more "open." I'm sure there were other reasons, but that's what looms in my mind.

deleted 2y ago

[deleted]

tnvmadhav 2y ago

i have something positive to say. I really liked the website UX on mobile

localghost3000 2y ago

It took me a bit to realize the author is selling me something. I guess good job there sir. I’ve built a bunch of distributed architectures. In every case I did, we would have been better served with a monolith architecture and a single relational DB like Postgres. In fact I’ve only worked on one system that had the kind of scale that would justify the additional complexity of a distributed architecture. Ironically that system was a monolith with Postgres.

ambicapter 2y ago
> In fact I’ve only worked on one system that had the kind of scale that would justify the additional complexity of a distributed architecture. Ironically that system was a monolith with Postgres. This...doesn't seem to support your case at all? Maybe if you'd turned all those distributed architectures into monoliths you would've then thought the distributed architecture was justified (since you have a 1 of 1 case where that was the case). I'm guessing the truth is somewhere in the middle, but unfortunately it's not very useful to the reader to say "well, some systems are better distributed, some systems are better as monolith". The interesting question is which is which.
- localghost3000 2y ago
  > This...doesn't seem to support your case at all? Hm ok well I am not sure what you mean but its the internet so... <shrug> What I am saying here is that you would be shocked at how far you can get with a simpler architecture. Distributed systems have massive trade offs and are the kind of thing you shouldn't do unless you are FORCED to.
szundi 2y ago
Your experience suggest that monoliths on Postgres might be key to win on the market.
anonzzzies 2y ago
They are basically advocating that (postgres replaces everything else normally used) however they need to add in enterprisey stuff.

bazizbaziz 2y ago

Workflows/orchestration/reconciliation-loops are basically table stakes for any service that is solving significant problems for customers. You might think you don't need this, but when you start needing to run async jobs in response to customer requests, you will always eventually implement one of the above solutions. IMO the next big improvement in this space is improving the authoring experience. In short, when it comes to workflows, we are basically still writing assembly code. Writing workflows today is done in either a totally separate language (StepFunctions), function-level annotations (Temporal, DBOS, etc), or event/reconciliation loops that read state from the DB/queue. In all cases, devs must manually determine when state should be written back to the persistence layer. This adds a level of complexity most devs aren't used to and shouldn't have to reason about. Personally, I think the ideal here is writing code in any structure the language supports, and having the language runtime automatically persist program state at appropriate times. The runtime should understand when persistence is needed (i.e. which API calls are idempotent and for how long) and commit the intermediate state accordingly.

junto 2y ago
There seems to be a lot of negativity about this opinion, but I heartily agree with you. Anytime you’re dealing with large systems that have a multitude of external integrations you’re going to need some kind of orchestration. Anytime you perform a write operation, you cannot safely and idempotently do another IO operation in the same process, without risking a non-retryable exception event of the entire process. Most people when faced with problem will look at some kind of queuing abstraction. The message fails, and you try it automatically later. If you’re a masochist you’ll let it go in a dead letter queue and deal with it manually later. Sagas is one way to orchestrate this kind of system design. Routing slips is another that has the benefit of no central orchestrator and state is just carried in the routing slip. Both are adequate but in the end you’ll end up with a lot of infrastructure and architecture to make it work. Systems like Temporal take a lot of that pain away, allowing developers to focus on writing business code and not infrastructural and architectural code. So I am fully in on this new pattern for the horrible integrations I’m forced do deal with. Web services that are badly written RPC claiming to be REST, or poorly designed SOAP services. REST services that choose to make me to a GET request for the object I just created, because REST purists don’t return objects on creation, only location headers. Flaky web services that are routed over 2 VPN’s because that’s the way the infrastructure team decided to manage it. The worst cast I ever had to deal with was having to process XML instructions over email. And not as an attachment, I mean XML as text in the body of the email. Some people are just cruel. Give someone a greenfield and I’d agree, simplicity rules. But when you’re playing in someone else’s dirty sandpit, you’re always designing for the worst case failure. And for the readers that are still wondering why this matters, I recommend this video from 7 years ago called “six little lines of fail”. https://youtu.be/LGG3IIHUG_w https://youtu.be/LGG3IIHUG_w

Toine 2y ago

Was this written by an infamous 26 yo "senior" developer with 4 years of experience ?

anonzzzies 2y ago
Sure mate. https://en.wikipedia.org/wiki/Michael_Stonebraker https://en.wikipedia.org/wiki/Michael_Stonebraker

gizzlon 2y ago

This is pretty weak, makes very bold statements ("this is the way it has to be now") with no evidence. Reads like the set up for a sales pitch, which came at the end

whilenot-dev 2y ago

Following the Getting Started[0] section it seems like DBOS requires the configuration of a Postgres-compatible database[1] (NOTE: DBOS currently only supports Postgres-compatible databases.). Then, after decorating your application functions as workflow steps[2], you'll basically run those workflows by spawning a bunch of worker threads[3] next to your application process. Isn't that a bit... unoptimized? The orchestrator domain doesn't seem to be demanding on compute, so why aren't they making proper use of asyncio here in the first place? And why aren't they outsourcing their runtime to an independent process? EDIT: So "To manage this complexity, we believe that any good solution to the orchestration problem should combine the orchestration and application tiers." (from the article) means that your application runtime will also become the orchestrator for its own workflow steps. Is that a good solution? EDIT2: Are they effectively just shifting any uptime responsibility (delivery guarantees included) to the application process? [0]: https://github.com/dbos-inc/dbos-transact-py/tree/a3bb7cb6dd53ec58ef4d96a4c9314a16391a0aa5#getting-started https://github.com/dbos-inc/dbos-transact-py/tree/a3bb7cb6dd... [1]: https://docs.dbos.dev/python/reference/configuration#database https://docs.dbos.dev/python/reference/configuration#databas... [2]: https://github.com/dbos-inc/dbos-transact-py/blob/a3bb7cb6dd53ec58ef4d96a4c9314a16391a0aa5/dbos/_core.py#L846 https://github.com/dbos-inc/dbos-transact-py/blob/a3bb7cb6dd... [3]: https://github.com/dbos-inc/dbos-transact-py/blob/a3bb7cb6dd53ec58ef4d96a4c9314a16391a0aa5/dbos/_dbos.py#L784 https://github.com/dbos-inc/dbos-transact-py/blob/a3bb7cb6dd...

jedberg 2y ago
The point is that your application already has uptime responsibilities, so why not build the orchestration right into it instead of adding another service that will have its own uptime responsibilities?
- whilenot-dev 2y ago
  Well, my application servers are usually designed stateless to provide sub-second responses, whereas orchestration workflows can take up to hours. I ususally scale my workers differently than my REST APIs, as their failure scenario looks quite different: an unresponse orchestration engine might just delay its jobs (inconsistent, outdated data), whereas an unavailable API won't provide any responses at all (no data). How'd that work in a microservice architecture anyway? Does each service has some part of the orchestration logic defined? Or will I end up writing a separate orchestration engine as one service anyway? Wouldn't that then contradict the promise of the article?