3 ms·

We solve the problem of 50 devs working in a single monolith with folder and file structure, separation of concerns, basic stuff like this

mjr00 1y ago
Folder and file structure and separation of concerns doesn't change the fact that if you have one deployable artifact, it's all sharing the same runtime when deployed. Which means the underlying versions of Java/Go/Python/etc, or core shared libraries, all need to be updated at the same time. All the code is far more coupled than it first seems.
- monero-xmr 1y ago
  All of that is so much easier with a single monorepo
  - mjr00 1y ago
    Monorepo is orthogonal to services though. You can have a monorepo with multiple services in it. Even with a monorepo, you will hit a point where you have 1, 10, 100 million lines of e.g. Python, realize you should upgrade from 3.8 to 3.14 because it's EOL, and feel a lot of pain as you have to do a big-bang, all-at-once change, fixing every single breaking change, including from libraries which you also have to update. There's no way around this in current mainstream languages.
- roguecoder 1y ago
  That is not really an issue I've had with Java, but I would absolutely agree that Python is wildly unsuited as a production backend language. I don't think it's much better if you have to spend a year and a half updating 400+ different repos, though. It's much easier to use an operationalized language that knows backwards compatibility matters.
  - mjr00 1y ago
    I was at AWS RDS when they upgraded the shared control plane code from Java 7 to 8. IIRC it was about 6 months for 5-10 developers more or less full-time. Absolutely massive timesink. The move to separate services happened shortly after that. > I don't think it's much better if you have to spend a year and a half updating 400+ different repos, though. There's two things going for separate services (which may or may not be separate repos; remember a single repo can have multiple services): 1. You can do it piecemeal. 90% of your services will be 15-minute changes: update versions in a few files, let automated tests run, it's good to go. The 10% that have deeper compatibility issues can be addressed separately without holding back the rest. You can't separate this if you have a single deployable artifact. 2. Complexity is superlinear with respect to lines of code. Upgrading a single 1mLOC service isn't 10x harder than updating ten 100kLOC services, it's more like 20, 30x harder. Obviously this is hard to measure, but there's a reason these massive legacy codebases get stuck on ancient versions of dependencies. (And a reason companies pay out the ass for Oracle's extended Java 8 support, which they still offer.)
roguecoder 1y ago
Part of the problem is that many current programmers came up through functional programming or framework-based development. Microservices are often the first time they encountered modular programming or encapsulation, and so they equate "literally any architecture" with "microservices". I've worked on monoliths with 400+ developers that were great, but it takes skills that people who have only ever worked in orgs that mandate microservice just don't have.
- djtango 1y ago
  Could you elaborate on how functional programming relates to people's relationship with Microservices?
  - roguecoder 1y ago
    Sure! Functional programming precludes encapsulation, so it doesn't scale indefinitely the way fractal paradigms can. Eventually, the complexity becomes overwhelming. One effective solution to that is introducing microservices: programmers can still write entirely functional code, but have encapsulation in the form of services. They have to be micro, though, because conventionally-sized services are still big enough to strain the paradigm. But I see junior engineers who aren't expected to think about the "architecture", by which they mean the modular design. They are handed a spec and they implement it, Mythical Man Month style. That treats organizing lines of code and organizing services as two completely-distinct activities, and depending on the company junior engineers are often not exposed to modular design until five or ten years into their careers.
    - wild_egg 1y ago
      > Functional programming precludes encapsulation Since when? Maybe we have different definitions of "encapsulation" but this clause seems nonsensical to me. FP is huge on encapsulation
    - antonvs 1y ago
      You’re suffering from a misunderstanding there. Functional programming is all about encapsulation, starting at the individual function boundary (closures can encapsulate state) and then at every layer above that. Functional languages have some of the most rigorous module systems available. In fact Java adopted such a system recently, showing the weaknesses in its previous support for encapsulation via classes and packages.
bunderbunder 1y ago
This is really what it comes down to right here. The real challenge is Conway's Law. Both the software architecture and the org chart need to be designed with Conway's Law in mind. If that hasn't happened then deciding between microservices and monolith is ultimately just deciding how you will be punished for your mistake.
- roguecoder 1y ago
  People misunderstanding Conway's Law is a big part of the problem for sure. The law says nothing about team boundaries: it talks about communication pathways. The paranoid socialist in me thinks big companies like team-sized microservices because it lets them prevent workers from talking to each other without completely ruling out producing running software. When companies instead encourage forums for communication across team boundaries, it unlocks completely different architectural patterns.
  - bunderbunder 1y ago
    If team boundaries aren't a major influence on your communication pathways, what on earth do you even mean by "team"?
    - roguecoder 1y ago
      The most common alternative to organizing teams by service boundaries is to organize teams around the business problems to be solved. That is a lot easier to budget for than trying to staff by microservice boundary, doesn't have the coordination and planning overhead, and it means you aren't reliant on up-front planning to get to a functional solution or design. In high-uncertainty greenfield development, Explore projects or Lean Startup-style experimentation, having developer be close to the users they are serving is very efficient. It also lets those companies reteam frequently, without needing to change the software to match the new team boundaries, which is very helpful when growing the team.
dec0dedab0de 1y ago
Don't know why you're downvoted, but this is the way. Even if you're using micro services, it's usually best to have them in the same repo organized into different directories. No matter how many people you have, you really should minimize working on the same files concurrently. This is trivial with most languages

mjr00 1y ago

Folder and file structure and separation of concerns doesn't change the fact that if you have one deployable artifact, it's all sharing the same runtime when deployed. Which means the underlying versions of Java/Go/Python/etc, or core shared libraries, all need to be updated at the same time. All the code is far more coupled than it first seems.

monero-xmr 1y ago
All of that is so much easier with a single monorepo
- mjr00 1y ago
  Monorepo is orthogonal to services though. You can have a monorepo with multiple services in it. Even with a monorepo, you will hit a point where you have 1, 10, 100 million lines of e.g. Python, realize you should upgrade from 3.8 to 3.14 because it's EOL, and feel a lot of pain as you have to do a big-bang, all-at-once change, fixing every single breaking change, including from libraries which you also have to update. There's no way around this in current mainstream languages.
roguecoder 1y ago
That is not really an issue I've had with Java, but I would absolutely agree that Python is wildly unsuited as a production backend language. I don't think it's much better if you have to spend a year and a half updating 400+ different repos, though. It's much easier to use an operationalized language that knows backwards compatibility matters.
- mjr00 1y ago
  I was at AWS RDS when they upgraded the shared control plane code from Java 7 to 8. IIRC it was about 6 months for 5-10 developers more or less full-time. Absolutely massive timesink. The move to separate services happened shortly after that. > I don't think it's much better if you have to spend a year and a half updating 400+ different repos, though. There's two things going for separate services (which may or may not be separate repos; remember a single repo can have multiple services): 1. You can do it piecemeal. 90% of your services will be 15-minute changes: update versions in a few files, let automated tests run, it's good to go. The 10% that have deeper compatibility issues can be addressed separately without holding back the rest. You can't separate this if you have a single deployable artifact. 2. Complexity is superlinear with respect to lines of code. Upgrading a single 1mLOC service isn't 10x harder than updating ten 100kLOC services, it's more like 20, 30x harder. Obviously this is hard to measure, but there's a reason these massive legacy codebases get stuck on ancient versions of dependencies. (And a reason companies pay out the ass for Oracle's extended Java 8 support, which they still offer.)

roguecoder 1y ago

Part of the problem is that many current programmers came up through functional programming or framework-based development. Microservices are often the first time they encountered modular programming or encapsulation, and so they equate "literally any architecture" with "microservices". I've worked on monoliths with 400+ developers that were great, but it takes skills that people who have only ever worked in orgs that mandate microservice just don't have.

djtango 1y ago
Could you elaborate on how functional programming relates to people's relationship with Microservices?
- roguecoder 1y ago
  Sure! Functional programming precludes encapsulation, so it doesn't scale indefinitely the way fractal paradigms can. Eventually, the complexity becomes overwhelming. One effective solution to that is introducing microservices: programmers can still write entirely functional code, but have encapsulation in the form of services. They have to be micro, though, because conventionally-sized services are still big enough to strain the paradigm. But I see junior engineers who aren't expected to think about the "architecture", by which they mean the modular design. They are handed a spec and they implement it, Mythical Man Month style. That treats organizing lines of code and organizing services as two completely-distinct activities, and depending on the company junior engineers are often not exposed to modular design until five or ten years into their careers.
  - wild_egg 1y ago
    > Functional programming precludes encapsulation Since when? Maybe we have different definitions of "encapsulation" but this clause seems nonsensical to me. FP is huge on encapsulation
  - antonvs 1y ago
    You’re suffering from a misunderstanding there. Functional programming is all about encapsulation, starting at the individual function boundary (closures can encapsulate state) and then at every layer above that. Functional languages have some of the most rigorous module systems available. In fact Java adopted such a system recently, showing the weaknesses in its previous support for encapsulation via classes and packages.

bunderbunder 1y ago

This is really what it comes down to right here. The real challenge is Conway's Law. Both the software architecture and the org chart need to be designed with Conway's Law in mind. If that hasn't happened then deciding between microservices and monolith is ultimately just deciding how you will be punished for your mistake.

roguecoder 1y ago
People misunderstanding Conway's Law is a big part of the problem for sure. The law says nothing about team boundaries: it talks about communication pathways. The paranoid socialist in me thinks big companies like team-sized microservices because it lets them prevent workers from talking to each other without completely ruling out producing running software. When companies instead encourage forums for communication across team boundaries, it unlocks completely different architectural patterns.
- bunderbunder 1y ago
  If team boundaries aren't a major influence on your communication pathways, what on earth do you even mean by "team"?
  - roguecoder 1y ago
    The most common alternative to organizing teams by service boundaries is to organize teams around the business problems to be solved. That is a lot easier to budget for than trying to staff by microservice boundary, doesn't have the coordination and planning overhead, and it means you aren't reliant on up-front planning to get to a functional solution or design. In high-uncertainty greenfield development, Explore projects or Lean Startup-style experimentation, having developer be close to the users they are serving is very efficient. It also lets those companies reteam frequently, without needing to change the software to match the new team boundaries, which is very helpful when growing the team.

dec0dedab0de 1y ago

Don't know why you're downvoted, but this is the way. Even if you're using micro services, it's usually best to have them in the same repo organized into different directories. No matter how many people you have, you really should minimize working on the same files concurrently. This is trivial with most languages