6 ms·

> - Becomes hard to make cross-cutting changes This does seem like a negative, doesn't it? But it's not. Making it hard to make cross-cutting changes is exact

> - Becomes hard to make cross-cutting changes This does seem like a negative, doesn't it? But it's not. Making it hard to make cross-cutting changes is exactly the point of splitting up a repo. It forces you to slow down, and—knowing that you can only rarely make cross-cutting changes—you have a strong incentive to move module boundaries to where they should be. It puts pressure on you to really, actually separate concerns. Not just put "concerns" into a source file that can reach into any of a million other source files and twiddle the bits. "Easy to make sweeping changes" really means "easy to limp along with a bad architecture." I think that's one of the reasons why so much code rots: developers thinking it should be easy to make arbitrary changes. No, it should be hard to make arbitrary changes. It should be easy to make changes with very few side effects, and hard to make changes that affect lots of other code. That's how you get modules that get smaller and smaller, and change less and less often, while still executing often. That's the opposite of code rot: code nirvana.

vtbassmatt 9y ago
As the linked article says, modularizing and living in separate repos was the plan of record for a while. But after evaluating the tradeoffs, we decided that Windows needs to optimize for big, rapid refactors at this stage in its development. "Easy to make sweeping changes" also means "easy to clean up architecture and refactor for cleaner boundaries". The Windows build system is where component boundaries get enforced. Having version control introduce additional arbitrary boundaries makes the problem of good modularity harder to solve.
phkahler 9y ago
Remember when they said IE was an integral part of the OS? Yeah...
Goronmon 9y ago
No, it should be hard to make arbitrary changes. If you change the word "arbitrary" to "necessary" (implying a different bias than the one you went with) then all of a sudden this attitude sounds less helpful. Similarly "easy to limp along with a bad architecture" could be re-written as "easy to work with the existing architecture". At the end of the day, it's about getting work done, not making decisions that are the most "pure".
- colechristensen 9y ago
  You have to balance getting work done vs. purity, and Microsoft has spent years trying to fix a bad balance. Windows ME/Vista/8 were terrible and widely hated pieces of software because of "getting things done" instead of making good decisions. They made billions of dollars doing it, don't get me wrong, but they've also lost a lot of market share too and have been piling on bad sentiment for years. They've been pivoting and it has nothing to do with "getting work done" but by going back and making better decisions.
  - ern 9y ago
    I assumed that Windows 8 was hated because it broke the Start Menu and tried to force users onto Metro.
    - sjwright 9y ago
      It also broke a lot of working user interfaces, e.g. wireless connection management.
  - contextfree 9y ago
    Those releases (well, Vista and 8 anyway, I don't know about ME) came out of a long and slow planning process - if they made bad decisions I don't think it was about not taking long enough to make them.
- vec 9y ago
  Arguing about purity is only pointless and sanctimonious if the water isn't contaminated. Being unable to break a several hundred megabyte codebase into modules isn't a "tap water vs bottled" purity argument, it's a "lets not all die of cholera" purity argument.
- erikpukinskis 9y ago
  > At the end of the day, it's about getting work done, not making decisions that are the most "pure". This attitude will lead to a total breakdown of the development process over the long term. You are privileging Work Done At The End Of The Day over everything else. You need to consider work done at every relevant time scale. How much can you get done today? How much can you get done this month? How much can you get done in 5 years? Ignore any of these questions at your peril. I fundamentally agree with you about purity though. I'm not sure what in my piece made you think I think Purity Uber Alles is the right way to go.
likpok 9y ago
From my experience with Java-style hard module dependencies, this makes it extremely difficult to refactor anything touching external interfaces. You say this forces you to think ahead, but predicting the future is quite difficult. The result is that you limp along with known-broken code because it would take so much effort to make the breaking changes to clean it up. For example, lets say you discover that people are frequently misuing a blocking function because they don't realize that it blocks. Let's say that we have a function `bool doThing()`. We discover that the bool return type is underspecified: there's a number of not-exactly-failure not-exactly-success cases. In a monorepo, it's pretty easy to modify this so that `doThing()` can return a `Result` instead. With multiple repos and artifacts, you either bring up the transitive closure of projects, or you leave it for someone to do later. For a widely used function, this can be prohibitive. That makes people frequently choose the "rename and deprecate" model, which means you get an increasing pile of known-bad functions.
Eyas 9y ago
You say that, but it is very telling that every large company out there (Google and Facebook come to mind) go for the single-repository approach. I'm sure that, when dealing with stakeholder structures where different organizations can depend on different bits and pieces, having multiple repositories with difficulty of making breaking and cross-cutting changes, becomes good. From the view of a single organization where the only users of a component are other components in the same organization, it seems like there is consensus around single-repository.
- Jach 9y ago
  It is very telling. Google has a cloud supercomputer doing nothing but building code to support their devs. I don't know about Facebook. (I really don't -- I'm constantly amazed that they have as many engineers that they do, what do they all work on?) Where I work (https://medium.com/salesforce-engineering/monolith-to-microservices-the-first-peel-6ff3484e3fb4 https://medium.com/salesforce-engineering/monolith-to-micros...) there's a big monolith but with more push towards breaking things up, at the architecture level and also on the code organization level. We also use and commit to open source projects (that use git), so to integrate those with the core requires a bit more effort than if they were there already but it's not a big burden and the benefits of having their tendrils being self-contained are big. Which brings me to the point that in the open source world, you can't get away with a single-repository approach for your large system. And that also is telling, along with open source's successes. So which approach is better in the long run? I'd bet on the open source methods.
  - flai 9y ago
    You forget Conway's law: > organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations Open source has a very different communication structure than a company. While the big three (MS, Google, FB) try to work towards good inter-departmemt relations, it is usually either - a single person - a small group that are the gatekeepers for a small amout of code, typically encapsulated in a "project". They do commit a lot to their project, yet rarely touch other projects in comparision. Also, collaboration is infinitely harder, as in the office you can simply walk up to someome, call them, or chat them - in OSS a lot of communication works via Issues and PRs, which are a fundamentally different way to communicate. This all is reflected by the structure of how a functionality is managed: Each set of gatekeepers gets their own repository, for their function. Interestingly this even happens with bigger repositories: DefinitelyTyped is a repository for TypeScript "typings" for different JS libraries, which has hundreds of collaborators. Yet, if you open a pull-request for an existing folder the ones that have previously made big-ish changes can approve / decline the PR, so each folder is its own little repo. https://github.com/DefinitelyTyped/DefinitelyTyped/pull/16727#issuecomment-303844631 https://github.com/DefinitelyTyped/DefinitelyTyped/pull/1672... So: maybe the solution is big repos for closed companies, small repos for open-source?
johnfn 9y ago
Have you actually worked on a repository at Windows scale..? If not, how can you know that your guesses about the workflow are accurate?
emmelaich 9y ago
Making something difficult even more difficult is not helpful to anyone. And what happens if the inherent difficult disappears or reduces? You're still left with the imposed external difficulty.

vtbassmatt 9y ago

As the linked article says, modularizing and living in separate repos was the plan of record for a while. But after evaluating the tradeoffs, we decided that Windows needs to optimize for big, rapid refactors at this stage in its development. "Easy to make sweeping changes" also means "easy to clean up architecture and refactor for cleaner boundaries". The Windows build system is where component boundaries get enforced. Having version control introduce additional arbitrary boundaries makes the problem of good modularity harder to solve.

phkahler 9y ago

Remember when they said IE was an integral part of the OS? Yeah...

Goronmon 9y ago

No, it should be hard to make arbitrary changes. If you change the word "arbitrary" to "necessary" (implying a different bias than the one you went with) then all of a sudden this attitude sounds less helpful. Similarly "easy to limp along with a bad architecture" could be re-written as "easy to work with the existing architecture". At the end of the day, it's about getting work done, not making decisions that are the most "pure".

colechristensen 9y ago
You have to balance getting work done vs. purity, and Microsoft has spent years trying to fix a bad balance. Windows ME/Vista/8 were terrible and widely hated pieces of software because of "getting things done" instead of making good decisions. They made billions of dollars doing it, don't get me wrong, but they've also lost a lot of market share too and have been piling on bad sentiment for years. They've been pivoting and it has nothing to do with "getting work done" but by going back and making better decisions.
- ern 9y ago
  I assumed that Windows 8 was hated because it broke the Start Menu and tried to force users onto Metro.
  - sjwright 9y ago
    It also broke a lot of working user interfaces, e.g. wireless connection management.
- contextfree 9y ago
  Those releases (well, Vista and 8 anyway, I don't know about ME) came out of a long and slow planning process - if they made bad decisions I don't think it was about not taking long enough to make them.
vec 9y ago
Arguing about purity is only pointless and sanctimonious if the water isn't contaminated. Being unable to break a several hundred megabyte codebase into modules isn't a "tap water vs bottled" purity argument, it's a "lets not all die of cholera" purity argument.
erikpukinskis 9y ago
> At the end of the day, it's about getting work done, not making decisions that are the most "pure". This attitude will lead to a total breakdown of the development process over the long term. You are privileging Work Done At The End Of The Day over everything else. You need to consider work done at every relevant time scale. How much can you get done today? How much can you get done this month? How much can you get done in 5 years? Ignore any of these questions at your peril. I fundamentally agree with you about purity though. I'm not sure what in my piece made you think I think Purity Uber Alles is the right way to go.

likpok 9y ago

From my experience with Java-style hard module dependencies, this makes it extremely difficult to refactor anything touching external interfaces. You say this forces you to think ahead, but predicting the future is quite difficult. The result is that you limp along with known-broken code because it would take so much effort to make the breaking changes to clean it up. For example, lets say you discover that people are frequently misuing a blocking function because they don't realize that it blocks. Let's say that we have a function `bool doThing()`. We discover that the bool return type is underspecified: there's a number of not-exactly-failure not-exactly-success cases. In a monorepo, it's pretty easy to modify this so that `doThing()` can return a `Result` instead. With multiple repos and artifacts, you either bring up the transitive closure of projects, or you leave it for someone to do later. For a widely used function, this can be prohibitive. That makes people frequently choose the "rename and deprecate" model, which means you get an increasing pile of known-bad functions.

Eyas 9y ago

You say that, but it is very telling that every large company out there (Google and Facebook come to mind) go for the single-repository approach. I'm sure that, when dealing with stakeholder structures where different organizations can depend on different bits and pieces, having multiple repositories with difficulty of making breaking and cross-cutting changes, becomes good. From the view of a single organization where the only users of a component are other components in the same organization, it seems like there is consensus around single-repository.

Jach 9y ago
It is very telling. Google has a cloud supercomputer doing nothing but building code to support their devs. I don't know about Facebook. (I really don't -- I'm constantly amazed that they have as many engineers that they do, what do they all work on?) Where I work (https://medium.com/salesforce-engineering/monolith-to-microservices-the-first-peel-6ff3484e3fb4 https://medium.com/salesforce-engineering/monolith-to-micros...) there's a big monolith but with more push towards breaking things up, at the architecture level and also on the code organization level. We also use and commit to open source projects (that use git), so to integrate those with the core requires a bit more effort than if they were there already but it's not a big burden and the benefits of having their tendrils being self-contained are big. Which brings me to the point that in the open source world, you can't get away with a single-repository approach for your large system. And that also is telling, along with open source's successes. So which approach is better in the long run? I'd bet on the open source methods.
- flai 9y ago
  You forget Conway's law: > organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations Open source has a very different communication structure than a company. While the big three (MS, Google, FB) try to work towards good inter-departmemt relations, it is usually either - a single person - a small group that are the gatekeepers for a small amout of code, typically encapsulated in a "project". They do commit a lot to their project, yet rarely touch other projects in comparision. Also, collaboration is infinitely harder, as in the office you can simply walk up to someome, call them, or chat them - in OSS a lot of communication works via Issues and PRs, which are a fundamentally different way to communicate. This all is reflected by the structure of how a functionality is managed: Each set of gatekeepers gets their own repository, for their function. Interestingly this even happens with bigger repositories: DefinitelyTyped is a repository for TypeScript "typings" for different JS libraries, which has hundreds of collaborators. Yet, if you open a pull-request for an existing folder the ones that have previously made big-ish changes can approve / decline the PR, so each folder is its own little repo. https://github.com/DefinitelyTyped/DefinitelyTyped/pull/16727#issuecomment-303844631 https://github.com/DefinitelyTyped/DefinitelyTyped/pull/1672... So: maybe the solution is big repos for closed companies, small repos for open-source?

johnfn 9y ago

Have you actually worked on a repository at Windows scale..? If not, how can you know that your guesses about the workflow are accurate?

emmelaich 9y ago

Making something difficult even more difficult is not helpful to anyone. And what happens if the inherent difficult disappears or reduces? You're still left with the imposed external difficulty.