4 ms·

I can't help thinking, there is more to this than is discussed in the article. This tech can also be used to give developers selective access to a git repo. I

I can't help thinking, there is more to this than is discussed in the article. This tech can also be used to give developers selective access to a git repo. It lets Microsoft and other commercial software vendors get on the git bandwagon with out actually sharing all the code with all the developers. Each git repo is a complete copy because git was designed to support Open-Source. In Open-Source there are no hidden parts. But that's a hard pill for corps like Microsoft to swallow. This looks more like a solution to meet developer demands to use git and also satisfy the demands of management to maintain control of their team's code based.

Sevii 9y ago
Got vfs is more motivated by Microsoft wanting to use git as a monorepo. I think they have around 2k windows devs on one git tree with this vfs.
- philipov 9y ago
  Are there advantages to using a monorepo rather than having a proper package manager? Seems like the problems they're trying to solve would be better solved by not insisting on using a monorepo.
  - AaronFriel 9y ago
    While this debate keeps being hashed out, it's important to note that some of the largest and most successful tech companies have decided to use monorepos. Facebook, Google, Microsoft, and others use monorepos. I suspect one factor for Microsoft is that many patches will be cross-cutting, and tooling with multiple repositories or even submodules remains poor.
    - philipov 9y ago
      I think it's dangerous to assume a causal link between success and their use of monorepos. As aaronAgain points out, Microsoft's codebase existed since before monoliths were recognised to be a bad idea. They're already screwed, and need a solution for where they are today. That's a valid and important use-case, but doesn't suggest that someone starting from scratch should follow their lead.
  - aaronAgain 9y ago
    It will take years to break apart some of the code into packages. In the meantime Git has to work on the monolith, so why not contribute back how we did that. Disclosure: Work at MS
    - philipov 9y ago
      If that's the reason, that's a great use-case for it. I am looking at it from a green fields perspective, and it seems more profitable to start with my company's codebase split up into packages from the outset, and focus on building tooling to handle package dependency version management instead of building tools to handle monorepos. The existence of large companies using monorepos creates some cargo cult pressure on those trying to make a decision which path to take, and so I think it's important to discuss what is really the goal to determine the best fit for my own use-case.
      - asmosoinio 9y ago
        When you say "start your company", I assume a small total repo size. I would say definitely go for a single repo, every time, all the time. Internal version dependencies can be surprisingly hard to handle even with a handful of components/apps. It is extremely unlikely that your companys code size would become too much to handle in a single repo. And before that, imho it's just premature optimization to split things up.
        philipov 9y ago
        While it's possible for it to become premature optimization, there is a level of division that hits a sweet spot that should absolutely be done up-front. I think it is not unreasonable to have a repository per project, and have another "Core" repository for code you factor out of individual project repos for reuse. As the core becomes more complex, you are then already set up to handle it. Likewise, if you don't write at least some documentation up-front, you're much more likely to never get it done at all.
        9y ago
  - marcoperaza 9y ago
    If you separate things out, you have to now manage version compatibility among different components. Which is a massive waste of time when they are going to be shipped as a single unit anyway. Things are indeed being broken up into packages, but they still all live in the same repository. Microsoft goes one step further and checks the entire build system into the source tree too, compiler and all. The effect of all this is that you can sync to any known-good version of Windows and just build it without worrying about other dependencies.
  - InclinedPlane 9y ago
    I've worked at both Microsoft and Amazon, and they have opposite strategies for integrating large code bases. Microsoft prefers putting everything into one giant project and having different teams touch different parts of it (there are lots of advantages in this, incidentally, especially in terms of servicing). Amazon goes the opposite way, by essentially using package management to keep track of and manage dependencies between a huge number of smaller projects. Amazon's solution works well for them, partly because it matches their service oriented architecture. But at the same time, it's the only thing of its sort I've ever seen that was actually built to a sufficient level of quality. And it represents a tremendous investment of dev work over a more than a decade. Microsoft is capable of doing that kind of work (and they sort of are with some of the azure stuff they're working on) but it's a non-trivial project that requires a tremendous amount of investment of time and resources. And that's just to get to square one, let alone something that is competitive with their other infrastructure (keep in mind that microsoft also has tremendous investments in build and CI infrastructure). That's a very hard sell, especially from a risk management perspective.
    - int_19h 9y ago
      In Microsoft, it really depends on which org you're in, and which product you're working on. Older codebases (Windows, Office) tend to be monorepos. Newer ones are generally less so. And some of the older ones have been migrating slowly via componentization (VS).
binthere 9y ago
As stated in the article, git is also challenging for game development because of the large asset files. Git has limitations and they are part of the reasons a lot of companies and even indie game devs didn't jump into it (Jonathan Blow, John Carmack, being some of the critics). It's more about speed, simplicity, and efficiency rather than "Corp" vs "Open Source" conspiracy.
- KallDrexx 9y ago
  Git's issues with game dev isn't just because of large file size though. Binary files are hard to control in a DVCS because if two people spend 24 hours to modify a binary asset at the same time you can't just merge the changes together like with source code. Someone has to manually go in and replicate both changes together.
muricula 9y ago
It's not about maintaining control -- downloading any one of those files is as easy as running `touch` on them. It's about only checking out which pieces you actually need to build the component you care about. That way I don't need to download the whole IE source to build the NT kernel. (I just checked code into windows yesterday)

Sevii 9y ago

Got vfs is more motivated by Microsoft wanting to use git as a monorepo. I think they have around 2k windows devs on one git tree with this vfs.

philipov 9y ago
Are there advantages to using a monorepo rather than having a proper package manager? Seems like the problems they're trying to solve would be better solved by not insisting on using a monorepo.
- AaronFriel 9y ago
  While this debate keeps being hashed out, it's important to note that some of the largest and most successful tech companies have decided to use monorepos. Facebook, Google, Microsoft, and others use monorepos. I suspect one factor for Microsoft is that many patches will be cross-cutting, and tooling with multiple repositories or even submodules remains poor.
  - philipov 9y ago
    I think it's dangerous to assume a causal link between success and their use of monorepos. As aaronAgain points out, Microsoft's codebase existed since before monoliths were recognised to be a bad idea. They're already screwed, and need a solution for where they are today. That's a valid and important use-case, but doesn't suggest that someone starting from scratch should follow their lead.
- aaronAgain 9y ago
  It will take years to break apart some of the code into packages. In the meantime Git has to work on the monolith, so why not contribute back how we did that. Disclosure: Work at MS
  - philipov 9y ago
    If that's the reason, that's a great use-case for it. I am looking at it from a green fields perspective, and it seems more profitable to start with my company's codebase split up into packages from the outset, and focus on building tooling to handle package dependency version management instead of building tools to handle monorepos. The existence of large companies using monorepos creates some cargo cult pressure on those trying to make a decision which path to take, and so I think it's important to discuss what is really the goal to determine the best fit for my own use-case.
    - asmosoinio 9y ago
      When you say "start your company", I assume a small total repo size. I would say definitely go for a single repo, every time, all the time. Internal version dependencies can be surprisingly hard to handle even with a handful of components/apps. It is extremely unlikely that your companys code size would become too much to handle in a single repo. And before that, imho it's just premature optimization to split things up.
      - philipov 9y ago
        While it's possible for it to become premature optimization, there is a level of division that hits a sweet spot that should absolutely be done up-front. I think it is not unreasonable to have a repository per project, and have another "Core" repository for code you factor out of individual project repos for reuse. As the core becomes more complex, you are then already set up to handle it. Likewise, if you don't write at least some documentation up-front, you're much more likely to never get it done at all.
        9y ago
- marcoperaza 9y ago
  If you separate things out, you have to now manage version compatibility among different components. Which is a massive waste of time when they are going to be shipped as a single unit anyway. Things are indeed being broken up into packages, but they still all live in the same repository. Microsoft goes one step further and checks the entire build system into the source tree too, compiler and all. The effect of all this is that you can sync to any known-good version of Windows and just build it without worrying about other dependencies.
- InclinedPlane 9y ago
  I've worked at both Microsoft and Amazon, and they have opposite strategies for integrating large code bases. Microsoft prefers putting everything into one giant project and having different teams touch different parts of it (there are lots of advantages in this, incidentally, especially in terms of servicing). Amazon goes the opposite way, by essentially using package management to keep track of and manage dependencies between a huge number of smaller projects. Amazon's solution works well for them, partly because it matches their service oriented architecture. But at the same time, it's the only thing of its sort I've ever seen that was actually built to a sufficient level of quality. And it represents a tremendous investment of dev work over a more than a decade. Microsoft is capable of doing that kind of work (and they sort of are with some of the azure stuff they're working on) but it's a non-trivial project that requires a tremendous amount of investment of time and resources. And that's just to get to square one, let alone something that is competitive with their other infrastructure (keep in mind that microsoft also has tremendous investments in build and CI infrastructure). That's a very hard sell, especially from a risk management perspective.
  - int_19h 9y ago
    In Microsoft, it really depends on which org you're in, and which product you're working on. Older codebases (Windows, Office) tend to be monorepos. Newer ones are generally less so. And some of the older ones have been migrating slowly via componentization (VS).

binthere 9y ago

As stated in the article, git is also challenging for game development because of the large asset files. Git has limitations and they are part of the reasons a lot of companies and even indie game devs didn't jump into it (Jonathan Blow, John Carmack, being some of the critics). It's more about speed, simplicity, and efficiency rather than "Corp" vs "Open Source" conspiracy.

KallDrexx 9y ago
Git's issues with game dev isn't just because of large file size though. Binary files are hard to control in a DVCS because if two people spend 24 hours to modify a binary asset at the same time you can't just merge the changes together like with source code. Someone has to manually go in and replicate both changes together.

muricula 9y ago

It's not about maintaining control -- downloading any one of those files is as easy as running `touch` on them. It's about only checking out which pieces you actually need to build the component you care about. That way I don't need to download the whole IE source to build the NT kernel. (I just checked code into windows yesterday)