8 ms·

How Kubernetes Broke Git

76 points by kiyanwang 4y ago

jwildeboer 4y ago
I’ve read the article 3 times now. I still fail to find the part where git broke. I also fail to find a concise explanation on what the solution is. I may need a beer to understand this.
- mdaniel 4y ago
  As best I can tell, the heart of the alleged issue is this from the middle: > A system that could record atomic commits across projects or a better submodule experience would have allowed for more flexible developer organization, especially as the project grew to a new scale. but otherwise I'm with you that this could have used a better title or something
  - thrashh 4y ago
    I can’t tell if it’s because they had a good process and the tools didn’t fit or because they have a terrible process and nothing was going to work for them anyway I see the latter much more often when I jump companies and Google’s projects have had terrible API stability so I’m not really sure Git is to blame here
- ec109685 4y ago
  GitHub is the system that broke. All the communication systems within it needed to be disabled for custom notifications since it would otherwise be too noisy.
- mikl 4y ago
  It’s just clickbait, the author posts clickbait on here regularly. It works, unfortunately.
politelemon 4y ago
That was an interesting read thanks for sharing it! My takeaway from this is that the k8s codebase is complicated with complicated workflows. Simple tools won't suffice for this development system. They had to create some of their own tooling; in a way that's quite in line with the usage of k8s, a lot of existing OS concepts had to be recreated just for k8s. > authorization > package management > So why shouldn't a VCS embrace its role as a collaboration tool and explore more generic merge-based optimizations like a queue? In the bottom section there are a few 'wishlist' items, can I call it that. But those aren't good VCS features, they're a reflection of the k8s development world which is not how most of us do development. It's also assuming that because k8s is an all-in-one-doing-many-things, that the VCS it uses should also be a huge all-in-one. I don't think it should; all that would happen is the leaking of k8s' already complex existence from k8s into git. Then it really would break git by making git worse for everyone. I would suggest finding another tool, or making your own.
t3rabytes 4y ago
I feel the title is incorrect. This isn’t so much a critique of Git, it’s a critique of the way Kubernetes-the-project uses it.
- TacticalCoder 4y ago
  > This isn’t so much a critique of Git, it’s a critique of the way Kubernetes-the-project uses it. I'd hope so: the Linux codebase is an order of magnitude bigger than the K8s one and it's not breaking Git.
  - _fat_santa 4y ago
    He mentioned in the article that Kubernetes didn’t have a Linus Tovalds to oversee the project. I found a similar thing to be true with git, as often scaling isn’t so much a technical challenge at first, rather it’s a bunch of org challenges that manifest themselves in technical ways. The model of the “benevolent dictator” kinda works in this case. My last project I was on we managed to scale to about 50k LOC without anything special, the key is I knew the repo like the back of my hand and could catch potential integration issues. While the model works well, it’s very hard to setup as you need a real nerd of a team lead to constantly watch the repo.
    - thrashh 4y ago
      I don’t see why a project should strive for a lack of leadership Most things in life that don’t have leadership become messy and disorganized and eventually disintegrate into an unstable hell with no real focus
- speed_spread 4y ago
  The title is correct as it blames k8s for mishandling git and "breaking" it. It's not git's fault.
  - Karellen 4y ago
    I'm still not quite sure how that counts as breaking git? If you misuse a tool, and the tool performs poorly at the job it's not designed for, but never fails in an unexpected way, and still maintains all the functionality it always had, how have you broken that tool? If try to hammer in a nail with the butt of a screwdriver, and make a complete pigs ear of it, but the screwdriver absorbs the abuse and is still perfectly usable as a screwdriver afterwards, did I "break" the screwdriver? Or, am I misunderstanding how the word "breaking" is being used here? Is there a meaning I'm not getting?
zmmmmm 4y ago
I'd be interested to know why projects like this don't make use of git submodules. They take some getting used to but once understood seem to do a reasonable job of multi-repository projects to me.
- theshrike79 4y ago
  Submodules are an utter clusterfuck. I consider myself a pretty experienced user and every time I try to use submodules, they bite me in the ass eventually. The way they are operated just doesn't fit the way any human thinks.
  - cflewis 4y ago
    Gosh, I’m so pleased for all the replies here. I felt the same way. Submodules made me feel stupid, like my mental model of Git was just entirely broken. What I wanted to do was be able to have people work against a pinned version of a different Git repo, then update the sub module whenever we felt the need and handle the build breaks. This task seemed impossible to do correctly over time, which I just could not understand. How was the submodule getting updated when I didn’t call anything? Why are submodule changes appearing in other commits? I just couldn’t figure it out. I am joining a new project and they started talking about submodules and I what I said was “yeah uhhuh cool” but inside I was pretty nervous . But I couldn’t be sure it wasn’t because I was a dummy and they knew exactly what they were doing, so I kept quiet.
- tssva 4y ago
  The first section of the article discusses submodules and some of the pain points they suffered while moving to them.
- aseipp 4y ago
  Because submodules absolutely suck, and if you have the option of merging the two repos, they're 1000x more terrible. This terribleness increases nearly linearly with the number of active project participants. Why? Because they completely violate Git's operating model, which is to track content, not pointers to content. This is the fundamental problem, regardless of project dynamics, Kubernetes, whatever. In fact, that they make "git commit", one of the most fundamental operations you can perform, significantly riskier, because you can easily introduce submodule pointer changes from your dirty working copy. And people do this all the time. One of the most common cases is committing a submodule "update" while accidentally in a dirty working tree, so when you push the commit pointer simply doesn't exist anywhere. They also fail immediately once you try to `git merge` (or less commonly, git cherry-pick), because their merge conflicts, by definition, cannot be resolved automatically. Because they are pointers, not content. But git works on content. So just to be clear, two of the most fundamental day-to-day operations you can perform are turned into massive liabilities from this feature, ones that are likely to either break your build and/or just make your life harder. As someone who had to maintain stable and development branches of a project, cherry pick between the two, cut releases, etc, submodules are simply hell, because they make an already difficult job worse. This is a good sign that they are a liability. In a past life we actually had so many people push invalid submodule updates over time that we eventually wrote a git hook on our server to reject all commits with submodule updates that didn't exist in the corresponding repository, and that were not specifically tagged in the commit message as updating a submodule (through a magic set of keywords.) The fact we even had to do this is its own pain. I have maintained projects that have long-standing histories with dozens of submodules. And every single time we eliminated one of those submodules (often by merging into the parent repository, or simply dropping the dependency entirely), we all breathed a sigh of relief, and our lives all got significantly better from that point forward. As you can tell, this experience has made me very prepared to fight against submodules everywhere I might see or encounter them. But trust me: it's for your sake, not mine; 'cause there ain't a chance in hell anyone is adding any to my repositories.
cjpearson 4y ago
I didn't really understand the first part. Isn't a monorepo how you get atomicity across sub-projects? If that was a problem for kubernetes, wouldn't a single commit that affected multiple repositories have the same issue? The merge issues seem like they would be solved by your code-hosting platform. (GitLab has Code Owners and Merge Trains and I imagine GitHub has something similar) To me, these features are something you'd implement in your centralized tool rather than git which has to support a decentralized workflow. Perhaps someone clever could think up a decentralized authorization system for git, but is it worth it when almost every project has a centralized source-of-truth repo?
secondcoming 4y ago
I swear to god.. if Kubernetes-creep reaches how git works I'll scream.
smitty1e 4y ago
> The solution might seem simple, but even simple problems become difficult at scale, especially when many different people and organizations are involved. In modeling & simulation, this is called "emergent behavior". While that may be imprecise in terms of the definition, stand by for the effects. Doing anything at scale separates the pros from the dilettantes, e.g., me.
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
aseipp 4y ago
Good overview, I know these sorts of pains well. Lots of hard questions and few definitive wins/right answers. How to organize a massive repository out in the open is still an open question. On that note, recently, I've been experimenting with this project called josh, which basically is like 'git subtree on extreme steroids, functioning as a git proxy': https://josh-project.github.io/josh/ https://josh-project.github.io/josh/ It basically lets you unify/view many repositories as a single one, or equivalent to split a mono-repo into smaller sized units of work for CI, specific teams, etc. It's bidirectional, so you push and pull from josh and everything goes into a single linear history in the mono repo. And because it's bidirectional, people in the mono-repo can still do things like make large-scale atomic changes across all sub-repositories, and those get reflected. Josh currently isn't suitable for a lot of workloads due to various reasons (authentication is one that stands out), but it's actually the first tool I have seen that manages to offer BitKeeper-like "subtrees" that work really well, at scale, for large repos and teams. It requires some care to make sure "sub-trees" can be usable units of work, but it was one of the best features of BK in my opinion and really great for people doing one-off contributions, or isolating trees/changes to specific developers. I'd be interested to know if there are other open alternatives to this. It's a nice point in the design space between solutions like "integrate with the filesystem layer to do sparse clones" or "just split up the repos."
- hirsin 4y ago
  Curious why you say this isn't suitable for authN? This seems maximally suited for processes where changes (or more likely, additions) in a split out system (authn, authz) may immediately rely on behavior or interface changes elsewhere.
  - mh- 4y ago
    I interpreted that to be an example reason, not an example workload.
  - aseipp 4y ago
    I just mean that, I don't see how to properly integrate the josh-server with my SSH keys to do access, how to apply basic roles of who can push/read to what workspace, etc. If I wanted to do this at work "How does it integrate with authentication" is like a #1 concern from an infosec team who'd sign off. If the answer is "It doesn't, use something in front of it" that might be OK too. In fact I think authenticating and authorizing access to components of a monorepo is definitely in scope for its design, and could allow really powerful and cool things. But I just don't think it does that yet. Maybe it shouldn't. That's all I mean. It's not like a hard rule. There were some other things that I thought might hold me up, but I can't remember them now... Not sure if you've ever used it but there's some self-hosted software out there called 'gitolite' that does a lot of this. Perhaps they can be made to work together seamlessly...
chrsig 4y ago
Not that I think submodules are great, but they can be used while still maintaining atomicity. It's just that the atomic update happens when updating the submodule commit in the parent project. This isn't any different than how say a b+tree (or other persistent data structures) rewrite their nodes from the leaf to the root, but leave non-involved subtrees as they were. There winds up being a lot of activity on the superproject that amounts to just updating its submodules, but the commit log for it becomes a linearized history of stable/compatible commit versions. There's definitely room for improvement wrt usability, but the claim that git has No atomicity across subprojects doesn't ring true to me
- mdaniel 4y ago
  I would pay $20 if git would disallow committing from within the submodule directory (single source of truth), because it makes it so easy to forget a git push from within that submodule and now "works on my machine," but "special wtf edition!"
  - tedmiston 4y ago
    Are you saying submodule SHA was updated in the parent repo, but the new submodule commit which the parent repo is now pointing to was never pushed? I haven't experienced that scenario before but it seems like there'd be an obvious git error?
    - mdaniel 4y ago
      No, the opposite: git clone --recursive some/repo.git cd repo/subrepo sed -i"" s/hello/goodbye/ README.md git commit -am 'lololo' cd .. git commit -am'subrepo with *local* sha reference' git status # everything is clean! now that I know that can happen, running `git submodule foreach git status` will surface the "your branch is ahead of" magic text that indicates what has gone on, but it would be tons better if the system understood what was happening and didn't allow such a bad outcome
  - pflanze 4y ago
    You could put a failing script (printing a message and exit 1) at .git/hooks/pre-commit in the submodule[1] (note that this is by-passed if you give --no-verify to git commit). Or you could put a script at .git/hooks/pre-push.sample in the parent repo (untested) that verifies if all commits in submodules are in the respective upstreams. I guess having that functionality in the core as a warning might not be a bad idea (I'm not a Git developer, you'll have to ask them). [1] or rather ../.git/modules/"$submodulename"/hooks/pre-commit, depending on how the submodule was added
stoplying1 4y ago
I guess it's not popular to say but most of k8s's problems here stem from inflexibility of Go. Otherwise it'd probably just be a monorepo still.
- fjdbeb76 4y ago
  Do other devs just open up multiple vs code windows for each project or something? I can't stand not being able to run everything in the same window with ctrl P picking up files from across projects as a reference. I feel like I'm the odd one out because I've noticed a lot of languages and Lang servers are making these assumptions about how devs work and organise code. Or they're just being perfectionist opinionated twats.
canucklady 4y ago
The problem with k8s is the lack of stable interfaces within the codebase. The resources themselves are well-versioned but even though the codebase is split up the individual pieces have dependencies on each other down to specific minor releases. The "separate" repos are so tightly coupled you wonder why they don't just smush them all together. If you've ever tried to develop software that depends on k8s modules you know what I mean - you inevitably get a diamond dependency conflict that go mod can't easily handle because some package needs version 0.45 of apimachinery but something else needs 0.46 (made up versions but you get the point). If they wanted to have many small repos they should have some rigor around versioning and public interfaces between those repos, rather than this magic manifest of specific releases that work together.
- deleted 4y ago
  [deleted]
- dehrmann 4y ago
  Some of this is Google devs think in monorepos, where external interfaces and dependencies are somewhat easier problems.
mirekrusin 4y ago
What will happen when Linus is no longer with us?
- moomin 4y ago
  A damn good question. These old school mega merges are widely avoided in most places these days, and with good reason. So the skills for doing this are thin on the ground. Plus it’s one of the largest projects in the world and whilst Linus got to grow into the current role over decades, a replacement would be starting on an already speeding train. An obvious solution is modularization and stable internal ABIs, but the Linux community have avoided that approach for a long time, and with good reason.
  - mirekrusin 4y ago
    Whatever happens I hope (not for me/us really, but for our kids and next generations) there is a plan and will materialize as single upstream.
- seabrookmx 4y ago
  GKH takes over? IIRC he's already the maintainer for the "stable" branch so a lot of the work people think Linus does is already on his plate.
- cesarb 4y ago
  Your question is traditionally phrased as "what if Linus gets hit by a bus?", and if you search the web for variants of that phrase, you'll see it being discussed as early as last century (it didn't take me long, for instance, to find a slashdot comment from 1999 mentioning that hypothetical scenario). The answer back then was "Alan Cox takes over"; the specific maintainer who takes over has changed over the years (nowadays it's probably Greg Kroah-Hartman), but other than that, the answer has remained the same.
phyrex 4y ago
> The Kubernetes build system is bash. The project experimented with bazel but removed it (too complicated, bad developer experience). I know I’m being very uncharitable but I had to giggle at the irony here
- jbverschoor 4y ago
  Flat out laughed my ass off
deleted 4y ago
[deleted]

76 points by kiyanwang 4y ago

I’ve read the article 3 times now. I still fail to find the part where git broke. I also fail to find a concise explanation on what the solution is. I may need a beer to understand this.

mdaniel 4y ago
As best I can tell, the heart of the alleged issue is this from the middle: > A system that could record atomic commits across projects or a better submodule experience would have allowed for more flexible developer organization, especially as the project grew to a new scale. but otherwise I'm with you that this could have used a better title or something
- thrashh 4y ago
  I can’t tell if it’s because they had a good process and the tools didn’t fit or because they have a terrible process and nothing was going to work for them anyway I see the latter much more often when I jump companies and Google’s projects have had terrible API stability so I’m not really sure Git is to blame here
ec109685 4y ago
GitHub is the system that broke. All the communication systems within it needed to be disabled for custom notifications since it would otherwise be too noisy.
mikl 4y ago
It’s just clickbait, the author posts clickbait on here regularly. It works, unfortunately.

politelemon 4y ago

That was an interesting read thanks for sharing it! My takeaway from this is that the k8s codebase is complicated with complicated workflows. Simple tools won't suffice for this development system. They had to create some of their own tooling; in a way that's quite in line with the usage of k8s, a lot of existing OS concepts had to be recreated just for k8s. > authorization > package management > So why shouldn't a VCS embrace its role as a collaboration tool and explore more generic merge-based optimizations like a queue? In the bottom section there are a few 'wishlist' items, can I call it that. But those aren't good VCS features, they're a reflection of the k8s development world which is not how most of us do development. It's also assuming that because k8s is an all-in-one-doing-many-things, that the VCS it uses should also be a huge all-in-one. I don't think it should; all that would happen is the leaking of k8s' already complex existence from k8s into git. Then it really would break git by making git worse for everyone. I would suggest finding another tool, or making your own.

t3rabytes 4y ago

I feel the title is incorrect. This isn’t so much a critique of Git, it’s a critique of the way Kubernetes-the-project uses it.

TacticalCoder 4y ago
> This isn’t so much a critique of Git, it’s a critique of the way Kubernetes-the-project uses it. I'd hope so: the Linux codebase is an order of magnitude bigger than the K8s one and it's not breaking Git.
- _fat_santa 4y ago
  He mentioned in the article that Kubernetes didn’t have a Linus Tovalds to oversee the project. I found a similar thing to be true with git, as often scaling isn’t so much a technical challenge at first, rather it’s a bunch of org challenges that manifest themselves in technical ways. The model of the “benevolent dictator” kinda works in this case. My last project I was on we managed to scale to about 50k LOC without anything special, the key is I knew the repo like the back of my hand and could catch potential integration issues. While the model works well, it’s very hard to setup as you need a real nerd of a team lead to constantly watch the repo.
  - thrashh 4y ago
    I don’t see why a project should strive for a lack of leadership Most things in life that don’t have leadership become messy and disorganized and eventually disintegrate into an unstable hell with no real focus
speed_spread 4y ago
The title is correct as it blames k8s for mishandling git and "breaking" it. It's not git's fault.
- Karellen 4y ago
  I'm still not quite sure how that counts as breaking git? If you misuse a tool, and the tool performs poorly at the job it's not designed for, but never fails in an unexpected way, and still maintains all the functionality it always had, how have you broken that tool? If try to hammer in a nail with the butt of a screwdriver, and make a complete pigs ear of it, but the screwdriver absorbs the abuse and is still perfectly usable as a screwdriver afterwards, did I "break" the screwdriver? Or, am I misunderstanding how the word "breaking" is being used here? Is there a meaning I'm not getting?

zmmmmm 4y ago

I'd be interested to know why projects like this don't make use of git submodules. They take some getting used to but once understood seem to do a reasonable job of multi-repository projects to me.

theshrike79 4y ago
Submodules are an utter clusterfuck. I consider myself a pretty experienced user and every time I try to use submodules, they bite me in the ass eventually. The way they are operated just doesn't fit the way any human thinks.
- cflewis 4y ago
  Gosh, I’m so pleased for all the replies here. I felt the same way. Submodules made me feel stupid, like my mental model of Git was just entirely broken. What I wanted to do was be able to have people work against a pinned version of a different Git repo, then update the sub module whenever we felt the need and handle the build breaks. This task seemed impossible to do correctly over time, which I just could not understand. How was the submodule getting updated when I didn’t call anything? Why are submodule changes appearing in other commits? I just couldn’t figure it out. I am joining a new project and they started talking about submodules and I what I said was “yeah uhhuh cool” but inside I was pretty nervous . But I couldn’t be sure it wasn’t because I was a dummy and they knew exactly what they were doing, so I kept quiet.
tssva 4y ago
The first section of the article discusses submodules and some of the pain points they suffered while moving to them.
aseipp 4y ago
Because submodules absolutely suck, and if you have the option of merging the two repos, they're 1000x more terrible. This terribleness increases nearly linearly with the number of active project participants. Why? Because they completely violate Git's operating model, which is to track content, not pointers to content. This is the fundamental problem, regardless of project dynamics, Kubernetes, whatever. In fact, that they make "git commit", one of the most fundamental operations you can perform, significantly riskier, because you can easily introduce submodule pointer changes from your dirty working copy. And people do this all the time. One of the most common cases is committing a submodule "update" while accidentally in a dirty working tree, so when you push the commit pointer simply doesn't exist anywhere. They also fail immediately once you try to `git merge` (or less commonly, git cherry-pick), because their merge conflicts, by definition, cannot be resolved automatically. Because they are pointers, not content. But git works on content. So just to be clear, two of the most fundamental day-to-day operations you can perform are turned into massive liabilities from this feature, ones that are likely to either break your build and/or just make your life harder. As someone who had to maintain stable and development branches of a project, cherry pick between the two, cut releases, etc, submodules are simply hell, because they make an already difficult job worse. This is a good sign that they are a liability. In a past life we actually had so many people push invalid submodule updates over time that we eventually wrote a git hook on our server to reject all commits with submodule updates that didn't exist in the corresponding repository, and that were not specifically tagged in the commit message as updating a submodule (through a magic set of keywords.) The fact we even had to do this is its own pain. I have maintained projects that have long-standing histories with dozens of submodules. And every single time we eliminated one of those submodules (often by merging into the parent repository, or simply dropping the dependency entirely), we all breathed a sigh of relief, and our lives all got significantly better from that point forward. As you can tell, this experience has made me very prepared to fight against submodules everywhere I might see or encounter them. But trust me: it's for your sake, not mine; 'cause there ain't a chance in hell anyone is adding any to my repositories.

cjpearson 4y ago

I didn't really understand the first part. Isn't a monorepo how you get atomicity across sub-projects? If that was a problem for kubernetes, wouldn't a single commit that affected multiple repositories have the same issue? The merge issues seem like they would be solved by your code-hosting platform. (GitLab has Code Owners and Merge Trains and I imagine GitHub has something similar) To me, these features are something you'd implement in your centralized tool rather than git which has to support a decentralized workflow. Perhaps someone clever could think up a decentralized authorization system for git, but is it worth it when almost every project has a centralized source-of-truth repo?

secondcoming 4y ago

I swear to god.. if Kubernetes-creep reaches how git works I'll scream.

smitty1e 4y ago

> The solution might seem simple, but even simple problems become difficult at scale, especially when many different people and organizations are involved. In modeling & simulation, this is called "emergent behavior". While that may be imprecise in terms of the definition, stand by for the effects. Doing anything at scale separates the pros from the dilettantes, e.g., me.

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

aseipp 4y ago

Good overview, I know these sorts of pains well. Lots of hard questions and few definitive wins/right answers. How to organize a massive repository out in the open is still an open question. On that note, recently, I've been experimenting with this project called josh, which basically is like 'git subtree on extreme steroids, functioning as a git proxy': https://josh-project.github.io/josh/ https://josh-project.github.io/josh/ It basically lets you unify/view many repositories as a single one, or equivalent to split a mono-repo into smaller sized units of work for CI, specific teams, etc. It's bidirectional, so you push and pull from josh and everything goes into a single linear history in the mono repo. And because it's bidirectional, people in the mono-repo can still do things like make large-scale atomic changes across all sub-repositories, and those get reflected. Josh currently isn't suitable for a lot of workloads due to various reasons (authentication is one that stands out), but it's actually the first tool I have seen that manages to offer BitKeeper-like "subtrees" that work really well, at scale, for large repos and teams. It requires some care to make sure "sub-trees" can be usable units of work, but it was one of the best features of BK in my opinion and really great for people doing one-off contributions, or isolating trees/changes to specific developers. I'd be interested to know if there are other open alternatives to this. It's a nice point in the design space between solutions like "integrate with the filesystem layer to do sparse clones" or "just split up the repos."

hirsin 4y ago
Curious why you say this isn't suitable for authN? This seems maximally suited for processes where changes (or more likely, additions) in a split out system (authn, authz) may immediately rely on behavior or interface changes elsewhere.
- mh- 4y ago
  I interpreted that to be an example reason, not an example workload.
- aseipp 4y ago
  I just mean that, I don't see how to properly integrate the josh-server with my SSH keys to do access, how to apply basic roles of who can push/read to what workspace, etc. If I wanted to do this at work "How does it integrate with authentication" is like a #1 concern from an infosec team who'd sign off. If the answer is "It doesn't, use something in front of it" that might be OK too. In fact I think authenticating and authorizing access to components of a monorepo is definitely in scope for its design, and could allow really powerful and cool things. But I just don't think it does that yet. Maybe it shouldn't. That's all I mean. It's not like a hard rule. There were some other things that I thought might hold me up, but I can't remember them now... Not sure if you've ever used it but there's some self-hosted software out there called 'gitolite' that does a lot of this. Perhaps they can be made to work together seamlessly...

chrsig 4y ago

Not that I think submodules are great, but they can be used while still maintaining atomicity. It's just that the atomic update happens when updating the submodule commit in the parent project. This isn't any different than how say a b+tree (or other persistent data structures) rewrite their nodes from the leaf to the root, but leave non-involved subtrees as they were. There winds up being a lot of activity on the superproject that amounts to just updating its submodules, but the commit log for it becomes a linearized history of stable/compatible commit versions. There's definitely room for improvement wrt usability, but the claim that git has No atomicity across subprojects doesn't ring true to me

mdaniel 4y ago
I would pay $20 if git would disallow committing from within the submodule directory (single source of truth), because it makes it so easy to forget a git push from within that submodule and now "works on my machine," but "special wtf edition!"
- tedmiston 4y ago
  Are you saying submodule SHA was updated in the parent repo, but the new submodule commit which the parent repo is now pointing to was never pushed? I haven't experienced that scenario before but it seems like there'd be an obvious git error?
  - mdaniel 4y ago
    No, the opposite: git clone --recursive some/repo.git cd repo/subrepo sed -i"" s/hello/goodbye/ README.md git commit -am 'lololo' cd .. git commit -am'subrepo with *local* sha reference' git status # everything is clean! now that I know that can happen, running `git submodule foreach git status` will surface the "your branch is ahead of" magic text that indicates what has gone on, but it would be tons better if the system understood what was happening and didn't allow such a bad outcome
- pflanze 4y ago
  You could put a failing script (printing a message and exit 1) at .git/hooks/pre-commit in the submodule[1] (note that this is by-passed if you give --no-verify to git commit). Or you could put a script at .git/hooks/pre-push.sample in the parent repo (untested) that verifies if all commits in submodules are in the respective upstreams. I guess having that functionality in the core as a warning might not be a bad idea (I'm not a Git developer, you'll have to ask them). [1] or rather ../.git/modules/"$submodulename"/hooks/pre-commit, depending on how the submodule was added

stoplying1 4y ago

I guess it's not popular to say but most of k8s's problems here stem from inflexibility of Go. Otherwise it'd probably just be a monorepo still.

fjdbeb76 4y ago
Do other devs just open up multiple vs code windows for each project or something? I can't stand not being able to run everything in the same window with ctrl P picking up files from across projects as a reference. I feel like I'm the odd one out because I've noticed a lot of languages and Lang servers are making these assumptions about how devs work and organise code. Or they're just being perfectionist opinionated twats.

canucklady 4y ago

The problem with k8s is the lack of stable interfaces within the codebase. The resources themselves are well-versioned but even though the codebase is split up the individual pieces have dependencies on each other down to specific minor releases. The "separate" repos are so tightly coupled you wonder why they don't just smush them all together. If you've ever tried to develop software that depends on k8s modules you know what I mean - you inevitably get a diamond dependency conflict that go mod can't easily handle because some package needs version 0.45 of apimachinery but something else needs 0.46 (made up versions but you get the point). If they wanted to have many small repos they should have some rigor around versioning and public interfaces between those repos, rather than this magic manifest of specific releases that work together.

deleted 4y ago
[deleted]
dehrmann 4y ago
Some of this is Google devs think in monorepos, where external interfaces and dependencies are somewhat easier problems.

mirekrusin 4y ago

What will happen when Linus is no longer with us?

moomin 4y ago
A damn good question. These old school mega merges are widely avoided in most places these days, and with good reason. So the skills for doing this are thin on the ground. Plus it’s one of the largest projects in the world and whilst Linus got to grow into the current role over decades, a replacement would be starting on an already speeding train. An obvious solution is modularization and stable internal ABIs, but the Linux community have avoided that approach for a long time, and with good reason.
- mirekrusin 4y ago
  Whatever happens I hope (not for me/us really, but for our kids and next generations) there is a plan and will materialize as single upstream.
seabrookmx 4y ago
GKH takes over? IIRC he's already the maintainer for the "stable" branch so a lot of the work people think Linus does is already on his plate.
cesarb 4y ago
Your question is traditionally phrased as "what if Linus gets hit by a bus?", and if you search the web for variants of that phrase, you'll see it being discussed as early as last century (it didn't take me long, for instance, to find a slashdot comment from 1999 mentioning that hypothetical scenario). The answer back then was "Alan Cox takes over"; the specific maintainer who takes over has changed over the years (nowadays it's probably Greg Kroah-Hartman), but other than that, the answer has remained the same.

phyrex 4y ago

> The Kubernetes build system is bash. The project experimented with bazel but removed it (too complicated, bad developer experience). I know I’m being very uncharitable but I had to giggle at the irony here

jbverschoor 4y ago
Flat out laughed my ass off

deleted 4y ago

[deleted]