4 ms·

Think if you have that many developers you need to start doing it Linux kernel style and have maintainers responsible for certain parts of the project making su

by ilyt 3y ago

Think if you have that many developers you need to start doing it Linux kernel style and have maintainers responsible for certain parts of the project making sure merges go in fine. And maybe have actual communication in other means than via comments on tickets. The whole hypothetical example could be avoided by typing "hey, I'm changing this field to be that, anything against?" on team's chat > Since my_branch was merged before it was rebased onto someone_elses_branch, this error goes unnoticed and BAM - main is broken! This is known as a semantic merge conflict — a merge is technically possible, but results in a regression. Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches. I'm also confused how that even fixes anything, someone still have to go back and fix that code.

luketheobscure 3y ago
> Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches. On projects with a high commit velocity and a large team, simply merging main and running the tests is not enough. The likelihood that a new commit lands on main before your tests finish is high, and restarting the process every time there's a new commit is time consuming. Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits.
- pperi11 3y ago
  You could just use smaller PRs See: https://graphite.dev/blog/the-ideal-pr-is-50-lines-long https://graphite.dev/blog/the-ideal-pr-is-50-lines-long
- throwbadubadu 3y ago
  > Do people not merge master into their branch before pushing? No, they rebase! (Sorry, couldn't resist, yeah the other half does this (!attention, opinion!) ugly cross merges that make the commit graph look like my cables under the desk... > Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits. Not the only one (though you may argue this is a degenerated queue): use merge locks! Yeah, we really use git svn style, the locking is done via a chat channel. No, please don't laugh. Acquire the lock, merge develop or rebase, check everything again, release lock :D :D (No again: I am not suggesting this to anyone!! :D )
  - zizee 3y ago
    The point the GP was making is that if merging to main is frequent (bigger team) and build times are long (bigger projects), catching your branch up to master before merging can be hard, as master is moving ahead faster than you rebase+build.
    - throwbadubadu 3y ago
      Fully understood.. we also have a big team, a too slow CI, and much too many LOCs. The "merge lock" will ensure you catch up, you keep the lock as long as it takes (or until others get too annoyed and urge you to step back) - as long as you have it nothing comes between you and main at the point you claimed it.
      - Thorentis 3y ago
        That sounds like an awful dev loop.
        deleted 3y ago
        [deleted]
- ilyt 3y ago
  Right but biggest projects out there like Linux kernel manage it just fine > The likelihood that a new commit lands on main before your tests finish is high, and restarting the process every time there's a new commit is time consuming. Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits. But the code is still broken and still someone has to fix it ? That doesn't help in that. As for "broken main", how is just not allowing code that doesn't pass tests to merge to main not enough ? It all seems like a lot of complexity added because devs don't want to change shitty work practices...
deleted 3y ago
[deleted]
zuckerborgggg 3y ago
[flagged]
Sohcahtoa82 3y ago
> Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches. Well, yes, but you have to run your tests. The article outlines this scenario, but to give a more concrete definition, imagine your tests take 10 minutes, and this timeline happens: 10:00 Co-worker rebases their branch with master and begins the unit testing. 10:05 You rebase your branch with master and begin unit testing. 10:10 Co-worker's tests finish and everything looks green, pushes to master. 10:15 Your tests are done and everything is green. You try to push, but your branch is now out of date because master contains your co-workers changes! You rebase on master, restart your tests... 10:25 Your tests finish, it's green, and you finally push to master. Ten minutes got wasted. It doesn't seem like a lot, but with enough engineers (Or long enough testing time), this scenario can happen a lot, especially with very short-lived branches.
- ilyt 3y ago
  And how merge queue not wastes that time ? Someone will have to take that step to update their code and fix it then test
  - Sohcahtoa82 3y ago
    With the merge queue, it works like this: 10:00 Co-worker pushes their code to the merge queue. CI/CD pipeline begins running unit tests. 10:05 You push your code to the queue. 10:10 CI/CD finishes the tests. It looks green. Their code gets merged to master. CI/CD starts running your unit tests with the new master branch and your code applied. 10:20 CI/CD finishes the tests with your code, tests are green, and merges to master. Five minutes were saved by using a merge queue.
dilyevsky 3y ago
When i starter my career working for large corp we had dedicated people for this called Release Engineers. The concept of specialization seems to have fallen out of favor within engineering orgs…
Cthulhu_ 3y ago
> The whole hypothetical example could be avoided by typing "hey, I'm changing this field to be that, anything against?" on team's chat This doesn't scale. With 15 developers, you could be working on ten different features at the same time; getting alignment on any field change (not even APIs) between that many people and projects does not scale. And 15 is still considered a small team. > Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches. Again, this does not scale . Every time you merge master into your branch, you have to wait for the CI to pass before you can merge. If in the meantime someone else merges their change before you - and this happens asynchronously, whenever it's ready / whenever someone gets to it - you have to merge master again and wait for CI again. And that takes your focus off of whatever else you want to work on next, if you're waiting for CI and hoping nobody else merges before you, you're epitomizing the "Compiling!" comic. Your idea works, but only for smaller teams, faster CI pipelines, and lower frequency of merge attempts. Merge queues on the other hand is fire & forget; it enforces a linear process - all features will be "master + this one feature" so you don't get two merges interfering with each other - and it allows the owner to move on to the next. Unless CI breaks, which is actionable. It allows for an asynchronous process with less gatekeeping, coordination, waiting for things to happen, etc. It scales better.

by ilyt 3y ago

luketheobscure 3y ago

> Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches. On projects with a high commit velocity and a large team, simply merging main and running the tests is not enough. The likelihood that a new commit lands on main before your tests finish is high, and restarting the process every time there's a new commit is time consuming. Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits.

pperi11 3y ago
You could just use smaller PRs See: https://graphite.dev/blog/the-ideal-pr-is-50-lines-long https://graphite.dev/blog/the-ideal-pr-is-50-lines-long
throwbadubadu 3y ago
> Do people not merge master into their branch before pushing? No, they rebase! (Sorry, couldn't resist, yeah the other half does this (!attention, opinion!) ugly cross merges that make the commit graph look like my cables under the desk... > Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits. Not the only one (though you may argue this is a degenerated queue): use merge locks! Yeah, we really use git svn style, the locking is done via a chat channel. No, please don't laugh. Acquire the lock, merge develop or rebase, check everything again, release lock :D :D (No again: I am not suggesting this to anyone!! :D )
- zizee 3y ago
  The point the GP was making is that if merging to main is frequent (bigger team) and build times are long (bigger projects), catching your branch up to master before merging can be hard, as master is moving ahead faster than you rebase+build.
  - throwbadubadu 3y ago
    Fully understood.. we also have a big team, a too slow CI, and much too many LOCs. The "merge lock" will ensure you catch up, you keep the lock as long as it takes (or until others get too annoyed and urge you to step back) - as long as you have it nothing comes between you and main at the point you claimed it.
    - Thorentis 3y ago
      That sounds like an awful dev loop.
      - deleted 3y ago
        [deleted]
ilyt 3y ago
Right but biggest projects out there like Linux kernel manage it just fine > The likelihood that a new commit lands on main before your tests finish is high, and restarting the process every time there's a new commit is time consuming. Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits. But the code is still broken and still someone has to fix it ? That doesn't help in that. As for "broken main", how is just not allowing code that doesn't pass tests to merge to main not enough ? It all seems like a lot of complexity added because devs don't want to change shitty work practices...

deleted 3y ago

[deleted]

zuckerborgggg 3y ago

[flagged]

Sohcahtoa82 3y ago

> Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches. Well, yes, but you have to run your tests. The article outlines this scenario, but to give a more concrete definition, imagine your tests take 10 minutes, and this timeline happens: 10:00 Co-worker rebases their branch with master and begins the unit testing. 10:05 You rebase your branch with master and begin unit testing. 10:10 Co-worker's tests finish and everything looks green, pushes to master. 10:15 Your tests are done and everything is green. You try to push, but your branch is now out of date because master contains your co-workers changes! You rebase on master, restart your tests... 10:25 Your tests finish, it's green, and you finally push to master. Ten minutes got wasted. It doesn't seem like a lot, but with enough engineers (Or long enough testing time), this scenario can happen a lot, especially with very short-lived branches.

ilyt 3y ago
And how merge queue not wastes that time ? Someone will have to take that step to update their code and fix it then test
- Sohcahtoa82 3y ago
  With the merge queue, it works like this: 10:00 Co-worker pushes their code to the merge queue. CI/CD pipeline begins running unit tests. 10:05 You push your code to the queue. 10:10 CI/CD finishes the tests. It looks green. Their code gets merged to master. CI/CD starts running your unit tests with the new master branch and your code applied. 10:20 CI/CD finishes the tests with your code, tests are green, and merges to master. Five minutes were saved by using a merge queue.

dilyevsky 3y ago

When i starter my career working for large corp we had dedicated people for this called Release Engineers. The concept of specialization seems to have fallen out of favor within engineering orgs…

Cthulhu_ 3y ago

> The whole hypothetical example could be avoided by typing "hey, I'm changing this field to be that, anything against?" on team's chat This doesn't scale. With 15 developers, you could be working on ten different features at the same time; getting alignment on any field change (not even APIs) between that many people and projects does not scale. And 15 is still considered a small team. > Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches. Again, this does not scale . Every time you merge master into your branch, you have to wait for the CI to pass before you can merge. If in the meantime someone else merges their change before you - and this happens asynchronously, whenever it's ready / whenever someone gets to it - you have to merge master again and wait for CI again. And that takes your focus off of whatever else you want to work on next, if you're waiting for CI and hoping nobody else merges before you, you're epitomizing the "Compiling!" comic. Your idea works, but only for smaller teams, faster CI pipelines, and lower frequency of merge attempts. Merge queues on the other hand is fire & forget; it enforces a linear process - all features will be "master + this one feature" so you don't get two merges interfering with each other - and it allows the owner to move on to the next. Unless CI breaks, which is actionable. It allows for an asynchronous process with less gatekeeping, coordination, waiting for things to happen, etc. It scales better.