3 ms·

> 1 PR, you know what introduced the breakage. 30? Good luck. To be clearer, I mean there would be say 30 pull requests merged into "develop" and they would o

> 1 PR, you know what introduced the breakage. 30? Good luck. To be clearer, I mean there would be say 30 pull requests merged into "develop" and they would only be merged into "master" and pushed live after detailed manual testing. This is versus merging each pull request directly into "master" and going live immediately, where you wouldn't have time to do detailed manual testing every merge. You would still have the git history in both approaches to debug.

lostcolony 6y ago
Right. I'm saying there are two different things here. Regardless of your merge strategy. You either test after each PR merge, or you test only when you 'craft a release'. Doesn't matter whether this merge is into a develop branch or straight into master; it's solely a question of when you test it. If you manually test, testing only a batch of changes, together, is obviously easier, from a testing perspective. However, the effort to figure out and fix the issue can be very large. Plus, after fixing that one issue, you have to retest everything. So every bug requires retesting in any case (so while it's still a lower testing burden, unless you introduced 30+ unrelated issues, it's not as low as you think it might be on the surface). Compared with testing (and at that point you might as well release if your pipeline and org let you) with each PR - any manual testing effort is obviously higher due to testing so often (whereas automated is no extra effort), but, figuring out what caused the breakage, and addressing it, is much, much easier, order of magnitude easier, to figure out, and fixing it is much less likely to introduce new issues. Which is more to the original point; catching a bug earlier, allows the fix to be more targeted, which makes it more likely to be right, which reduces the likelihood of it making it to prod. There is also, to the original point, a statistical thing to consider. Let's say in 30 PRs, there is one that introduces a bug. Well, if QA is testing the one PR that broke something, they're more likely to spot it, since they're focusing on that PR. They're less likely if it's one of thirty sets of changes. But beyond that, let's say, in both situations, QA misses it. If you have one deploy, you've broken prod 100% of your releases (1 of 1 release). If you have 30 deploys, you've broken prod only ~3% of your releases. So immediately the original statement is validated; if your testing effort is the same (big if!), more frequent releases, with smaller changes, means the same or better percentage of deploying working code. Also, if you have to rollback with one mega release, you lose all 29 other changes; if you have to rollback with separate changes, you lose no other changes.

lostcolony 6y ago

Right. I'm saying there are two different things here. Regardless of your merge strategy. You either test after each PR merge, or you test only when you 'craft a release'. Doesn't matter whether this merge is into a develop branch or straight into master; it's solely a question of when you test it. If you manually test, testing only a batch of changes, together, is obviously easier, from a testing perspective. However, the effort to figure out and fix the issue can be very large. Plus, after fixing that one issue, you have to retest everything. So every bug requires retesting in any case (so while it's still a lower testing burden, unless you introduced 30+ unrelated issues, it's not as low as you think it might be on the surface). Compared with testing (and at that point you might as well release if your pipeline and org let you) with each PR - any manual testing effort is obviously higher due to testing so often (whereas automated is no extra effort), but, figuring out what caused the breakage, and addressing it, is much, much easier, order of magnitude easier, to figure out, and fixing it is much less likely to introduce new issues. Which is more to the original point; catching a bug earlier, allows the fix to be more targeted, which makes it more likely to be right, which reduces the likelihood of it making it to prod. There is also, to the original point, a statistical thing to consider. Let's say in 30 PRs, there is one that introduces a bug. Well, if QA is testing the one PR that broke something, they're more likely to spot it, since they're focusing on that PR. They're less likely if it's one of thirty sets of changes. But beyond that, let's say, in both situations, QA misses it. If you have one deploy, you've broken prod 100% of your releases (1 of 1 release). If you have 30 deploys, you've broken prod only ~3% of your releases. So immediately the original statement is validated; if your testing effort is the same (big if!), more frequent releases, with smaller changes, means the same or better percentage of deploying working code. Also, if you have to rollback with one mega release, you lose all 29 other changes; if you have to rollback with separate changes, you lose no other changes.