4 ms·

> I've heard descriptions and seen blog entries about many other large companies build systems, but to be honest, nothing even comes close to the amazing techno

> I've heard descriptions and seen blog entries about many other large companies build systems, but to be honest, nothing even comes close to the amazing technology Amazon has produced. > Say what you will about Amazon's frugality, their turnover, and their perks, but the tools available at Amazon make it a world-class place to build software. Haha, funny. I mainly object to the fawning here. There's a lot of good ideas in the build system, but there's also more to it, like CI. Any (hypothetical) build system that runs the tests after commits have been merged is not great. Does constantly blocked pipelines sound fun? (Edit: one of the comments mentioned dry-runs, but for example, if the merge behaviour of the code review tool was atrocious, so people were accustomed to overriding it, and people did not pay attention to the dry-run, because there's no huge green tick like GitHub, then that may be ignored far too often) There's also other issues. A build system that was originally made for e.g. C++ might struggle with other languages, and the experience be sucky. As good as something like Apollo sounds it might not deal too well with deploying e.g. Lambdas, or other cloud resources. Don't drink the cool aid. There are some good lessons to be learned, but you can have a pretty great system with e.g. GitHub or GitLab - maybe even better for developer productivity while not being weighed down by years of legacy and inertia.

nickm12 6y ago
> Any (hypothetical) build system that runs the tests after commits have been merged is not great. This is true and it's a problem with Brazil at Amazon. Certainly, if every developer does a dry-run build they can verify that their change will build and test. Dry run builds are super-easy to do, run pretty fast, and there's a great interface for seeing the test output logs if it fails. But if a developer has permission to push to a package they can do it with doing a dry-run and then the tip of your main branch is not green. That said, I've also worked with a CI/CD system (in a large monorepo) that tested every commit before merging. Even that system could not guarantee that the tip of the main branch was green, because the change velocity was so high it wasn't practical to serialize the changes. Instead, every change was tested against a known green revision and if it passed and merged cleanly it would be put on the master branch. However, the master branch still had to be tested again because it was possible for two changes to test green and merge cleanly into a revision that failed tests. Serializing would have taken a lot longer to run (since you couldn't test in parallel) and one bad commit would flush the entire queue, meaning your change might be delayed by someone else's bad commit. I think this was the right trade-off for that system.

nickm12 6y ago

> Any (hypothetical) build system that runs the tests after commits have been merged is not great. This is true and it's a problem with Brazil at Amazon. Certainly, if every developer does a dry-run build they can verify that their change will build and test. Dry run builds are super-easy to do, run pretty fast, and there's a great interface for seeing the test output logs if it fails. But if a developer has permission to push to a package they can do it with doing a dry-run and then the tip of your main branch is not green. That said, I've also worked with a CI/CD system (in a large monorepo) that tested every commit before merging. Even that system could not guarantee that the tip of the main branch was green, because the change velocity was so high it wasn't practical to serialize the changes. Instead, every change was tested against a known green revision and if it passed and merged cleanly it would be put on the master branch. However, the master branch still had to be tested again because it was possible for two changes to test green and merge cleanly into a revision that failed tests. Serializing would have taken a lot longer to run (since you couldn't test in parallel) and one bad commit would flush the entire queue, meaning your change might be delayed by someone else's bad commit. I think this was the right trade-off for that system.