4 ms·
> Does anyone have any other practices they can recommend for managing these type of projects? Honestly, the only way around these sorts of issues is to utiliz
by reportt 5y ago
> Does anyone have any other practices they can recommend for managing these type of projects?
Honestly, the only way around these sorts of issues is to utilize automation in some form.
I've found that setting up repositories (like devpi[0], Artifactory[1], or Docker Registry[2]) on a shared network location (for your project, could be local if you work alone) and using CI/CD tools (like Jenkins[3]) are the key. The goal is that you end up working on one portion of the code base at a time, and those need to go through the standard validation processes so that you can pull in the updated package version when you work on something down-stream. You making sure that the CI/CD environment _doesn't_ have access to other packages's non-versioned code is key for making sure things actually work as expected.
For example, if you have FooLib, and you need an update in that for BarApp, then even if you branch FooLib 1.2.3 to 1.2.3-1-gabc1234d (the `git describe` of the commit) on `feat/new-thingy` , then even if BarApp v2.3.4-1-gaf901234 depends on that new branch, it shouldn't be in any way able to reference that branch on the CI/CD build process. How do you get around this? Good development -- finish the FooLib branch, get that working, merge it in with the updated version, and push the package (with the new version) to the CI/CD-accessible repository. At that point, when you push your BarApp change, it can actually build and not die. But until FooLib has got a versioned update, BarApp's branch _shouldn't_ be able to build.
The statement of "But I want to work on the changes locally, in parallel" is valid. That's what local development is for -- giving you space to work on related things that don't impact the upstream codebase. You should have the option to utilize FooLib's branch code in your BarApp code locally, and you can often do that via things like `pip install` or `maven install` or whatever the relevant local install command is. At this point, the package still probably has the same version number, so the local build doesn't trigger issues. You can work on the two and tweak and twist as you want, but refrain from actually trying to push BarApp referencing FooLib's branch until it's actually in the repo.
This all takes a great deal of restraint and patience. The goal here is make it just a tad harder to introduce problems somewhere since you can't depend on something that hasn't been given the go-ahead. While there might be a lot of "Updated FooLib requirement to v1.2.4" throughout your codebase, why are you doing that just off-hand? If you are doing it because of a security issue or bug, let that be known in the commit message. If you are doing it because you can utilize a new feature/whatever, your commit message won't be just "Updated FooLib", you likely are doing "Added Feature X2Y, updated FooLib to 1.2.4".
PHP I try not to touch much, simply because I've always had bad experiences. I know for a fact that there are decent ways to do it with build tools like Maven[4], setuptools[5], and Docker[6]. Hell, I have used Docker as a way to introduce versioned dependency packaging, only needing to use Docker Registry (each dependent project does a multi-stage build, pulling in the dependencies via the versioned package images).
---
[0]: https://devpi.net/docs/devpi/devpi/latest/%2Bd/index.html https://devpi.net/docs/devpi/devpi/latest/%2Bd/index.html
[1]: https://jfrog.com/artifactory/ https://jfrog.com/artifactory/
[2]: https://docs.docker.com/registry/ https://docs.docker.com/registry/
[3]: https://www.jenkins.io/ https://www.jenkins.io/
[4]: https://maven.apache.org/ https://maven.apache.org/
[5]: https://setuptools.readthedocs.io/en/latest/ https://setuptools.readthedocs.io/en/latest/
[6]: https://www.docker.com/ https://www.docker.com/
- hvidgaard 5y agoI agree. Having a huge mono repo is basically throwing the towel in the ring with your automation/package management/dependency validation. In the .net world your CI/CD pipeline should continuously build and publish NuGet packages of your common code as you make changes. Since the old versions are obviously still available, other parts of the system are not forced to be updated to the new version of the dependency.
- tsimionescu 5y ago> But until FooLib has got a versioned update, BarApp's branch _shouldn't_ be able to build. This is such a horrible practice. You're creating mountains of extra work, and encouraging devs to delay integration testing, which is certain to lead to cycles of rework. It also only 'works' on toy features. When you're building a complex feature that requires a few weeks of work and a few devs, it quickly breaks down, and further prevents early QA testing of the new feature itself. Unfortunately this practice is often forced on people by reliance on the horrid SemVer scheme, which only makes any kind of sense for 3rd party dependencies, but is foisted on internal dependencies as well by many idiotic package managers, like Go mod or NPM.
- bluGill 5y agoIf you have a CI system your version patch will always increase and you can then always integrate the latest dev version. Most of your developer issues will be someone not using the latest versions for everything.
- tsimionescu 5y agoHow will that work with in development branches? At any one time, there are multiple sub-teams developing multiple independent sub-features all impacting some of the same components. How are they supposed to do this if they can't branch out the components and each work on their own independent branch of the integrated application? There isn't a single 'latest dev version', there are many.
- 5y ago