2 ms·

so what I'm hearing is that app-1.0 needs app-1.0-runtime-build-20240410 which was, itself, built from a base of runtime-y-2.0 and layering library-x-1.11 upon

so what I'm hearing is that app-1.0 needs app-1.0-runtime-build-20240410 which was, itself, built from a base of runtime-y-2.0 and layering library-x-1.11 upon it, kind of like # in some "app-runtimes" project, they assemble your app's runtime cat > Dockerfile <<FOO FROM public.ecr.aws/runtimes/runtime-y:2.0 ADD https://cache.example/library-x/1.1/library-x-1.1.jar FOO tar -cf - Dockerfile | podman build -t public.ecr.aws/app-runtimes/app-1.0-runtime-build:20240410 - # then you consume it in your project cat > Dockerfile <<FOO FROM public.ecr.aws/app-runtimes/app-1.0-runtime-build:20240410 ADD ./app-1.0.jar FOO cat > .gitlab-ci.yml <<'YML' # you can also distribute artifacts other than just docker images # https://docs.gitlab.com/ee/user/packages/package_registry/supported_package_managers.html cook image: stage: package script: # or this https://docs.gitlab.com/ee/topics/autodevops/customize.html#customize-buildpacks-with-cloud-native-buildpacks - podman build -t $CI_REGISTRY_IMAGE . # https://docs.gitlab.com/ee/user/packages/#container-registry is built in - podman push $CI_REGISTRY_IMAGE review env: stage: staging script: auto-devops deploy # for free: https://docs.gitlab.com/ee/ci/review_apps/index.html environment: name: review/${CI_COMMIT_REF_SLUG} url: https://${CI_ENVIRONMENT_SLUG}.int.example on_stop: teardown-review teardown-review: stage: staging script: auto-devops stop when: manual environment: name: review/${CI_COMMIT_REF_SLUG} action: stop ... etc ... YML and then, yadda, yadda, blue-green, incremental rollout <https://gitlab.com/gitlab-org/gitlab/-/blob/v16.10.2-ee/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml#L81-84 https://gitlab.com/gitlab-org/gitlab/-/blob/v16.10.2-ee/lib/...>, feature flags <https://docs.gitlab.com/ee/operations/feature_flags.html https://docs.gitlab.com/ee/operations/feature_flags.html>, error capture <https://docs.gitlab.com/ee/operations/error_tracking.html#integrated-error-tracking https://docs.gitlab.com/ee/operations/error_tracking.html#in...>, project-managed provisioning <https://docs.gitlab.com/ee/user/infrastructure/iac/#integrate-your-project-with-terraform https://docs.gitlab.com/ee/user/infrastructure/iac/#integrat...>, on call management <https://docs.gitlab.com/ee/operations/incident_management/ https://docs.gitlab.com/ee/operations/incident_management/>, on call runbooks <https://docs.gitlab.com/ee/user/project/clusters/runbooks/index.html https://docs.gitlab.com/ee/user/project/clusters/runbooks/in...> you can orchestrate all that from ~~Slack~~ Chime :-D if you're into that kind of thing https://docs.gitlab.com/ee/ci/chatops/ https://docs.gitlab.com/ee/ci/chatops/

xyzzy_plugh 3y ago
No, not even close. You might even have it exactly backwards.
- mdaniel 3y ago
  which is why, as I originally asked GP: what have you already tried and what features were they missing I presume by "exactly backwards" you mean that one should have absolutely zero knobs to influence anything because the Almighty Jeff Build System does all the things, which GitLab also supports but is less amusing to look at on an Internet forum because it's "you can't modify anything, it just works, trust me" Or, you know, if you have something constructive to add to this discussion feel free to use more words than "lol, no"
  - xyzzy_plugh 3y ago
    I don't work at Amazon, and haven't for a long time, and this format is insufficient to fully express what they're doing, so I won't try. You're better off searching for how Brazil and Apollo work. That being said, the short of it is that: imagine when you push a new revision to source control, you (you) can run jobs testing every potential consumer of that new revision. As in, you push libx-1.1.2 and anyone consuming libx >= 1.1 (or any variety of filters) is identified. If the tests succeed, you can update their dependencies on your package and even deploy them, safely and gradually, to production without involving the downstream teams at all. If they don't, you can choose your own adventure: pin them, fork, fix them, patch the package, revise the versioning, whatever you want. It's designed to be extremely safe and put power in the hands of those updating dependencies to do so safely within reason. Imagine you work on a library and you can test your PR against every consumers. It's not unlike what Google and other monorepos accomplish but it's quite different also. You can have many live versions simultaneously. You don't have to slog it out and patch all the dependents -- maybe you should, but you have plenty of options. It all feels very simple. I'm glossing over a lot.
  - shepherdjerred 3y ago
    Sorry, I wish I could phrase it better for you. All I can say is that I have tried a _lot_ of tools, and nothing has come close. Amazon has done a lot of work to make efficient tools. Here's a better explanation: https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6283b0 https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6...

xyzzy_plugh 3y ago

No, not even close. You might even have it exactly backwards.

mdaniel 3y ago
which is why, as I originally asked GP: what have you already tried and what features were they missing I presume by "exactly backwards" you mean that one should have absolutely zero knobs to influence anything because the Almighty Jeff Build System does all the things, which GitLab also supports but is less amusing to look at on an Internet forum because it's "you can't modify anything, it just works, trust me" Or, you know, if you have something constructive to add to this discussion feel free to use more words than "lol, no"
- xyzzy_plugh 3y ago
  I don't work at Amazon, and haven't for a long time, and this format is insufficient to fully express what they're doing, so I won't try. You're better off searching for how Brazil and Apollo work. That being said, the short of it is that: imagine when you push a new revision to source control, you (you) can run jobs testing every potential consumer of that new revision. As in, you push libx-1.1.2 and anyone consuming libx >= 1.1 (or any variety of filters) is identified. If the tests succeed, you can update their dependencies on your package and even deploy them, safely and gradually, to production without involving the downstream teams at all. If they don't, you can choose your own adventure: pin them, fork, fix them, patch the package, revise the versioning, whatever you want. It's designed to be extremely safe and put power in the hands of those updating dependencies to do so safely within reason. Imagine you work on a library and you can test your PR against every consumers. It's not unlike what Google and other monorepos accomplish but it's quite different also. You can have many live versions simultaneously. You don't have to slog it out and patch all the dependents -- maybe you should, but you have plenty of options. It all feels very simple. I'm glossing over a lot.
- shepherdjerred 3y ago
  Sorry, I wish I could phrase it better for you. All I can say is that I have tried a _lot_ of tools, and nothing has come close. Amazon has done a lot of work to make efficient tools. Here's a better explanation: https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6283b0 https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6...