A big problem with making an abstraction over CI providers is that they are, on the surface, very little more than wrappers around steps made up of small script

A big problem with making an abstraction over CI providers is that they are, on the surface, very little more than wrappers around steps made up of small scripts, but that abstraction leaks like hell. So I think you would wind up needing to go quite high-level to make something that is useful here, but even if you do do this, the variance in runtime environment will make it very difficult to make sure things are relatively portable, and attempts to work around this will lead to problems (e.g. you could just wrap everything in Docker, but fetching large images on each CI invocation isn't really a good use of resources.) Let's say you have a project using CMake with vcpkg. Vcpkg has support for the GitHub Actions Cache API, so it can integrate very well into GHA and provide good performance for caching dependencies so they do not need to be built on each invocation. Doesn't seem too unreasonable, you could have a vcpkg+CMake plugin of some kind that tries to do the best thing for each CI provider. But, what version of vcpkg do you use? what version of CMake do you use? Vcpkg is essentially just a rolling-release Git repo, there's no specific version to use. But it would save a lot of time to make use of the vcpkg installation that already exists on GitHub Actions images. And obviously, a tool like CMake depends super heavily on the environment around it. A lot of people use the Ninja provider for better performance... but if you use it on Windows, you need to be careful: CMake with the Ninja provider on Windows can not automatically pick up the Visual Studio installation, unlike the default Windows provider (which is, well, Visual Studio.) So instead it might pick up MinGW, which is often unexpected, and does not always work correctly. Does Microsoft have an answer for how to deal with this? Not really. You might think to use the official setup-msbuild step, which has been mistakenly recommended by Microsoft, but it only sets up enough environment to do .NET development. So actually your CI abstraction needs plugins that deal with: - Every CI provider - Using every operating system - For each build step And, also, it needs to be updated fairly frequently, especially if you want to keep up to date on e.g. macOS versions. Personally, I think this is an unwinnable situation, though I'd love to be proven wrong. I think it would take something fairly dramatic to actually solve this problem most of the way, though. Something that might need a lot of collaboration between stakeholders, possibly the CI providers themselves, and probably even build/packaging tools at some level. One thing I thought was interesting was the approach of using Nix[1]. I've tried rudimentary implementations of Nix with GitHub Actions though, and it does definitely have some issues still. The most obvious issue is that it doesn't really support Windows, followed by the fact that binaries made using Nix are generally not going to be very easy to distribute (unless you're building an OCI or AppImage or something like that.) But for just running tests, it's enticing at least, and it can do a whole lot of caching in theory (for dependencies; you'll still need to handle caching of intermediate files for local tests/builds in your build system underneath Nix.) The more I think about this problem, the more I wonder which angle is the best one to come at it from. Is this ultimately a problem about software SDKs in disguise? [1]: https://determinate.systems/posts/nix-github-actions https://determinate.systems/posts/nix-github-actions