I've been working on Depot (https://depot.dev https://depot.dev) specifically for this reason: it's a hosted Docker builder service that runs BuildKit on manage

I've been working on Depot (https://depot.dev https://depot.dev) specifically for this reason: it's a hosted Docker builder service that runs BuildKit on managed VMs. When it receives incoming build requests, it routes them to a VM running the target architecture, x86 VMs run in Fly.io, arm64 VMs run in AWS. Since it's all BuildKit, you can swap `docker buildx build` for `depot build` and it works exactly the same - I made a depot/build-push-action to drop in place of the docker/build-push-action in GitHub Actions. It also has a persistent SSD cache disk for each builder, that was my other pain with GitHub Actions, time saving and loading layer cache was negating the speedups from cache hits - with a persistent disk, there's no saving or loading. Anyways, combo of having a local cache and running on real ARM machines gives like an order of magnitude speedup to builds compared to the QEMU emulation. Still a new project, not yet officially launched, and hosted services aren't for everyone, but exactly as you said, the status quo is amazingly painful.