3 ms·

Congrats on the launch! Sorry to ask this silly question, but since your team is an expert in the "fast Docker images" area, could somebody avoid the tradition

Congrats on the launch! Sorry to ask this silly question, but since your team is an expert in the "fast Docker images" area, could somebody avoid the traditional `docker build` with say NixOS or Bazel and achieve the same results as Depot (aka, fast building with the output being an OCI/Docker image)? Is that what Depot is doing at a high level? Was this considered? > Our CLI is wrapping the Buildx library I'm surprised you're able to build Docker images faster than Docker using their code/libraries?

jlhawn 4y ago
> I'm surprised you're able to build Docker images faster than Docker using their code/libraries? It's not a code/library problem. Knowing what Buildkit options to use is the easy part. It's almost entirely a storage infrastructure and networking problem as it has huge implications on whether or not you'll be able to easily cache build layers.
- MuffinFlavored 4y ago
  > It's almost entirely a storage infrastructure and networking problem as it has huge implications on whether or not you'll be able to easily cache build layers. On a single machine, would NixOS/Bazel handle this better than Dockerfile/Docker/BuildKit?
  - jacobwg 4y ago
    Potentially - if Nix or Bazel has already built the binaries, and just needs to construct an OCI-compliant image tarball with them, that can be quite quick, similar to a Dockerfile with only COPY instructions. Nix and Bazel can also give you deterministic / reproducible images that take more effort to construct with Dockerfiles. I've also seen people use Nix or Bazel inside their Dockerfile, like ultimately the build has to execute somewhere, be that inside or outside a Dockerfile.
    - lewo 4y ago
      FYI, the nix2container [1] project (author here) aims to speedup the standard Nix container workflow (dockerTools.buildImage) by basically skipping the tarball step: it directly streams non already pushed layers. [1] https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container
jacobwg 4y ago
> could somebody avoid the traditional `docker build` with say NixOS or Bazel Yes! You can think of an OCI image as a special kind of tarball, so things like NixOS and Bazel are able to construct that same tarball, potentially fairly quick if it just has to copy prebuilt artifacts from the store. Today we're running BuildKit, so we support all the typical Docker things as well as other systems that use BuildKit, e.g. Dagger, and I believe there are nix frontends for BuildKit. In that sense, we can be an accelerated compute provider for anything compatible with BuildKit. > build Docker images faster than Docker Today the trick is in the hosting and orchestration. We're using fast machines, launching Graviton instances for Arm builds (no emulation) or multiple machines for multi-platform build requests, orchestrating persistent volumes, etc. It's more advanced than what hosted CI providers give you today, and closer to something you'd need to script together yourself with your own runners. There's also some Docker build features (e.g. cache mounts) that _only_ work with a persistent disk.

jlhawn 4y ago

> I'm surprised you're able to build Docker images faster than Docker using their code/libraries? It's not a code/library problem. Knowing what Buildkit options to use is the easy part. It's almost entirely a storage infrastructure and networking problem as it has huge implications on whether or not you'll be able to easily cache build layers.

MuffinFlavored 4y ago
> It's almost entirely a storage infrastructure and networking problem as it has huge implications on whether or not you'll be able to easily cache build layers. On a single machine, would NixOS/Bazel handle this better than Dockerfile/Docker/BuildKit?
- jacobwg 4y ago
  Potentially - if Nix or Bazel has already built the binaries, and just needs to construct an OCI-compliant image tarball with them, that can be quite quick, similar to a Dockerfile with only COPY instructions. Nix and Bazel can also give you deterministic / reproducible images that take more effort to construct with Dockerfiles. I've also seen people use Nix or Bazel inside their Dockerfile, like ultimately the build has to execute somewhere, be that inside or outside a Dockerfile.
  - lewo 4y ago
    FYI, the nix2container [1] project (author here) aims to speedup the standard Nix container workflow (dockerTools.buildImage) by basically skipping the tarball step: it directly streams non already pushed layers. [1] https://github.com/nlewo/nix2container https://github.com/nlewo/nix2container

jacobwg 4y ago

> could somebody avoid the traditional `docker build` with say NixOS or Bazel Yes! You can think of an OCI image as a special kind of tarball, so things like NixOS and Bazel are able to construct that same tarball, potentially fairly quick if it just has to copy prebuilt artifacts from the store. Today we're running BuildKit, so we support all the typical Docker things as well as other systems that use BuildKit, e.g. Dagger, and I believe there are nix frontends for BuildKit. In that sense, we can be an accelerated compute provider for anything compatible with BuildKit. > build Docker images faster than Docker Today the trick is in the hosting and orchestration. We're using fast machines, launching Graviton instances for Arm builds (no emulation) or multiple machines for multi-platform build requests, orchestrating persistent volumes, etc. It's more advanced than what hosted CI providers give you today, and closer to something you'd need to script together yourself with your own runners. There's also some Docker build features (e.g. cache mounts) that _only_ work with a persistent disk.