11 ms·

Modus: A language for building Docker/OCI container images

56 points by mrigger 4y ago

mrigger 4y ago
More information at https://modus-continens.com/ https://modus-continens.com/.
stoplying1 4y ago
It looks like nearly the same primitives with the same reproducability problems of every non-Bazel, non-nix solution. Why would one learn this instead of just using Nix to create images and getting more actual tangible benefits?
- kevinmgranger 4y ago
  I'm not familiar enough to speak to Bazel or Nix, but do they handle all of the advantages listed on the modus site? https://modus-continens.com/#parameterised-builds https://modus-continens.com/#parameterised-builds
  - coryrc 4y ago
    Yes, and then some.
    - JustLurking2022 4y ago
      Except they have major issues at scale.
      - stoplying1 4y ago
        No, they really don't. I manage a fleet of hundreds of servers with NixOS. Surprisingly huge companies are using Nix, feel free to look around. I can build for 4 different architectures, with identical system configurations, from a single readable codebase. I've actually never even heard this particular complaint. What exactly have you heard about problems with nix "at scale", or are you just exaggerating what a "learning curve" is again?
- darthrupert 4y ago
  Because Nix is a convoluted mess that is nearly impossible to learn, perhaps?
  - rgoulter 4y ago
    That somewhat overstates it, but sure. I think the point is, what trade-offs are you paying such that you'd be better off using Modus, but not a Dockerfile or a Nix expression. -- If you want to benefit from nicer programmatic building of Dockerfiles, and can afford the risk of adopting some non-mainstream technology, you're probably better off trying Nix than this.
- dboreham 4y ago
  Bazel and Nix stand out as the two things that recently a) I've heard about then initially investigated, concluding that they look promising then b) people who I trust who have used them in depth tell me to avoid at all costs.
  - rgoulter 4y ago
    Nix is wonderful to use for personal workstations & such. I'd suggest if the idea of purely declaring a package as a function of its inputs sounds interesting to you, it's well worth taking time to look into using Nix. I think in terms of adopting in a team environment.. I don't think it's a good idea for a company to adopt high-cost-high-reward tools at a place without at least 3 developers who know the technology well. (You don't want just 1 dev knowing it; and you want slack in case 1 quits or whatever).
- OJFord 4y ago
  I hadn't realised you could use Nix for building images, or at least not standalone. An article: https://thewagner.net/blog/2021/02/25/building-container-images-with-nix/ https://thewagner.net/blog/2021/02/25/building-container-ima...
  - yewenjie 4y ago
    There is also https://nixery.dev/ https://nixery.dev/ which lets you get Docker images with arbitrary Nix packages - useful for quick hacking.
    - rmorey 4y ago
      woah - this is great, thanks
    - MuffinFlavored 4y ago
      Kind of confusing. NixOS vs Nix the language... You use Nix the language to build Docker images and avoid the NixOS CLI tools/runtime?
      - k__ 4y ago
        Nix the package manager. Nix the language. NixOS the Linux built with Nix as it's default package manager that uses the Nix language.
        MuffinFlavored 4y ago
        Nix the language can compile OCI (Open Container Initiative) format images that can be deployed/ran/executed via Docker or k8s? Or is that Nix the package manager? Does Nix the package manager compile Nix the language images?
        piperswe 4y ago
        [nixpkgs the package collection] (which is used by [Nix the package manager], and also contains the [Nix the programming language] expressions that define [NixOS the operating system]) has expressions for easily building OCI images: https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-dockerTools https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-dockerTool...
- mechtaev 4y ago
  Disclaimer: I am a co-author of Modus. Modus is not really an alternative to Nix as a package manager, but to Nix as a language. It has several advantages, for example, it is non-Turing-complete, so there can not be infinite loops, and its definitions are more concise thanks to its logic programming foundations. Saying that, I agree that an integration with package managers can be beneficial for reproducibility. This is the area of our current research.
- JustLurking2022 4y ago
  Hey, look it's the Nix zealots again. Sure, let's all jump onboard that tool that even it's advocates admit it's quirky and has a huge learning curve.
  - stoplying1 4y ago
    I'm a zealot for pointing out that this tool distinctly doesn't solve what is often cited as a major footgun and misunderstanding of how docker containers are almost always built? I'll take a learning curve with actual reproducability any day anytime. Add this to the "git is hard" list of complaints. Bending over 8hrs a day and line cooking is harder. Do you know how many stupid "you're using docker wrong if you don't do this quirky trick!!" articles I've read or written myself over the last 5 years? And still virtually no one bothers to follow all of them because they're virtually impossible to achieve with conventional package managers? Just the zealots noticing this?
awinter-py 4y ago
eternal wishlist item for container builds: package manager integration. That means per-file caching for library downloads, out of order installation for when you change one thing in your dependency file. (you can kind of make the caching happen with buildkit but it's annoying to set up) I'm not sure any layer DAG tool will solve the packaging issue because dags are inherently ordered right? cloud providers and users are probably aligned here in terms of goals: faster builds, smaller incremental deploys
- verdverm 4y ago
  You might see real solutions with BuildKit's new FS merge feature. This might be possible today with https://dagger.io https://dagger.io which exposes that feature
  - awinter-py 4y ago
    thank you. will try them out
- arccy 4y ago
  give up the dag and be more declarative https://github.com/chainguard-dev/apko https://github.com/chainguard-dev/apko
  - awinter-py 4y ago
    love this, but sounds from chainguard dev's tweet[1] like these images are single-layer? will that require a full rebuild + full re-transfer on every deploy? 1. https://twitter.com/puerco/status/1542018385550123008 https://twitter.com/puerco/status/1542018385550123008

56 points by mrigger 4y ago

More information at https://modus-continens.com/ https://modus-continens.com/.

It looks like nearly the same primitives with the same reproducability problems of every non-Bazel, non-nix solution. Why would one learn this instead of just using Nix to create images and getting more actual tangible benefits?

kevinmgranger 4y ago
I'm not familiar enough to speak to Bazel or Nix, but do they handle all of the advantages listed on the modus site? https://modus-continens.com/#parameterised-builds https://modus-continens.com/#parameterised-builds
- coryrc 4y ago
  Yes, and then some.
  - JustLurking2022 4y ago
    Except they have major issues at scale.
    - stoplying1 4y ago
      No, they really don't. I manage a fleet of hundreds of servers with NixOS. Surprisingly huge companies are using Nix, feel free to look around. I can build for 4 different architectures, with identical system configurations, from a single readable codebase. I've actually never even heard this particular complaint. What exactly have you heard about problems with nix "at scale", or are you just exaggerating what a "learning curve" is again?
darthrupert 4y ago
Because Nix is a convoluted mess that is nearly impossible to learn, perhaps?
- rgoulter 4y ago
  That somewhat overstates it, but sure. I think the point is, what trade-offs are you paying such that you'd be better off using Modus, but not a Dockerfile or a Nix expression. -- If you want to benefit from nicer programmatic building of Dockerfiles, and can afford the risk of adopting some non-mainstream technology, you're probably better off trying Nix than this.
dboreham 4y ago
Bazel and Nix stand out as the two things that recently a) I've heard about then initially investigated, concluding that they look promising then b) people who I trust who have used them in depth tell me to avoid at all costs.
- rgoulter 4y ago
  Nix is wonderful to use for personal workstations & such. I'd suggest if the idea of purely declaring a package as a function of its inputs sounds interesting to you, it's well worth taking time to look into using Nix. I think in terms of adopting in a team environment.. I don't think it's a good idea for a company to adopt high-cost-high-reward tools at a place without at least 3 developers who know the technology well. (You don't want just 1 dev knowing it; and you want slack in case 1 quits or whatever).
OJFord 4y ago
I hadn't realised you could use Nix for building images, or at least not standalone. An article: https://thewagner.net/blog/2021/02/25/building-container-images-with-nix/ https://thewagner.net/blog/2021/02/25/building-container-ima...
- yewenjie 4y ago
  There is also https://nixery.dev/ https://nixery.dev/ which lets you get Docker images with arbitrary Nix packages - useful for quick hacking.
  - rmorey 4y ago
    woah - this is great, thanks
  - MuffinFlavored 4y ago
    Kind of confusing. NixOS vs Nix the language... You use Nix the language to build Docker images and avoid the NixOS CLI tools/runtime?
    - k__ 4y ago
      Nix the package manager. Nix the language. NixOS the Linux built with Nix as it's default package manager that uses the Nix language.
      - MuffinFlavored 4y ago
        Nix the language can compile OCI (Open Container Initiative) format images that can be deployed/ran/executed via Docker or k8s? Or is that Nix the package manager? Does Nix the package manager compile Nix the language images?
        piperswe 4y ago
        [nixpkgs the package collection] (which is used by [Nix the package manager], and also contains the [Nix the programming language] expressions that define [NixOS the operating system]) has expressions for easily building OCI images: https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-dockerTools https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-dockerTool...
mechtaev 4y ago
Disclaimer: I am a co-author of Modus. Modus is not really an alternative to Nix as a package manager, but to Nix as a language. It has several advantages, for example, it is non-Turing-complete, so there can not be infinite loops, and its definitions are more concise thanks to its logic programming foundations. Saying that, I agree that an integration with package managers can be beneficial for reproducibility. This is the area of our current research.
JustLurking2022 4y ago
Hey, look it's the Nix zealots again. Sure, let's all jump onboard that tool that even it's advocates admit it's quirky and has a huge learning curve.
- stoplying1 4y ago
  I'm a zealot for pointing out that this tool distinctly doesn't solve what is often cited as a major footgun and misunderstanding of how docker containers are almost always built? I'll take a learning curve with actual reproducability any day anytime. Add this to the "git is hard" list of complaints. Bending over 8hrs a day and line cooking is harder. Do you know how many stupid "you're using docker wrong if you don't do this quirky trick!!" articles I've read or written myself over the last 5 years? And still virtually no one bothers to follow all of them because they're virtually impossible to achieve with conventional package managers? Just the zealots noticing this?

awinter-py 4y ago

eternal wishlist item for container builds: package manager integration. That means per-file caching for library downloads, out of order installation for when you change one thing in your dependency file. (you can kind of make the caching happen with buildkit but it's annoying to set up) I'm not sure any layer DAG tool will solve the packaging issue because dags are inherently ordered right? cloud providers and users are probably aligned here in terms of goals: faster builds, smaller incremental deploys

verdverm 4y ago
You might see real solutions with BuildKit's new FS merge feature. This might be possible today with https://dagger.io https://dagger.io which exposes that feature
- awinter-py 4y ago
  thank you. will try them out
arccy 4y ago
give up the dag and be more declarative https://github.com/chainguard-dev/apko https://github.com/chainguard-dev/apko
- awinter-py 4y ago
  love this, but sounds from chainguard dev's tweet[1] like these images are single-layer? will that require a full rebuild + full re-transfer on every deploy? 1. https://twitter.com/puerco/status/1542018385550123008 https://twitter.com/puerco/status/1542018385550123008