7 ms·

Articles like this are pointless. I get that guix and nix are neat, and I think that every single time something about one of them is posted, but I don't have

Articles like this are pointless. I get that guix and nix are neat, and I think that every single time something about one of them is posted, but I don't have the slightest clue how to use either one of them. Do you want to convince people that something like guix is better than docker? Then take something that is currently distributed using docker and actually show how the guix approach is simpler. i.e. I have a random app I recently worked on where the dockerfile was something like FROM python:2.7 WORKDIR /app ADD requirements.txt /app RUN pip install -r requirements.txt ADD . /app RUN groupadd -r notifier && useradd --no-log-init -r -g notifier notifier USER notifier EXPOSE 8080/tcp CMD ./notify.py How do I actually take a random application like that and build a guix package of it? Another project I work on is built on top of zeromq, and it would be great to use something like guix to define all the libsodium+zeromq+czmq+zyre dependancies and be able to spit out an 'ultimate container image' of all of that, but all this post shows me how to do is install an existing guile package.

t0nt0n 8y ago
Packaged zyre and czmq. I'll send the patch for their inclusion, but until then here is the code: https://notabug.org/thomassgn/guixsd-configuration/src/master/modules/ton-tull.scm#L210 https://notabug.org/thomassgn/guixsd-configuration/src/maste...
t0nt0n 8y ago
With Guix you get full introspection of your entire package dependency graph, you can check and manipulate every aspect - and it is still simple and easy to work with. With GuixSD you get this same introspection and overview, but of your entire system. creating a container, vm or even a docker image is a simple '$ guix system <container|vm> config.scm' away. And your config.scm is as complex as you like it to. The simplest way would be to package the app for guix and you could just run '$ guix environment <name-of-package>' and you would be dropped into an environment with all your dependencies and whatever else the application requires in your path ready for hacking, get your sources and editor and start working. If you need a vm or similar though I'd translate your example above into a system config where: - packages include python-2.7 and whatever is in requirements.txt (this may mean you have to package a few things, but again this is usually super easy) - users and groups are added to the config, as they always are, no extra step necessary. - exposing ports and networking is available as options for qemu script guix produces to launch the vm. - CMD ./notify.py: create a "simple" service that can be autostarted by the system on boot. - filesystem access is also handled by arguments to the qemu script. As always though there are several paths to Rome, and these are just two of them. Zeromq and libsodium are already packaged on guix, czmq and zyre looks like they would be simple to package, guix is really quite simple to work with, which I think is the reason so many of the users and devs are running it as our daily drivers, even though it is strictly beta (0.14. I think is the last release). And pointless, come on - what does that even mean? Does it mean you don't value them? I was quite happy to read about a neat new thing I can use my favorite tool for.
- justinsaccount 8y ago
  > With Guix you get full introspection of your entire package dependency graph Yes, I know all that. It's neat. I would like to learn more about it. > The simplest way would be to package the app for guix I was asking how to package the app for guix, and your response is the simplest way would be to package the app for guix... > If you need a vm or soimilar though I'd translate your example above into a system config where: - packages include python-2.7 and whatever is in requirements.txt (this may mean you have to package a few things, but again this is usually super easy) - users and groups are added to the config, as they always are, no extra step necessary. - exposing ports and networking is available as options for qemu script guix produces to launch the vm. - CMD ./notify.py: create a "simple" service that can be autostarted by the system on boot. - filesystem access is also handled by arguments to the qemu script. Yes, I'm sure it is super easy. How do I do it? Do you know how to use the dockerfile I posted above? You run docker build -t myapp . docker run myapp that's super easy. 9 lines and 2 commands. You can now add docker expert to your resume. > Zeromq and libsodium are already packaged on guix, czmq and zyre looks like they would be simple to package, Well, I was working on a fork of things, so I would have needed to install my forks. > guix is really quite simple to work with I'm sure it is! > And pointless, come on - what does that even mean? Does it mean you don't value them? I was quite happy to read about a neat new thing I can use my favorite tool for. You are correct, I don't really value posts saying how cool and easy something is and how much better it is than other solutions, when they don't actually present a complete solution someone can actually use. I get that it is not other peoples job to teach me how to use something like guix, but do people not understand why things like Docker won?
  - t0nt0n 8y ago
    Right, your dockerfile contains a requirements.txt with unknown complexity and number of packages, your app is without a name and does not have any links to code. I'd be happy to provide some examples. Say you want your fork of libsodium: (define-public my-libsodium (package (inherit libsodium) ; now anything not defined in this package will be inherited from libsodium (source (origin (method url-fetch) (uri "url-to-your-sources") (sha256 (base32 "hash")))) ; Add whatever other fields your fork needs. )) Sure it's slightly more verbose. That's a bit of the cost of having something you can actually rely on, with that degree of hackability. If you actually want help to package these things ask on our mailinglist or IRC, we're happy to help with specifics. But you're basicly complaining that I didn't give you a concrete solution to a problem with several missing details that are important. Docker would not be able to instantiate your python project if it did not know the contents of your requirements.txt. The thing is docker is huge and bloated; is far from secure, and will probably stay that way for the foreseeable future; has a more or less complete lack of introspection; and is not strictly reproducible (sure, it gets quite far along the way, but it really is not). Guix on the other hand is rather lightweight, and you have a fair amount of control over how lightweight it should be; builds from source, and has a sort of hotpatching system for security fixes; has introspection and is quite close to bitreproducible. Sure, docker is _easy_, as long as it works. And I'd argue that because of its complexity and obscurity it is not practically free software.
davexunit 8y ago
It's hard to give you any specific recommendations with so little context, but I will try. For starters, I should point out that you can't really compare Guix directly to Docker. Guix is a package manager, Docker isn't. The article talks about 'guix pack', which makes it possible for Guix to interoperate with non-Guix systems, and one supported system is Docker. You can deploy software with just Guix, too, either on GuixSD or a foreign distro with Guix installed. Anyway, in your Dockerfile I see that your application uses Python and you do some package management and service management stuff that is mixed together. In Guix, these things are separated. So the first step would be to define a package for your software, and then you would deploy that package. For a real world example of a Python application, here is what the AWS CLI package looks like: (define-public awscli (package (name "awscli") (version "1.14.41") (source (origin (method url-fetch) (uri (pypi-uri name version)) (sha256 (base32 "0sispclx263lybbk19zp1n9yhg8xxx4jddypzgi24vpjaqnsbwlc")))) (build-system python-build-system) (propagated-inputs `(("python-colorama" ,python-colorama) ("python-botocore" ,python-botocore) ("python-s3transfer" ,python-s3transfer) ("python-docutils" ,python-docutils) ("python-pyyaml" ,python-pyyaml) ("python-rsa" ,python-rsa))) (arguments '(#:tests? #f)) (home-page "https://aws.amazon.com/cli/") (synopsis "Command line client for AWS") (description "AWS CLI provides a unified command line interface to the Amazon Web Services (AWS) API.") (license license:asl2.0))) The package recipe contains all the metadata, build instructions, and dependencies. Now that you have a package, it can be built with Guix and then deployed in a variety of ways. Judging from the Dockerfile, your software is some daemon that listens on port 8080, so: * You can install the software directly using 'guix package -i your-package-name' and run the notify.py program. Good for trying things out. * If you are deploying to the Guix system distribution, you could write a service definition so that you can manage the daemon via the init system. The service would take care of creating the notifier user and group, starting the service on boot, etc. * You could use 'guix pack --format=docker' to export an image suitable for running with 'docker load' * You could use a different 'guix pack' format (and maybe make it relocatable) for running on some other non-Guix system I should also add that I don't think the work is fully done yet on handling the entirety of Docker use-cases. It's a work in progress. I can think of a number of things that I want to add to Guix to make this workflow better that I haven't had a chance to hack on yet.
jancsika 8y ago
@justinsaccount: can you give t0nt0n a clue what the contents of requirements.txt are, plus anything else needed to create a complete port to guix? Then it would be great to see t0nt0n or someone else who knows guix do the port so we can fully compare these two approaches.
- justinsaccount 8y ago
  it's not really application specific, just stuff like requests==2.18.4 the actual packages generally aren't important. The cases were that would become interesting are where they require some C library dependencies first, like libpq-dev. In those cases something like guix/nix would be nice because it could be used to pull in the specific external dependencies as well.
Ruud-v-A 8y ago
> it would be great to use something like guix to define all the libsodium+zeromq+czmq+zyre dependancies and be able to spit out an 'ultimate container image' You define a package for your own project that depends on libsodium/zeromq/etc from GuixSD. Then you export your own package with 'guix pack'. For an example of what a package definition looks like, take a look in /gnu/packages in the GuixSD repository, for instance libsodium [1] or Vim [2]. I did something similar recently to build an Nginx "application bundle" [3]. It uses Nix (previously Guix, but Nix worked better for me in the end) to build a squashfs image. You can then run the binary on that filesystem with systemd-nspawn, or as a regular service by setting RootImage=. Some advantages over the Docker approach are that you can easily customise the build (e.g. changing the ./configure flags for Nginx without having to manually perform all other build steps), and bit by bit reproducibility (if you build the same commit six months from now, on a different machine, you will still get the same image out). [1]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/crypto.scm?id=cbf58afa05b6014f59b49ec043aff6239fbe7313#n67 https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages... [2]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/vim.scm?id=cbf58afa05b6014f59b49ec043aff6239fbe7313#n61 https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages... [3]: https://github.com/ruuda/miniserver#readme https://github.com/ruuda/miniserver#readme
rekado 8y ago
> Do you want to convince people that something like guix is better than docker No, we show that Guix is a tool that gives you a way to work with software environments at a higher level; but at the same time you don't have to give up on application bundles like Docker. You can simply generate Docker images or other forms of applications bundles from that higher-level representation. You are welcome to take a look at this paper that I co-authored where we explain why we use Guix for a reproducible bioinformatics pipeline, and the rigorous, declarative functional package management approach instead of the imperative approach of Docker files: https://www.biorxiv.org/content/early/2018/04/21/298653 We're also providing Docker images, but we generate them from a higher-level declarative specification that ensures a high degree of bit-reproducibility.
myWindoonn 8y ago
From memory, not tested, not spell-checked: FROM nixos/nix RUN nix-channel --update RUN nix-env -i python2.7-{twisted,treq,txgithub} WORKDIR /app ADD . /app EXPOSE 8080/tcp CMD python notify.py The next level would be using the nixpkgs Docker builder directly: https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools

t0nt0n 8y ago

Packaged zyre and czmq. I'll send the patch for their inclusion, but until then here is the code: https://notabug.org/thomassgn/guixsd-configuration/src/master/modules/ton-tull.scm#L210 https://notabug.org/thomassgn/guixsd-configuration/src/maste...

t0nt0n 8y ago

With Guix you get full introspection of your entire package dependency graph, you can check and manipulate every aspect - and it is still simple and easy to work with. With GuixSD you get this same introspection and overview, but of your entire system. creating a container, vm or even a docker image is a simple '$ guix system <container|vm> config.scm' away. And your config.scm is as complex as you like it to. The simplest way would be to package the app for guix and you could just run '$ guix environment <name-of-package>' and you would be dropped into an environment with all your dependencies and whatever else the application requires in your path ready for hacking, get your sources and editor and start working. If you need a vm or similar though I'd translate your example above into a system config where: - packages include python-2.7 and whatever is in requirements.txt (this may mean you have to package a few things, but again this is usually super easy) - users and groups are added to the config, as they always are, no extra step necessary. - exposing ports and networking is available as options for qemu script guix produces to launch the vm. - CMD ./notify.py: create a "simple" service that can be autostarted by the system on boot. - filesystem access is also handled by arguments to the qemu script. As always though there are several paths to Rome, and these are just two of them. Zeromq and libsodium are already packaged on guix, czmq and zyre looks like they would be simple to package, guix is really quite simple to work with, which I think is the reason so many of the users and devs are running it as our daily drivers, even though it is strictly beta (0.14. I think is the last release). And pointless, come on - what does that even mean? Does it mean you don't value them? I was quite happy to read about a neat new thing I can use my favorite tool for.

justinsaccount 8y ago
> With Guix you get full introspection of your entire package dependency graph Yes, I know all that. It's neat. I would like to learn more about it. > The simplest way would be to package the app for guix I was asking how to package the app for guix, and your response is the simplest way would be to package the app for guix... > If you need a vm or soimilar though I'd translate your example above into a system config where: - packages include python-2.7 and whatever is in requirements.txt (this may mean you have to package a few things, but again this is usually super easy) - users and groups are added to the config, as they always are, no extra step necessary. - exposing ports and networking is available as options for qemu script guix produces to launch the vm. - CMD ./notify.py: create a "simple" service that can be autostarted by the system on boot. - filesystem access is also handled by arguments to the qemu script. Yes, I'm sure it is super easy. How do I do it? Do you know how to use the dockerfile I posted above? You run docker build -t myapp . docker run myapp that's super easy. 9 lines and 2 commands. You can now add docker expert to your resume. > Zeromq and libsodium are already packaged on guix, czmq and zyre looks like they would be simple to package, Well, I was working on a fork of things, so I would have needed to install my forks. > guix is really quite simple to work with I'm sure it is! > And pointless, come on - what does that even mean? Does it mean you don't value them? I was quite happy to read about a neat new thing I can use my favorite tool for. You are correct, I don't really value posts saying how cool and easy something is and how much better it is than other solutions, when they don't actually present a complete solution someone can actually use. I get that it is not other peoples job to teach me how to use something like guix, but do people not understand why things like Docker won?
- t0nt0n 8y ago
  Right, your dockerfile contains a requirements.txt with unknown complexity and number of packages, your app is without a name and does not have any links to code. I'd be happy to provide some examples. Say you want your fork of libsodium: (define-public my-libsodium (package (inherit libsodium) ; now anything not defined in this package will be inherited from libsodium (source (origin (method url-fetch) (uri "url-to-your-sources") (sha256 (base32 "hash")))) ; Add whatever other fields your fork needs. )) Sure it's slightly more verbose. That's a bit of the cost of having something you can actually rely on, with that degree of hackability. If you actually want help to package these things ask on our mailinglist or IRC, we're happy to help with specifics. But you're basicly complaining that I didn't give you a concrete solution to a problem with several missing details that are important. Docker would not be able to instantiate your python project if it did not know the contents of your requirements.txt. The thing is docker is huge and bloated; is far from secure, and will probably stay that way for the foreseeable future; has a more or less complete lack of introspection; and is not strictly reproducible (sure, it gets quite far along the way, but it really is not). Guix on the other hand is rather lightweight, and you have a fair amount of control over how lightweight it should be; builds from source, and has a sort of hotpatching system for security fixes; has introspection and is quite close to bitreproducible. Sure, docker is _easy_, as long as it works. And I'd argue that because of its complexity and obscurity it is not practically free software.

davexunit 8y ago

It's hard to give you any specific recommendations with so little context, but I will try. For starters, I should point out that you can't really compare Guix directly to Docker. Guix is a package manager, Docker isn't. The article talks about 'guix pack', which makes it possible for Guix to interoperate with non-Guix systems, and one supported system is Docker. You can deploy software with just Guix, too, either on GuixSD or a foreign distro with Guix installed. Anyway, in your Dockerfile I see that your application uses Python and you do some package management and service management stuff that is mixed together. In Guix, these things are separated. So the first step would be to define a package for your software, and then you would deploy that package. For a real world example of a Python application, here is what the AWS CLI package looks like: (define-public awscli (package (name "awscli") (version "1.14.41") (source (origin (method url-fetch) (uri (pypi-uri name version)) (sha256 (base32 "0sispclx263lybbk19zp1n9yhg8xxx4jddypzgi24vpjaqnsbwlc")))) (build-system python-build-system) (propagated-inputs `(("python-colorama" ,python-colorama) ("python-botocore" ,python-botocore) ("python-s3transfer" ,python-s3transfer) ("python-docutils" ,python-docutils) ("python-pyyaml" ,python-pyyaml) ("python-rsa" ,python-rsa))) (arguments '(#:tests? #f)) (home-page "https://aws.amazon.com/cli/") (synopsis "Command line client for AWS") (description "AWS CLI provides a unified command line interface to the Amazon Web Services (AWS) API.") (license license:asl2.0))) The package recipe contains all the metadata, build instructions, and dependencies. Now that you have a package, it can be built with Guix and then deployed in a variety of ways. Judging from the Dockerfile, your software is some daemon that listens on port 8080, so: * You can install the software directly using 'guix package -i your-package-name' and run the notify.py program. Good for trying things out. * If you are deploying to the Guix system distribution, you could write a service definition so that you can manage the daemon via the init system. The service would take care of creating the notifier user and group, starting the service on boot, etc. * You could use 'guix pack --format=docker' to export an image suitable for running with 'docker load' * You could use a different 'guix pack' format (and maybe make it relocatable) for running on some other non-Guix system I should also add that I don't think the work is fully done yet on handling the entirety of Docker use-cases. It's a work in progress. I can think of a number of things that I want to add to Guix to make this workflow better that I haven't had a chance to hack on yet.

jancsika 8y ago

@justinsaccount: can you give t0nt0n a clue what the contents of requirements.txt are, plus anything else needed to create a complete port to guix? Then it would be great to see t0nt0n or someone else who knows guix do the port so we can fully compare these two approaches.

justinsaccount 8y ago
it's not really application specific, just stuff like requests==2.18.4 the actual packages generally aren't important. The cases were that would become interesting are where they require some C library dependencies first, like libpq-dev. In those cases something like guix/nix would be nice because it could be used to pull in the specific external dependencies as well.

Ruud-v-A 8y ago

> it would be great to use something like guix to define all the libsodium+zeromq+czmq+zyre dependancies and be able to spit out an 'ultimate container image' You define a package for your own project that depends on libsodium/zeromq/etc from GuixSD. Then you export your own package with 'guix pack'. For an example of what a package definition looks like, take a look in /gnu/packages in the GuixSD repository, for instance libsodium [1] or Vim [2]. I did something similar recently to build an Nginx "application bundle" [3]. It uses Nix (previously Guix, but Nix worked better for me in the end) to build a squashfs image. You can then run the binary on that filesystem with systemd-nspawn, or as a regular service by setting RootImage=. Some advantages over the Docker approach are that you can easily customise the build (e.g. changing the ./configure flags for Nginx without having to manually perform all other build steps), and bit by bit reproducibility (if you build the same commit six months from now, on a different machine, you will still get the same image out). [1]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/crypto.scm?id=cbf58afa05b6014f59b49ec043aff6239fbe7313#n67 https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages... [2]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/vim.scm?id=cbf58afa05b6014f59b49ec043aff6239fbe7313#n61 https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages... [3]: https://github.com/ruuda/miniserver#readme https://github.com/ruuda/miniserver#readme

rekado 8y ago

> Do you want to convince people that something like guix is better than docker No, we show that Guix is a tool that gives you a way to work with software environments at a higher level; but at the same time you don't have to give up on application bundles like Docker. You can simply generate Docker images or other forms of applications bundles from that higher-level representation. You are welcome to take a look at this paper that I co-authored where we explain why we use Guix for a reproducible bioinformatics pipeline, and the rigorous, declarative functional package management approach instead of the imperative approach of Docker files: https://www.biorxiv.org/content/early/2018/04/21/298653 We're also providing Docker images, but we generate them from a higher-level declarative specification that ensures a high degree of bit-reproducibility.

myWindoonn 8y ago

From memory, not tested, not spell-checked: FROM nixos/nix RUN nix-channel --update RUN nix-env -i python2.7-{twisted,treq,txgithub} WORKDIR /app ADD . /app EXPOSE 8080/tcp CMD python notify.py The next level would be using the nixpkgs Docker builder directly: https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools https://nixos.org/nixpkgs/manual/#sec-pkgs-dockerTools