8 ms·

Developing V8 with Guix

73 points by apaprocki 11y ago

knodi123 11y ago
What's the significance of "V8"? He means the javascript engine, right? But that never enters into the story.... It's really just "Developing with Guix".
- swah 11y ago
  wingo likes/studies v8? https://wingolog.org/tags/v8 https://wingolog.org/tags/v8
  - amirouche 11y ago
    he is knowledgeable in the VM implementation, and do develops features in v8 and GNU Guile VM.
- davexunit 11y ago
  Towards the end of the post he shows how he creates his reproducible v8 dev environment.
  - knodi123 11y ago
    My bad, I didn't make it that far. Dude spent the first half of the article bitching about his Debian box, and I skimmed the rest (apparently not very well).
    - amirouche 11y ago
      He explains how nix/guix technology in the realm of personnal computing can improve day-to-day work, in this case the work of a developper. Wingo's setup is kind of complicated: nixos+guix. (I use only Guix OS for Django dev). In my case, Guix is overkill most of the time, since most dependencies are python. Guix can also replace virtualenv taking into account binary dependencies.. Binary dependency is the case met by Wingo. I think he does not stress enough (first part) the fact that upgrading is painless in nixos and Guix OS (almost, it's still alpha) compared to debian-like distros. Guix is a good idea. It needs more packages that is all.
    - snorrah 11y ago
      The "bitching" about his Debian install was what I found most interesting. My view of Debian is of a slow-to-change, highly stable distribution. Now, bearing in mind he had switched to the unstable branch but still found it reliable over a lengthy period of time, it was somewhat disturbing to see he could end up in a pretty bad spot after a reboot.
vezzy-fnord 11y ago
They use "deco" as an init system, for example, which is kinda like systemd but not. The init system is actually GNU dmd. deco is the name of the control utility (like systemctl is to systemd). It also has absolutely nothing in common with systemd, so I don't know why the author made that comparison.
- davexunit 11y ago
  It's hard to talk about an alternative init system without comparing it to systemd these days. I wish wingo would have mentioned why we use dmd in the first place. It's not out of systemd hatred, but rather for better system integration. dmd is written in Guile Scheme, just like Guix, and that gives us the two programs really excellent integration with each other. dmd needs more love to approach the feature set of systemd, but I think using it is the right choice for the long term. In addition to using it on my GuixSD systems, I also use it as a user service manager at work and it works great. Also, there's a proposed project out there for someone to implement a Guile reader that reads in systemd unit files and produces the equivalent dmd service object. If this is done, it should assist adoption, or at the very least experimentation.
davexunit 11y ago
I wonder if wingo knows about the 'guix environment' tool or not. It can create dev environments with the correct environment variables configured on the fly, without creating a new profile or polluting an existing one. Profiles are still useful for persistence, but I use 'guix environment' for all of my day-to-day hacking so I don't have to periodically delete generations of a profile in order for the GC to collect them.
sandGorgon 11y ago
IMHO, there is only a little bit of separation between Docker and Nix/Guix. I wonder if the Linux desktops of tomorrow will be a thin hypervisor (or something like CoreOS), running a immutable VM on top.
- davexunit 11y ago
  There's quite a lot of separation, IMO. Docker uses an imperative, stateful system for image creation that doesn't actually help with reproducibility and specifically only creates container environments. Guix and Nix use a functional, declarative system for building anything (software builds, VM image creation, etc.) and can create any type of environment, including containers. Guix and Nix are much more general-purpose and are built upon a stronger foundation. So, rather than using Docker running Debian/Ubuntu/whatever images and another OS entirely managing the host, you can use Guix or Nix to manage every layer and take advantage of their very advanced features everywhere.
  - mands 11y ago
    I'm a big fan of Guix/Nix and would love to see them in greater use. Whilst I agree that they provide many of the reproducibly benefits of containers, isn't the other big advantage of containers the (theoretical) security benefits they bring? i.e. a process can't break out of its container and access other data. Perhaps I'm missing something but can such isolation be provided by Guix/Nix?
    - davexunit 11y ago
      Yes it can. The build daemon already uses containers to create the isolated environments to perform builds in so that we can view builds as pure functions. Nix uses systemd-nspawn for making NixOS containers. Guix uses a Scheme interface that I wrote called call-with-container to build GuixSD containers or simple containers without an init system via 'guix environment'. call-with-container is a work in progress, but some of the code has already made it to master, and the rest of it should land in master in time for the next release.
mrottenkolber 11y ago
Just an anecdote: Since I started using Slackware Linux, I have never had any problem with my system. I like the idea of functional package managers, Guix especially. But for me personally the problem they are trying to solve does not exist.
- vezzy-fnord 11y ago
  I too am a diehard Slackware user (I would say "Slack" like J.R. Dobbs intended it, but people would confuse it for the chat) and I'll echo your sentiment. Guix and Nix are the only ones in the field which deeply identify and solve issues in package management, and are the solutions I'd be most open to using. For personal usage, the Slackware tarball format without dependency resolution has never failed me. (I also have an affinity for Portage, but it's a delicate thing to use.)
  - ics 11y ago
    Bob be with you.
- 1_player 11y ago
  Functional package managers (FPM) to me are a viable solution to (cloud) deployment: what is the difference between a FPM controlling the totality of inputs and outputs of an application, and modern containerization solutions such as Docker, really? I've played with Docker (and rkt) and there's something off putting about using this stuff in production, to me at least, while I'm itching to see how Nix and Guix grow; waiting for someone with more free time than me to really cultivate this potential. Nix is more mature, but I dislike the Haskell syntax. I love Scheme, but I hate the free-software-only-religion around GNU Guix: I appreciate what GNU does, I'm a regular GNU/Linux user, but at this stage of development religious beliefs are just a hindrance to adoption and it makes me sad. EDIT: and let's not forget about node provisioning, something you can already do with NixOS (no idea about GuixSD): forget Ansible and its declarative configuration, but define the totality of the system state and configuration in a single file. Tell the machine "OK configure yourself like this", and voilà, you have a node which is (formally?) proven to be exactly like you've described. That's the future.
amelius 11y ago
> but it is just unacceptable and revolting that software development in 2015 is exposed to an upgrade process which (1) can break your system (2) by default and (3) can't be rolled back. I would like to add that building a system that makes it difficult to reproduce the exact same state on another machine, even if the central/distributed repository it originated from is not available, should be considered malpractice.
zzalpha 11y ago
Huh... I had no idea Nix and Guix could be used this way. I viewed them as replacements to apt, but what I'm hearing is I could just as easily use it as a souped up stow... interesting...
KirinDave 11y ago
I've tried using Nix a few times. The idea is in principle not bad, but the actual deployment for, say... an API endpoint? Horrendous. Problems include: 1. Scripts deployed almost never appeal to /usr/bin/env, they tend to hardwire interpreters to /bin/sh. This flat out doesn't work in NixOS, and the Nix team is ideologically opposed to maintaining a compat layer for specific cases like this. 2. Compiled binaries are compiled to this SPECIFIC version of Nix. That may not seem like a bad thing, but the reality is that it's very abusive to work with. Your binaries cannot even be invoked becuase they cannot find their links (including, you know, the C bootstrapping lib). This means you can make an executable on your dev box, try and push it to prod, and because prod doesn't have something like a dtrace library the entire system is incompatible with your executable. 3. There are tools to assist with these problems, because they're so common. However, they themselves are full of edge cases they don't solve for that they're very hit-or-miss. While this isn't bad in and of itself, it tends to make the established userbase very unsympathetic and often very formulaic when you approach them with more subtle linking issues. This can be quite frustrating. I found the entire system somewhat unworkable. I also feel like Docker or Rocket solves a lot of the practical problems I have there, despite protestations. Container images are perfectly reproducible, but also easy to update. They also tend not to rip themselves apart if you, in a moment of intellectual weakness, give a slightly confusing order to the nixos package manager and rip your system in half.
- baq 11y ago
  the moment your Dockerfile has apt-get update and apt-get install is the moment your docker images stop being reproducible unless you take extreme measures like mirroring all deb repos you have in your sources.list. disclaimer: happy docker user here.
  - orangeshark 11y ago
    Where is everyone getting the idea that Docker is for creating reproducible environments? I know when I first heard about Docker, it was mentioned, but Docker does not claim to provide that. Seems like a confusion between distributable and reproducible.
e12e 11y ago
I'm a little surprised that a veteran Debian user would fall into this kind of trap (again :). I think guix looks very interesting, but a couple of tips if "all" you need is to "code on the edge" in Debian: 1) Run Debian stable. Possibly add stuff from backports if you need to (eg: newer kernel for drivers. Newer xorg. Hopefully this shouldn't ever be needed for mainstream workstations - modulus closed source graphics drivers). 2) Don't mix'n'match [packages from testing/unstable with stable]. Don't pin. Just do not do it. [Don't run testing/unstable... unless you are testing testing]. 3) For utilities not in stable - some can go in ~/opt/{bin,lib,man} -- living in ~/opt/xstow/$package-$version/ -- see "man xstow/apt-get install xstow" -- and set your path, ld-path, man-path ("man man") and friends to point to ~/opt/man etc. I also have a ~/opt/venv/util/bin in my path so I can go "pip install mercurial" without worrying about system python packages etc. If you go down this path be aware of "apt-get build-dep mercurial|python-$foo" as a reasonably sane way to get system dev headers for c libraries things you pip install in your venv(s). Think of the venvs as disposable! Might have to trash them on a dist-upgrade and recreate. Ditto for ~/xstow. But none of that is much better than the mess Wingo found himself in, therefore: 4) Embrace the glorious trinity formed by lvm, schroot and debootstrap! With lvm-backed schroots you can have a source-chroot for each of testing, sid/unstable and experimental. Complete with rollback, named snapshots and automagic binding of $home (with among other obvious benefits - your x session cookie). Schroot documentation/wiki/howto do need a facelift, though. See also: https://www.pseudorandom.co.uk/2007/sbuild/ https://www.pseudorandom.co.uk/2007/sbuild/ https://wiki.debian.org/Schroot/e https://wiki.debian.org/Schroot/e https://wiki.debian.org/CrossCompiling https://wiki.debian.org/CrossCompiling [Note that auto-bind-mounting home can now be set in schroot.conf IIRC -- see man schroot] PS: If you have $home on nfs and mount it on both 32bit/64bit linux as well as Solaris... you can put stuff in ~/opt/$arch/.. and dance around in .xsession and/or .bashrc... but I don't recommend it unless you have to...

73 points by apaprocki 11y ago

What's the significance of "V8"? He means the javascript engine, right? But that never enters into the story.... It's really just "Developing with Guix".

swah 11y ago
wingo likes/studies v8? https://wingolog.org/tags/v8 https://wingolog.org/tags/v8
- amirouche 11y ago
  he is knowledgeable in the VM implementation, and do develops features in v8 and GNU Guile VM.
davexunit 11y ago
Towards the end of the post he shows how he creates his reproducible v8 dev environment.
- knodi123 11y ago
  My bad, I didn't make it that far. Dude spent the first half of the article bitching about his Debian box, and I skimmed the rest (apparently not very well).
  - amirouche 11y ago
    He explains how nix/guix technology in the realm of personnal computing can improve day-to-day work, in this case the work of a developper. Wingo's setup is kind of complicated: nixos+guix. (I use only Guix OS for Django dev). In my case, Guix is overkill most of the time, since most dependencies are python. Guix can also replace virtualenv taking into account binary dependencies.. Binary dependency is the case met by Wingo. I think he does not stress enough (first part) the fact that upgrading is painless in nixos and Guix OS (almost, it's still alpha) compared to debian-like distros. Guix is a good idea. It needs more packages that is all.
  - snorrah 11y ago
    The "bitching" about his Debian install was what I found most interesting. My view of Debian is of a slow-to-change, highly stable distribution. Now, bearing in mind he had switched to the unstable branch but still found it reliable over a lengthy period of time, it was somewhat disturbing to see he could end up in a pretty bad spot after a reboot.

vezzy-fnord 11y ago

They use "deco" as an init system, for example, which is kinda like systemd but not. The init system is actually GNU dmd. deco is the name of the control utility (like systemctl is to systemd). It also has absolutely nothing in common with systemd, so I don't know why the author made that comparison.

davexunit 11y ago
It's hard to talk about an alternative init system without comparing it to systemd these days. I wish wingo would have mentioned why we use dmd in the first place. It's not out of systemd hatred, but rather for better system integration. dmd is written in Guile Scheme, just like Guix, and that gives us the two programs really excellent integration with each other. dmd needs more love to approach the feature set of systemd, but I think using it is the right choice for the long term. In addition to using it on my GuixSD systems, I also use it as a user service manager at work and it works great. Also, there's a proposed project out there for someone to implement a Guile reader that reads in systemd unit files and produces the equivalent dmd service object. If this is done, it should assist adoption, or at the very least experimentation.

davexunit 11y ago

I wonder if wingo knows about the 'guix environment' tool or not. It can create dev environments with the correct environment variables configured on the fly, without creating a new profile or polluting an existing one. Profiles are still useful for persistence, but I use 'guix environment' for all of my day-to-day hacking so I don't have to periodically delete generations of a profile in order for the GC to collect them.

sandGorgon 11y ago

IMHO, there is only a little bit of separation between Docker and Nix/Guix. I wonder if the Linux desktops of tomorrow will be a thin hypervisor (or something like CoreOS), running a immutable VM on top.

davexunit 11y ago
There's quite a lot of separation, IMO. Docker uses an imperative, stateful system for image creation that doesn't actually help with reproducibility and specifically only creates container environments. Guix and Nix use a functional, declarative system for building anything (software builds, VM image creation, etc.) and can create any type of environment, including containers. Guix and Nix are much more general-purpose and are built upon a stronger foundation. So, rather than using Docker running Debian/Ubuntu/whatever images and another OS entirely managing the host, you can use Guix or Nix to manage every layer and take advantage of their very advanced features everywhere.
- mands 11y ago
  I'm a big fan of Guix/Nix and would love to see them in greater use. Whilst I agree that they provide many of the reproducibly benefits of containers, isn't the other big advantage of containers the (theoretical) security benefits they bring? i.e. a process can't break out of its container and access other data. Perhaps I'm missing something but can such isolation be provided by Guix/Nix?
  - davexunit 11y ago
    Yes it can. The build daemon already uses containers to create the isolated environments to perform builds in so that we can view builds as pure functions. Nix uses systemd-nspawn for making NixOS containers. Guix uses a Scheme interface that I wrote called call-with-container to build GuixSD containers or simple containers without an init system via 'guix environment'. call-with-container is a work in progress, but some of the code has already made it to master, and the rest of it should land in master in time for the next release.

mrottenkolber 11y ago

Just an anecdote: Since I started using Slackware Linux, I have never had any problem with my system. I like the idea of functional package managers, Guix especially. But for me personally the problem they are trying to solve does not exist.

vezzy-fnord 11y ago
I too am a diehard Slackware user (I would say "Slack" like J.R. Dobbs intended it, but people would confuse it for the chat) and I'll echo your sentiment. Guix and Nix are the only ones in the field which deeply identify and solve issues in package management, and are the solutions I'd be most open to using. For personal usage, the Slackware tarball format without dependency resolution has never failed me. (I also have an affinity for Portage, but it's a delicate thing to use.)
- ics 11y ago
  Bob be with you.
1_player 11y ago
Functional package managers (FPM) to me are a viable solution to (cloud) deployment: what is the difference between a FPM controlling the totality of inputs and outputs of an application, and modern containerization solutions such as Docker, really? I've played with Docker (and rkt) and there's something off putting about using this stuff in production, to me at least, while I'm itching to see how Nix and Guix grow; waiting for someone with more free time than me to really cultivate this potential. Nix is more mature, but I dislike the Haskell syntax. I love Scheme, but I hate the free-software-only-religion around GNU Guix: I appreciate what GNU does, I'm a regular GNU/Linux user, but at this stage of development religious beliefs are just a hindrance to adoption and it makes me sad. EDIT: and let's not forget about node provisioning, something you can already do with NixOS (no idea about GuixSD): forget Ansible and its declarative configuration, but define the totality of the system state and configuration in a single file. Tell the machine "OK configure yourself like this", and voilà, you have a node which is (formally?) proven to be exactly like you've described. That's the future.

amelius 11y ago

> but it is just unacceptable and revolting that software development in 2015 is exposed to an upgrade process which (1) can break your system (2) by default and (3) can't be rolled back. I would like to add that building a system that makes it difficult to reproduce the exact same state on another machine, even if the central/distributed repository it originated from is not available, should be considered malpractice.

zzalpha 11y ago

Huh... I had no idea Nix and Guix could be used this way. I viewed them as replacements to apt, but what I'm hearing is I could just as easily use it as a souped up stow... interesting...

KirinDave 11y ago

I've tried using Nix a few times. The idea is in principle not bad, but the actual deployment for, say... an API endpoint? Horrendous. Problems include: 1. Scripts deployed almost never appeal to /usr/bin/env, they tend to hardwire interpreters to /bin/sh. This flat out doesn't work in NixOS, and the Nix team is ideologically opposed to maintaining a compat layer for specific cases like this. 2. Compiled binaries are compiled to this SPECIFIC version of Nix. That may not seem like a bad thing, but the reality is that it's very abusive to work with. Your binaries cannot even be invoked becuase they cannot find their links (including, you know, the C bootstrapping lib). This means you can make an executable on your dev box, try and push it to prod, and because prod doesn't have something like a dtrace library the entire system is incompatible with your executable. 3. There are tools to assist with these problems, because they're so common. However, they themselves are full of edge cases they don't solve for that they're very hit-or-miss. While this isn't bad in and of itself, it tends to make the established userbase very unsympathetic and often very formulaic when you approach them with more subtle linking issues. This can be quite frustrating. I found the entire system somewhat unworkable. I also feel like Docker or Rocket solves a lot of the practical problems I have there, despite protestations. Container images are perfectly reproducible, but also easy to update. They also tend not to rip themselves apart if you, in a moment of intellectual weakness, give a slightly confusing order to the nixos package manager and rip your system in half.

baq 11y ago
the moment your Dockerfile has apt-get update and apt-get install is the moment your docker images stop being reproducible unless you take extreme measures like mirroring all deb repos you have in your sources.list. disclaimer: happy docker user here.
- orangeshark 11y ago
  Where is everyone getting the idea that Docker is for creating reproducible environments? I know when I first heard about Docker, it was mentioned, but Docker does not claim to provide that. Seems like a confusion between distributable and reproducible.

e12e 11y ago

I'm a little surprised that a veteran Debian user would fall into this kind of trap (again :). I think guix looks very interesting, but a couple of tips if "all" you need is to "code on the edge" in Debian: 1) Run Debian stable. Possibly add stuff from backports if you need to (eg: newer kernel for drivers. Newer xorg. Hopefully this shouldn't ever be needed for mainstream workstations - modulus closed source graphics drivers). 2) Don't mix'n'match [packages from testing/unstable with stable]. Don't pin. Just do not do it. [Don't run testing/unstable... unless you are testing testing]. 3) For utilities not in stable - some can go in ~/opt/{bin,lib,man} -- living in ~/opt/xstow/$package-$version/ -- see "man xstow/apt-get install xstow" -- and set your path, ld-path, man-path ("man man") and friends to point to ~/opt/man etc. I also have a ~/opt/venv/util/bin in my path so I can go "pip install mercurial" without worrying about system python packages etc. If you go down this path be aware of "apt-get build-dep mercurial|python-$foo" as a reasonably sane way to get system dev headers for c libraries things you pip install in your venv(s). Think of the venvs as disposable! Might have to trash them on a dist-upgrade and recreate. Ditto for ~/xstow. But none of that is much better than the mess Wingo found himself in, therefore: 4) Embrace the glorious trinity formed by lvm, schroot and debootstrap! With lvm-backed schroots you can have a source-chroot for each of testing, sid/unstable and experimental. Complete with rollback, named snapshots and automagic binding of $home (with among other obvious benefits - your x session cookie). Schroot documentation/wiki/howto do need a facelift, though. See also: https://www.pseudorandom.co.uk/2007/sbuild/ https://www.pseudorandom.co.uk/2007/sbuild/ https://wiki.debian.org/Schroot/e https://wiki.debian.org/Schroot/e https://wiki.debian.org/CrossCompiling https://wiki.debian.org/CrossCompiling [Note that auto-bind-mounting home can now be set in schroot.conf IIRC -- see man schroot] PS: If you have $home on nfs and mount it on both 32bit/64bit linux as well as Solaris... you can put stuff in ~/opt/$arch/.. and dance around in .xsession and/or .bashrc... but I don't recommend it unless you have to...