4 ms·

Wow this is huge. Having an OS be able to run both ZFS and KVM enables fantastic things. For example you can store each virtual machine disk template on a dedic

by mrb 15y ago

Wow this is huge. Having an OS be able to run both ZFS and KVM enables fantastic things. For example you can store each virtual machine disk template on a dedicated ZFS filesystem, and use "zfs clone" to rapidly deploy VMs (instead of using the KVM-level support for base disk images "qemu-img create -b ..."), as well as "zfs snapshot", "zfs revert", etc. The main advantage being that these clones and snapshots are possible while using the simple and fast "raw" KVM disk image format instead of the notoriously slower "qcow2" format that was, until today, the best format supporting base images and snapshots. I, for one, have been wanting to use ZFS specifically like that for a while. This does not compare at all to running ZFS on, say, an NFS server or iSCSI SAN serving data to a server running KVM (ZFS data integrity is only verified remotely on the storage server, it is slower, etc). Who ported KVM to Illumos? I know some old version of QEMU was running on Solaris at some point in the past, but I had no idea they had a full blown KVM port.

amazingman 15y ago
http://smartos.org/2011/08/15/kvm-on-illumos/ http://smartos.org/2011/08/15/kvm-on-illumos/
bcantrill 15y ago
KVM was ported to illumos by Max Bruning, Robert Mustacchi and me. Details are here: http://t.co/knW1UIJ http://t.co/knW1UIJ
- rednaught 15y ago
  Can you provide more details of Crossbow? Who maintains it/status? I hadn't heard anything about it since OpenSolaris went dead.
  - bcantrill 15y ago
    Could you be more specific? It's there, it works, it's in shipping illumos-based products (from Joyent and others) and it's supported by those who ship it or stand it up. In that regard, it's like any other aspect of the system...
    - rednaught 15y ago
      Thank you. I guess I just have to go dig through the Illumos/OpenIndiana documents.
- djcapelis 15y ago
  Uhm, isn't that a violation of the GPL given that you're mixing the complete work with non-compatible (by design) CDDL code? Did you ask for KVM to be relicensed for the port or something?
  - bcantrill 15y ago
    There is not a licensing issue here: both KVM and our KVM port remain licensed under the GPL.
    - djcapelis 15y ago
      Isn't your port of KVM integrated into the Solaris kernel? How are you ensuring they are two independent and separate works under copyright law?
      - bcantrill 15y ago
        No; the KVM kernel module is in its own repo: https://github.com/joyent/illumos-kvm https://github.com/joyent/illumos-kvm
        sp332 15y ago
        But distributing binaries built with some GPL'd code is a violation of the GPL if you don't publish all the code that went into the binary.
        sriramnrn 15y ago
        A number of people in the openindiana community have compiled the KVM bits, and no one has complained about binary-only modules, yet. Did you encounter any binary-only files ?
      - kmavm 15y ago
        My (somewhat casual) understanding is that the Solaris kernel has always, by necessity, supported modules under a different license much more deeply than Linux. Commercially significant software like the Veritas filesystem and device drivers have always worked this way. So, e.g., the Solaris kernel has a real, no-foolin' ABI.
- slillibri 15y ago
  Kind of an off topic rant, but can we at least avoid url shorteners that point to other url shorteners? Here is the original link http://www.slideshare.net/bcantrill/experiences-porting-kvm-to-smartos http://www.slideshare.net/bcantrill/experiences-porting-kvm-...
  - fanf2 15y ago
    SlideShare is also a world of hate. Try http://www.linux-kvm.org/wiki/images/7/71/2011-forum-porting-to-smartos.pdf http://www.linux-kvm.org/wiki/images/7/71/2011-forum-porting...
  - mambodog 15y ago
    Honestly, no one should be using URL shorteners on HN. Use a footnote[1] if necessary. [1] http://www.google.com/search?q=hi,+this+is+a+footnote http://www.google.com/search?q=hi,+this+is+a+footnote
    - RexRollman 15y ago
      I like the footnote form. I use that in e-mails as well.
- emaste 15y ago
  I really like the use of DTrace in #if 0'd out code for identifying the highest priorities on porting. Having faced a similar task that seemed impenetrable at the start I would have really liked to have this technique available. Nice work and congrats on the port.
jlawer 15y ago
While ZFS adds some advantages, if you were going for snap shots and quick deploy with KVM your best bet was to use LVM as a raw device. A quick lvm snapshot + virt-clone + virsh start will let me have a cloned vm up within seconds. Thats not saying ZFS doesn't add a lot (checksum backed store, heap of other features). But I would actually expect LVM to be faster, precisely because it is "crappier" (no checksumming, very basic address remapping, etc). The real question will be support, if they can convince people that this will have enough life to build a user base large enough to support it moving forward. Especially if oracle stops publishing code under open source licenses.
- mrb 15y ago
  The problem with LVM snapshots is that you have to manually reserve a portion of the logical volume to store them. And if you calculate your needs incorrectly, LVM runs out of space to preserve them, and they become corrupted (they are reported as "INVALID" in lvdisplay IIRC). That's why I prefer KVM's built-in base image support (qemu-img create -b ...) to quickly provision VMs, instead of LVM. That's how I architected KVM hosts for my employer, running 100+ lightweight VMs each, with 1000+ disk images available on disk.

by mrb 15y ago

amazingman 15y ago

http://smartos.org/2011/08/15/kvm-on-illumos/ http://smartos.org/2011/08/15/kvm-on-illumos/

bcantrill 15y ago

KVM was ported to illumos by Max Bruning, Robert Mustacchi and me. Details are here: http://t.co/knW1UIJ http://t.co/knW1UIJ

rednaught 15y ago
Can you provide more details of Crossbow? Who maintains it/status? I hadn't heard anything about it since OpenSolaris went dead.
- bcantrill 15y ago
  Could you be more specific? It's there, it works, it's in shipping illumos-based products (from Joyent and others) and it's supported by those who ship it or stand it up. In that regard, it's like any other aspect of the system...
  - rednaught 15y ago
    Thank you. I guess I just have to go dig through the Illumos/OpenIndiana documents.
djcapelis 15y ago
Uhm, isn't that a violation of the GPL given that you're mixing the complete work with non-compatible (by design) CDDL code? Did you ask for KVM to be relicensed for the port or something?
- bcantrill 15y ago
  There is not a licensing issue here: both KVM and our KVM port remain licensed under the GPL.
  - djcapelis 15y ago
    Isn't your port of KVM integrated into the Solaris kernel? How are you ensuring they are two independent and separate works under copyright law?
    - bcantrill 15y ago
      No; the KVM kernel module is in its own repo: https://github.com/joyent/illumos-kvm https://github.com/joyent/illumos-kvm
      - sp332 15y ago
        But distributing binaries built with some GPL'd code is a violation of the GPL if you don't publish all the code that went into the binary.
        sriramnrn 15y ago
        A number of people in the openindiana community have compiled the KVM bits, and no one has complained about binary-only modules, yet. Did you encounter any binary-only files ?
    - kmavm 15y ago
      My (somewhat casual) understanding is that the Solaris kernel has always, by necessity, supported modules under a different license much more deeply than Linux. Commercially significant software like the Veritas filesystem and device drivers have always worked this way. So, e.g., the Solaris kernel has a real, no-foolin' ABI.
slillibri 15y ago
Kind of an off topic rant, but can we at least avoid url shorteners that point to other url shorteners? Here is the original link http://www.slideshare.net/bcantrill/experiences-porting-kvm-to-smartos http://www.slideshare.net/bcantrill/experiences-porting-kvm-...
- fanf2 15y ago
  SlideShare is also a world of hate. Try http://www.linux-kvm.org/wiki/images/7/71/2011-forum-porting-to-smartos.pdf http://www.linux-kvm.org/wiki/images/7/71/2011-forum-porting...
- mambodog 15y ago
  Honestly, no one should be using URL shorteners on HN. Use a footnote[1] if necessary. [1] http://www.google.com/search?q=hi,+this+is+a+footnote http://www.google.com/search?q=hi,+this+is+a+footnote
  - RexRollman 15y ago
    I like the footnote form. I use that in e-mails as well.
emaste 15y ago
I really like the use of DTrace in #if 0'd out code for identifying the highest priorities on porting. Having faced a similar task that seemed impenetrable at the start I would have really liked to have this technique available. Nice work and congrats on the port.

jlawer 15y ago

While ZFS adds some advantages, if you were going for snap shots and quick deploy with KVM your best bet was to use LVM as a raw device. A quick lvm snapshot + virt-clone + virsh start will let me have a cloned vm up within seconds. Thats not saying ZFS doesn't add a lot (checksum backed store, heap of other features). But I would actually expect LVM to be faster, precisely because it is "crappier" (no checksumming, very basic address remapping, etc). The real question will be support, if they can convince people that this will have enough life to build a user base large enough to support it moving forward. Especially if oracle stops publishing code under open source licenses.

mrb 15y ago
The problem with LVM snapshots is that you have to manually reserve a portion of the logical volume to store them. And if you calculate your needs incorrectly, LVM runs out of space to preserve them, and they become corrupted (they are reported as "INVALID" in lvdisplay IIRC). That's why I prefer KVM's built-in base image support (qemu-img create -b ...) to quickly provision VMs, instead of LVM. That's how I architected KVM hosts for my employer, running 100+ lightweight VMs each, with 1000+ disk images available on disk.