4 ms·

I am, like many here, totally confused. Is this OS-based virtualization, HW-based virtualization, para-virtualization, or something completely different? On t

I am, like many here, totally confused. Is this OS-based virtualization, HW-based virtualization, para-virtualization, or something completely different? On the one hand, there are clear indicators that this is OS-based virtualization ("there is a catch; however, LXD is only for Linux on Linux"). That's fine; that would essentially boil down to bringing the complete containment model of FreeBSD jails and illumos zones to Linux -- which I'm sure would be welcome by Linux folks dealing with the structural problems of the current (piece-meal) Linux containment model. But then there's the line about "working with silicon companies to ensure hardware-assisted security and isolation for these containers" -- WTF?! If using OS-based virtualization, why would you need hardware assistance for "security and isolation"?! And if that "hardware assistance" is being used for something so basic as proper containment, what happens if you don't have that assistance? Is LXD then vulnerable to privilege escalation? And who are the "silicon companies" we're talking about (like, is this Intel or is this not Intel?), what is the ISA, when does it tape out, how is it being validated, etc. etc. etc. It's very frustrating for an announcement to be so putatively technical and yet provide so few answers; is there a deeper technical explanation of LXD somewhere?

derefr 12y ago
> WTF?! If using OS-based virtualization, why would you need hardware assistance for "security and isolation"?! I would guess that Canonical is talking about getting companies to contribute Linux kernel patches for cgroup interfaces to various northbridge-managed hardware virtualization tech (e.g. IOMMU tech like Intel's VT-d.)
- robryk 12y ago
  How would you use hardware virtualization tech (other than a normal MMU) to separate processes that run on the same kernel from each other?
  - derefr 12y ago
    VT-d in particular gives you the ability to expose one piece of hardware (that knows how to partition itself in some way) as multiple devices on the PCIe bus. With cgroup support, a container could be assigned one of the split devices, and act within the container as if it were the whole device. This is what regular hypervisors do, but they require a full set of virtualized devices (a virtual CPU, a virtual memory, etc.) while this approach allows you to virtualize only the resources your containers actually want to contend over. So you could have, say, one virtual ethernet card per container (letting you run a container as a promiscuous-mode packet filter for its own VPC subnet, while still not being able to snoop on other VPCs' traffic) or one virtual GPU per container (allowing you to containerize OpenCL apps), while still having your containers acting like regular processes otherwise.
    - robryk 12y ago
      In order to get one virtual slice of device per container I just need the device's driver to support that or I need to have a layer on top of the driver that partitions the device. As the different cgroups have the same kernel and thus the same set of drivers I see no advantage in splitting physical devices. What am I missing?
      - justincormack 12y ago
        The master device has the facility to create the subdevices which look like onrmal devices with soem config options missing. I am not convinced that this is what they are talking about though...
rdtsc 12y ago
> If using OS-based virtualization, why would you need hardware assistance for "security and isolation"?! Today most OS based virtualization is using "hardware assistance". Those are for often for memory and IO device managment (even passthrough). Not sure if this is _the_ assistance they mention but just an example of how it could work.
- bcantrill 12y ago
  No, actually OS virtualization doesn't generally use any hardware assistance. And my questions don't stem from ignorance; I have extensive experience with the implementation of both OS virtualization[1] and HW virtualization[2] -- which is why I find the LXD specifics so peculiar. (All the more so that they imply that the support is forthcoming, not current -- and that they are talking to "silicon companies" not microprocessor vendors.) I think the best guess is what derefr posited, above: that they are using HW network virt as a way of avoiding building in proper network stack virtualization like that found in Crossbow.[3] Then again, given the degree to which LXD appears to be aspirational rather than actual, we might be overthinking it: perhaps the conversations with "silicon companies" are like LXD itself -- a daydream about what might be rather than a concrete reality. [1] http://www.slideshare.net/bcantrill/os-virtualization-40700689 http://www.slideshare.net/bcantrill/os-virtualization-407006... [2] http://www.slideshare.net/bcantrill/experiences-porting-kvm-to-smartos http://www.slideshare.net/bcantrill/experiences-porting-kvm-... [3] http://en.wikipedia.org/wiki/OpenSolaris_Network_Virtualization_and_Resource_Control http://en.wikipedia.org/wiki/OpenSolaris_Network_Virtualizat...
justincormack 12y ago
The mailing list post[1] doesnt mention any of the hypervisor stuff, except to say make it feel more like a hypervisor. So I think maybe this has been exaggerated... [1] https://lists.linuxcontainers.org/pipermail/lxc-devel/2014-November/010817.html https://lists.linuxcontainers.org/pipermail/lxc-devel/2014-N...

derefr 12y ago

> WTF?! If using OS-based virtualization, why would you need hardware assistance for "security and isolation"?! I would guess that Canonical is talking about getting companies to contribute Linux kernel patches for cgroup interfaces to various northbridge-managed hardware virtualization tech (e.g. IOMMU tech like Intel's VT-d.)

robryk 12y ago
How would you use hardware virtualization tech (other than a normal MMU) to separate processes that run on the same kernel from each other?
- derefr 12y ago
  VT-d in particular gives you the ability to expose one piece of hardware (that knows how to partition itself in some way) as multiple devices on the PCIe bus. With cgroup support, a container could be assigned one of the split devices, and act within the container as if it were the whole device. This is what regular hypervisors do, but they require a full set of virtualized devices (a virtual CPU, a virtual memory, etc.) while this approach allows you to virtualize only the resources your containers actually want to contend over. So you could have, say, one virtual ethernet card per container (letting you run a container as a promiscuous-mode packet filter for its own VPC subnet, while still not being able to snoop on other VPCs' traffic) or one virtual GPU per container (allowing you to containerize OpenCL apps), while still having your containers acting like regular processes otherwise.
  - robryk 12y ago
    In order to get one virtual slice of device per container I just need the device's driver to support that or I need to have a layer on top of the driver that partitions the device. As the different cgroups have the same kernel and thus the same set of drivers I see no advantage in splitting physical devices. What am I missing?
    - justincormack 12y ago
      The master device has the facility to create the subdevices which look like onrmal devices with soem config options missing. I am not convinced that this is what they are talking about though...

rdtsc 12y ago

> If using OS-based virtualization, why would you need hardware assistance for "security and isolation"?! Today most OS based virtualization is using "hardware assistance". Those are for often for memory and IO device managment (even passthrough). Not sure if this is _the_ assistance they mention but just an example of how it could work.

bcantrill 12y ago
No, actually OS virtualization doesn't generally use any hardware assistance. And my questions don't stem from ignorance; I have extensive experience with the implementation of both OS virtualization[1] and HW virtualization[2] -- which is why I find the LXD specifics so peculiar. (All the more so that they imply that the support is forthcoming, not current -- and that they are talking to "silicon companies" not microprocessor vendors.) I think the best guess is what derefr posited, above: that they are using HW network virt as a way of avoiding building in proper network stack virtualization like that found in Crossbow.[3] Then again, given the degree to which LXD appears to be aspirational rather than actual, we might be overthinking it: perhaps the conversations with "silicon companies" are like LXD itself -- a daydream about what might be rather than a concrete reality. [1] http://www.slideshare.net/bcantrill/os-virtualization-40700689 http://www.slideshare.net/bcantrill/os-virtualization-407006... [2] http://www.slideshare.net/bcantrill/experiences-porting-kvm-to-smartos http://www.slideshare.net/bcantrill/experiences-porting-kvm-... [3] http://en.wikipedia.org/wiki/OpenSolaris_Network_Virtualization_and_Resource_Control http://en.wikipedia.org/wiki/OpenSolaris_Network_Virtualizat...

justincormack 12y ago

The mailing list post[1] doesnt mention any of the hypervisor stuff, except to say make it feel more like a hypervisor. So I think maybe this has been exaggerated... [1] https://lists.linuxcontainers.org/pipermail/lxc-devel/2014-November/010817.html https://lists.linuxcontainers.org/pipermail/lxc-devel/2014-N...