5 ms·

LVE: an alternative container technology to Docker and Virtuozzo/LXC

74 points by FooBarWidget 11y ago

tyingq 11y ago
This is very interesting, but it seems to solve a problem that may not exist much longer. The big difference with LVE is that it works at a thread level, such that you can, for example, place one http request within a running apache instance into a specific user's cgroup. That makes it useful in shared hosting environments, where multiple end users are sharing one instance of apache, mysql, etc. The trouble with this, to me, is that shared hosting is on its deathbed. It is more efficient than the VPS model, but so much money is going into the VPS world (digitalocean, linode, ovh, etc). Thus, that efficiency edge doesn't translate into any cost savings for the end user. Shared hosting seems to survive solely because they provide some amount of managed service and somewhat end user friendly interfaces (cpanel). Sooner or later, though, players like DigitalOcean will fill that gap, with things like their "one click installs", improved control panels, etc. I'm not sure if there's another use case for LVE outside of shared hosting.
- iseletsk 11y ago
  Shared hosting is still growing. VPS is not going to kill it, as customers are different with very different needs. The efficiency does translate into a saving for those hosters, as it is common to see 2,000 Happy customers on a 16GB server. Something completely impossible with VPS.
  - tyingq 11y ago
    >>The efficiency does translate into a saving for those hosters This is true from a hardware standpoint. However, after paying for CloudLinux and Cpanel licenses, the advantage may falter. And, they are racing deeper pockets to the bottom.
    - iseletsk 11y ago
      well, cPanel is needed by clients. VPS doesn't come with convenient fully managed UI and sysadmins to care about your site there. CloudLinux is required when you want to improve stability/density. Even if density un-important, improving stability is critical & revenue generating activity
      - tyingq 11y ago
        That is roughly how things are today, yes. In the future, if the VPS vendors don't go down the road of addressing those two gaps, it would remain that way.
- icebraining 11y ago
  What about PaaS environments? We are a B2B SaaS provider, and unlike seemingly every other provider out there, we want to have some isolation between accounts, so that a bug on our code doesn't allow for cross-account data leakages or worse. Right now we run a separately contained process for each account, but per-thread containment could help us gain some of the advantages of the fully multi-tenant architecture without losing the account isolation.
  - tyingq 11y ago
    I suppose it's whether the effort to integrate that thread-level isolation drives enough benefit to justify itself. For cloudlinux customers (shared hosting providers) the typical things they need already have the mods put in by cloudlinux (apache, mysql,etc). Since cloudlinux is, I believe, based on an older version OpenVZ, you would also have to likely backrev to a 2.6.x kernel.
    - iseletsk 11y ago
      you are correct about integration costs. We do have 3.10 (latest rhel) kernel support as well.
- jo909 11y ago
  Saas is shared hosting of a specific application. Paas can be shared hosting. Both most often do not use distinct vms per user.
kstenerud 11y ago
"LVE stands for Lightweight Virtual Environments, a proprietary technology from CloudLinux." Proprietary is the main reason why it will lose to containers. "Unlike Docker, Virtuozzo and LXC, which operate on the process level, LVE is able to operate on the thread level. This allows multithreaded servers such as Apache (with its 'worker' MPM) to take advantage of LVE without having to run a separate instance per LVE user." This is the other reason why it will lose: Increased attack surface to every API of every single process they use on the shared server, AKA "bad neighbor effect" with a vengeance.
- iseletsk 11y ago
  It is not that it will lose to the containers. It is just another way to consume containers. Is it going to be as popular as Docker -- no. It is a niche product for specific market that solves the needs of this market. There might be other needs that it can address. Some people address it now by using standard cgroup / namespaces interfaces.
rciorba 11y ago
From the article: "LVE stands for Lightweight Virtual Environments, a proprietary technology from CloudLinux." I'm confused about the proprietary bit. As far as I can find they released a kernel module in the past that was in violation of the GPL but then redressed the issue by releasing the sources under the GPL. Does anyone know what the current state is?
- iseletsk 11y ago
  We release our kernel & modules as GPL. We are fully GPL compliant.
- Xylakant 11y ago
  The GPL only requires you to distribute the source if you distribute the binary. It does not require to open the source if you built something you're only using internally. So while distributing the binary kernel module without the source would be a violation of the GPL, just using it internally on your own servers would not be. (IANAL, ...)
  - tyingq 11y ago
    They (cloudlinux) do distribute the binary.
    - Xylakant 11y ago
      and, as noted, the source as well.
      - tyingq 11y ago
        Somebody important seems unconvinced: https://github.com/torvalds/linux/blob/master/kernel/module.c#L3028 https://github.com/torvalds/linux/blob/master/kernel/module....
        iseletsk 11y ago
        This was done when we initially (several years ago), by mistake licensed our module as GPL without releasing the source. Took us one or two weeks to see if we have any other legal concerns to release the sources, and then released it. We did release all the sources back then, and continue to release sources each time we update module. We really never went around / removing that code -- because honestly speaking we couldn't care less. Our module wouldn't work without our kernel anyway, and in our kernel it is easy to remove :) And for those who care / want to use our kernel/module for other reasons, great -- they can work with mainstream to remove this. Basically -- this is just a dead code.
merb 11y ago
You don't need a kernel module for most of this stuff. Thanks to systemd / cgroups. I already said it that docker is a unnecessary overhead layer however, people like it just too much.
- iseletsk 11y ago
  I don't think you can. We tried & failed. Maybe in the future. The main problem is with switching process/thread from cgroup to cgroup at very rapid pace (with small overhead), and identifying to which cGroup you need to switch them. Don't get me wrong, it is possible, but requires some extra / tiny modifications to kernel.
nikolay 11y ago
Proprietary technology in a subscription-based Linux? Thanks, but no thanks! I wish I was able to undo my upvote!
cyphar 11y ago
I can't begin to see how it could possibly be secure to run different threads (read: shared memory) as your containerised primitive. That's just ridiculous. Sure, you might get some performance. But a bug in the version of Apache you're sharing between containers (which, in true shared hosting fashion, has taken security out of your hands) is going to break every container on the box (as well as whatever namespace the "master" Apache process is in). And I'm not sure this model would work well with pool-based concurrency models (like with nginx, where work is given to workers, workers aren't spawned for each request). However, it would be cool if this was just merged upstream, so that Apache could take advantage of the resource limiting. I'm not sure that it's a good general solution for shared hosts though (then again, you never trust shared hosts after you pop your first shell).
- iseletsk 11y ago
  We would be happy to merge it upstream, but at this moment merging upstream requires much more work that we can dedicate to it.

74 points by FooBarWidget 11y ago

This is very interesting, but it seems to solve a problem that may not exist much longer. The big difference with LVE is that it works at a thread level, such that you can, for example, place one http request within a running apache instance into a specific user's cgroup. That makes it useful in shared hosting environments, where multiple end users are sharing one instance of apache, mysql, etc. The trouble with this, to me, is that shared hosting is on its deathbed. It is more efficient than the VPS model, but so much money is going into the VPS world (digitalocean, linode, ovh, etc). Thus, that efficiency edge doesn't translate into any cost savings for the end user. Shared hosting seems to survive solely because they provide some amount of managed service and somewhat end user friendly interfaces (cpanel). Sooner or later, though, players like DigitalOcean will fill that gap, with things like their "one click installs", improved control panels, etc. I'm not sure if there's another use case for LVE outside of shared hosting.

iseletsk 11y ago
Shared hosting is still growing. VPS is not going to kill it, as customers are different with very different needs. The efficiency does translate into a saving for those hosters, as it is common to see 2,000 Happy customers on a 16GB server. Something completely impossible with VPS.
- tyingq 11y ago
  >>The efficiency does translate into a saving for those hosters This is true from a hardware standpoint. However, after paying for CloudLinux and Cpanel licenses, the advantage may falter. And, they are racing deeper pockets to the bottom.
  - iseletsk 11y ago
    well, cPanel is needed by clients. VPS doesn't come with convenient fully managed UI and sysadmins to care about your site there. CloudLinux is required when you want to improve stability/density. Even if density un-important, improving stability is critical & revenue generating activity
    - tyingq 11y ago
      That is roughly how things are today, yes. In the future, if the VPS vendors don't go down the road of addressing those two gaps, it would remain that way.
icebraining 11y ago
What about PaaS environments? We are a B2B SaaS provider, and unlike seemingly every other provider out there, we want to have some isolation between accounts, so that a bug on our code doesn't allow for cross-account data leakages or worse. Right now we run a separately contained process for each account, but per-thread containment could help us gain some of the advantages of the fully multi-tenant architecture without losing the account isolation.
- tyingq 11y ago
  I suppose it's whether the effort to integrate that thread-level isolation drives enough benefit to justify itself. For cloudlinux customers (shared hosting providers) the typical things they need already have the mods put in by cloudlinux (apache, mysql,etc). Since cloudlinux is, I believe, based on an older version OpenVZ, you would also have to likely backrev to a 2.6.x kernel.
  - iseletsk 11y ago
    you are correct about integration costs. We do have 3.10 (latest rhel) kernel support as well.
jo909 11y ago
Saas is shared hosting of a specific application. Paas can be shared hosting. Both most often do not use distinct vms per user.

kstenerud 11y ago

"LVE stands for Lightweight Virtual Environments, a proprietary technology from CloudLinux." Proprietary is the main reason why it will lose to containers. "Unlike Docker, Virtuozzo and LXC, which operate on the process level, LVE is able to operate on the thread level. This allows multithreaded servers such as Apache (with its 'worker' MPM) to take advantage of LVE without having to run a separate instance per LVE user." This is the other reason why it will lose: Increased attack surface to every API of every single process they use on the shared server, AKA "bad neighbor effect" with a vengeance.

iseletsk 11y ago
It is not that it will lose to the containers. It is just another way to consume containers. Is it going to be as popular as Docker -- no. It is a niche product for specific market that solves the needs of this market. There might be other needs that it can address. Some people address it now by using standard cgroup / namespaces interfaces.

rciorba 11y ago

From the article: "LVE stands for Lightweight Virtual Environments, a proprietary technology from CloudLinux." I'm confused about the proprietary bit. As far as I can find they released a kernel module in the past that was in violation of the GPL but then redressed the issue by releasing the sources under the GPL. Does anyone know what the current state is?

iseletsk 11y ago
We release our kernel & modules as GPL. We are fully GPL compliant.
Xylakant 11y ago
The GPL only requires you to distribute the source if you distribute the binary. It does not require to open the source if you built something you're only using internally. So while distributing the binary kernel module without the source would be a violation of the GPL, just using it internally on your own servers would not be. (IANAL, ...)
- tyingq 11y ago
  They (cloudlinux) do distribute the binary.
  - Xylakant 11y ago
    and, as noted, the source as well.
    - tyingq 11y ago
      Somebody important seems unconvinced: https://github.com/torvalds/linux/blob/master/kernel/module.c#L3028 https://github.com/torvalds/linux/blob/master/kernel/module....
      - iseletsk 11y ago
        This was done when we initially (several years ago), by mistake licensed our module as GPL without releasing the source. Took us one or two weeks to see if we have any other legal concerns to release the sources, and then released it. We did release all the sources back then, and continue to release sources each time we update module. We really never went around / removing that code -- because honestly speaking we couldn't care less. Our module wouldn't work without our kernel anyway, and in our kernel it is easy to remove :) And for those who care / want to use our kernel/module for other reasons, great -- they can work with mainstream to remove this. Basically -- this is just a dead code.

merb 11y ago

You don't need a kernel module for most of this stuff. Thanks to systemd / cgroups. I already said it that docker is a unnecessary overhead layer however, people like it just too much.

iseletsk 11y ago
I don't think you can. We tried & failed. Maybe in the future. The main problem is with switching process/thread from cgroup to cgroup at very rapid pace (with small overhead), and identifying to which cGroup you need to switch them. Don't get me wrong, it is possible, but requires some extra / tiny modifications to kernel.

nikolay 11y ago

Proprietary technology in a subscription-based Linux? Thanks, but no thanks! I wish I was able to undo my upvote!

cyphar 11y ago

I can't begin to see how it could possibly be secure to run different threads (read: shared memory) as your containerised primitive. That's just ridiculous. Sure, you might get some performance. But a bug in the version of Apache you're sharing between containers (which, in true shared hosting fashion, has taken security out of your hands) is going to break every container on the box (as well as whatever namespace the "master" Apache process is in). And I'm not sure this model would work well with pool-based concurrency models (like with nginx, where work is given to workers, workers aren't spawned for each request). However, it would be cool if this was just merged upstream, so that Apache could take advantage of the resource limiting. I'm not sure that it's a good general solution for shared hosts though (then again, you never trust shared hosts after you pop your first shell).

iseletsk 11y ago
We would be happy to merge it upstream, but at this moment merging upstream requires much more work that we can dedicate to it.