4 ms·

for the love of god - forget docker, use lxc containers - its simple, secure, goes with its own init, cron, and you dont need to do somersaults to achieve simpl

by parito 10y ago

for the love of god - forget docker, use lxc containers - its simple, secure, goes with its own init, cron, and you dont need to do somersaults to achieve simple tasks. Included with linux kernel. Your own isolated linux system. We use lxc in production for over three years, and we have over 3000 of them. No issues whatsoever.

austinjp 10y ago
Encouraging to hear. Who do you work for, who has these 3,000 LXC containers in production use? And I'm curious, what orchestration system do you use to manage them? Can you outline your toolset?
- parito 10y ago
  We use ansible and bash scripts for orchestration.
  - austinjp 10y ago
    If you'd care to go into more detail, or could point me to your technical docs, I'd be very interested. How, for example, do you handle roll-out, destruction, monitoring, backups, network configs, secrets, etc etc... is there any degree of automation? And why not take advantage of existing orchestration solutions? Was nothing mature enough for your needs? How big is the team managing these 3,000 containers, what sort of traffic are you handling? Asking out of genuine curiosity. I'm keen on understanding how plain LXC can be used robustly in a production environment.
choxi 10y ago
You're right that LXC containers have a similar API compared to Docker, but I think developers often underestimate the benefit of the community around a certain technology. Docker has significantly better documentation, extensions, package management tools, and third-party integrations. Overall, Docker has an incredibly more robust community than LXC or closer competitors like Kubernetes, and those features are just as important as the API for developers.
- parito 10y ago
  The point of the LXC is, you get a full blown standalone linux, rather than a single process - this simplifies everything a lot, meaning you don't have to have that much documentation about it in the first place.
  - majewsky 10y ago
    Can you clarify what "full-blown standalone Linux" means? It sounds like running a separate kernel, but since we're talking containers rather than VMs, this can't be it.
    - Kubuxu 10y ago
      It is shared kernel, separate userspace. It uses: X-namespaces (network, pid, user, ...) and cgroups to separate those userspaces from each other. I have community server running debian in which there are 10+ LXC containers running in which people are given normal root access, one container per user.
      - majewsky 10y ago
        So it's the same as with Docker.
  - deleted 10y ago
    [deleted]
winter_blue 10y ago
Doesn't Docker use lxc underneath? [1] [1] http://unix.stackexchange.com/a/254977/152994 http://unix.stackexchange.com/a/254977/152994
- justincormack 10y ago
  Not for a few years now.
flukus 10y ago
Do you have any examples of what LXC does better than docker? I'm very new to the whole containerization thing but I've already come across a couple of the issues you've mentioned.
- parito 10y ago
  Shameless copypaste from well written piece by Flockport: Docker restricts the container to a single process only. The default docker baseimage OS template is not designed to support multiple applications, processes or services like init, cron, syslog, ssh etc. As we saw earlier this introduces a certain amount of complexity for day to day usage scenarios. Since current architectures, applications and services are designed to operate in normal multi process OS environments you would need to find a Docker way to do things or use tools that support Docker. Take a simple application like WordPress. You would need to build 3 containers that consume services from each other. A PHP container, an Nginx container and a MySQL container plus 2 separate containers for persistent data for the Mysql DB and WordPress files. Then configure the WordPress files to be available to both the PHP-FPM and Nginx containers with the right permissions, and to make things more exciting figure out a way to make these talk to each other over the local network, without proper control of networking with randomly assigned IPs by the Docker daemon! And we have not yet figured cron and email that WordPress needs for account management. Phew! This is a can of worms and a recipe for brittleness. This is a lot of work that you would just not have to even think about with OS containers. This adds an unbelievable amount of complexity and fragility to basic deployment and now with hacks, workarounds and entire layers being developed to manage this complexity. This cannot be the most efficient way to use containers. Can you build all 3 in one container? You can, but then why not just simply use LXC which is designed for multi processes and is simpler to use. To run multiple processes in Docker you need a shell script or a separate process manager like runit or supervisor. But this is considered an 'anti-pattern' by the Docker ecosystem and the whole architecture of Docker is built around single process containers. Docker separates container storage from the application, you mount persistent data with bind mounts to the host (data volumes) or bind mounts to containers (data volume containers) This is one of the most baffling decisions, by bind mounting data to the host you are eliminating one of the biggest features of containers for end users; easy mobility of containers across hosts. Probably as a concession Docker gives you data volumes, which is a bind mount to a normal container and is portable but this is yet another additional layer of complexity, and reflects just how much Docker is driven by the PAAS provider use case of app instances.

by parito 10y ago

austinjp 10y ago

Encouraging to hear. Who do you work for, who has these 3,000 LXC containers in production use? And I'm curious, what orchestration system do you use to manage them? Can you outline your toolset?

parito 10y ago
We use ansible and bash scripts for orchestration.
- austinjp 10y ago
  If you'd care to go into more detail, or could point me to your technical docs, I'd be very interested. How, for example, do you handle roll-out, destruction, monitoring, backups, network configs, secrets, etc etc... is there any degree of automation? And why not take advantage of existing orchestration solutions? Was nothing mature enough for your needs? How big is the team managing these 3,000 containers, what sort of traffic are you handling? Asking out of genuine curiosity. I'm keen on understanding how plain LXC can be used robustly in a production environment.

choxi 10y ago

You're right that LXC containers have a similar API compared to Docker, but I think developers often underestimate the benefit of the community around a certain technology. Docker has significantly better documentation, extensions, package management tools, and third-party integrations. Overall, Docker has an incredibly more robust community than LXC or closer competitors like Kubernetes, and those features are just as important as the API for developers.

parito 10y ago
The point of the LXC is, you get a full blown standalone linux, rather than a single process - this simplifies everything a lot, meaning you don't have to have that much documentation about it in the first place.
- majewsky 10y ago
  Can you clarify what "full-blown standalone Linux" means? It sounds like running a separate kernel, but since we're talking containers rather than VMs, this can't be it.
  - Kubuxu 10y ago
    It is shared kernel, separate userspace. It uses: X-namespaces (network, pid, user, ...) and cgroups to separate those userspaces from each other. I have community server running debian in which there are 10+ LXC containers running in which people are given normal root access, one container per user.
    - majewsky 10y ago
      So it's the same as with Docker.
- deleted 10y ago
  [deleted]

winter_blue 10y ago

Doesn't Docker use lxc underneath? [1] [1] http://unix.stackexchange.com/a/254977/152994 http://unix.stackexchange.com/a/254977/152994

justincormack 10y ago
Not for a few years now.

flukus 10y ago

Do you have any examples of what LXC does better than docker? I'm very new to the whole containerization thing but I've already come across a couple of the issues you've mentioned.

parito 10y ago
Shameless copypaste from well written piece by Flockport: Docker restricts the container to a single process only. The default docker baseimage OS template is not designed to support multiple applications, processes or services like init, cron, syslog, ssh etc. As we saw earlier this introduces a certain amount of complexity for day to day usage scenarios. Since current architectures, applications and services are designed to operate in normal multi process OS environments you would need to find a Docker way to do things or use tools that support Docker. Take a simple application like WordPress. You would need to build 3 containers that consume services from each other. A PHP container, an Nginx container and a MySQL container plus 2 separate containers for persistent data for the Mysql DB and WordPress files. Then configure the WordPress files to be available to both the PHP-FPM and Nginx containers with the right permissions, and to make things more exciting figure out a way to make these talk to each other over the local network, without proper control of networking with randomly assigned IPs by the Docker daemon! And we have not yet figured cron and email that WordPress needs for account management. Phew! This is a can of worms and a recipe for brittleness. This is a lot of work that you would just not have to even think about with OS containers. This adds an unbelievable amount of complexity and fragility to basic deployment and now with hacks, workarounds and entire layers being developed to manage this complexity. This cannot be the most efficient way to use containers. Can you build all 3 in one container? You can, but then why not just simply use LXC which is designed for multi processes and is simpler to use. To run multiple processes in Docker you need a shell script or a separate process manager like runit or supervisor. But this is considered an 'anti-pattern' by the Docker ecosystem and the whole architecture of Docker is built around single process containers. Docker separates container storage from the application, you mount persistent data with bind mounts to the host (data volumes) or bind mounts to containers (data volume containers) This is one of the most baffling decisions, by bind mounting data to the host you are eliminating one of the biggest features of containers for end users; easy mobility of containers across hosts. Probably as a concession Docker gives you data volumes, which is a bind mount to a normal container and is portable but this is yet another additional layer of complexity, and reflects just how much Docker is driven by the PAAS provider use case of app instances.