6 ms·

Dockerlite: Lightweight Linux virtualization with BTRFS and LXC

86 points by oellegaard 13y ago

arunoda 13y ago
Where is INSTALL.md and MANUAL.md ?
mateuszf 13y ago
Nice. Seems like a great project to learn bash. Also, I wonder .. how efficient/correct can be the included JSON parser. Anyone has an idea?
- jpetazzo 13y ago
  If you want to know more about the JSON parser, it comes from https://github.com/dominictarr/JSON.sh https://github.com/dominictarr/JSON.sh.
zobzu 13y ago
so docker is mainly a json parser to command lxc i suppose. the bash json parser is the main part of this project - the rests are just wrapping lxc/btrs functions.
- jpetazzo 13y ago
  The JSON parser actually comes from https://github.com/dominictarr/JSON.sh https://github.com/dominictarr/JSON.sh (as mentioned in dockerlite's README file). The JSON parser is not "the main part of this project"; it was included just in case we needed to interact with the registry (but it wasn't deemed necessary).
zrail 13y ago
I don't have anything invested in either project, but I just want to point out that dockerlite's last commit was two months ago and the Docker project has had twelve releases in that same time frame[1]. Docker has moved quite a bit in the last two months and is getting close to production-ready. [1]: https://github.com/dotcloud/docker/releases https://github.com/dotcloud/docker/releases
- rschmitty 13y ago
  Truly production ready or "wait for version 2" production ready? I'm always afraid of 1.0 release stuff for actual production environments, despite who releases it
  - zrail 13y ago
    Well, that depends on your perspective, I guess. I would say "ready to try in a production-like environment".
  - jared314 13y ago
    This is a good point. There are two definitions of "production ready": 1) Works for my application. 2) Won't get my head chopped off. The second is what most developers working inside a hierarchy really want. I am always torn in trying to keep up with new things, because reliability is generally not the focus of new products.
  - darkarmani 13y ago
    I don't know anything about Docker being production ready, but both LXC (used by docker) and OpenVZ are production ready. OpenVZ is a mostly equivalent technology that has been around for a number of years, but IMHO hasn't gotten the management features and updates that docker is getting. You can use OpenVZ right now for similar things, but it isn't as easy to create small single-use containers. The nice thing OpenVZ has that docker doesn't currently support is mounting a Host directory (read-only or read-write) into the OpenVZ container so you can easily share lots of data to many containers with one copy. Right now, docker supports sharing volumes between containers, but not with the Host system. For example, you can make very small OpenVZ containers by using common /usr, /lib, /lib64 directories and mounting them read-only in all of your containers. It's easy to bring up OpenVZ on a Centos6 machine and you can run Ubuntu containers in it if you like. I have nothing against Docker and hope they keep adding features, but my current experience is with OpenVZ.
    - jpetazzo 13y ago
      Actually, Docker just got the "bind mounting" feature that you are requesting. If you want the long and boring details, it's in this pull request: https://github.com/dotcloud/docker/pull/602 https://github.com/dotcloud/docker/pull/602 Dockerlite doesn't have those features, though (keep in mind that dockerlite is just a playground for funky features, nothing else!)
ryandotsmith 13y ago
Both docker and dockerlite belong to Dotcloud's github organization. Perhaps dockerlite is just a proof of concept.
- oellegaard 13y ago
  The way I understand it, its indeed a proof of concept - still cool though :-)
  - jpetazzo 13y ago
    Yes, dockerlite is mainly a proof-of-concept, to experiment with a couple of important things. 1. Use BTRFS instead of AUFS, and see if any specific problem arises, or if we hit any corner case when doing that. 2. Setup the network without using LXC default userland tools, in a race-condition free way. This is not as obvious as it sounds. dockerlite was a success on both accounts. It paved the way for BTRFS support, and gave us some insights about how to make the network setup more flexible.
Ixiaus 13y ago
FreeBSD has had this for quite some time with zfs jails and ezjails.
cypher543 13y ago
Serious question: Is this proper English? > Where the name dockerlite comes from? I keep seeing that sort of arrangement in headings like "How it works?" where I expect it to say "How does it work?" and I'm starting to wonder if that's actually valid English or just a common error.
- klapinat0r 13y ago
  It depends on the phrasing. "This is [how it works]", "How does it work?" I was wondering [where the name dickerlite comes from]? Where does the name dickerlite come from?
  - jameswyse 13y ago
    "I was wondering [where the name dickerlite comes from]? Where does the name dickerlite come from?" I'm also wondering where 'dickerlite' came from..
    - klapinat0r 13y ago
      Freudian slip?
- nkurz 13y ago
  I can't speak to other areas, but it's not standard for American English. To my ear it sounds Slavic.
- ffk 13y ago
  This comes from the dotcloud team, who also makes docker. I believe this particular person speaks French as his main language. Cut him some slack :)
  - cypher543 13y ago
    I wasn't trying to be rude. I was genuinely curious, since I've seen that phrasing so often.
- 13y ago
ffk 13y ago
I believe this was created and presented as simple (contrast to easy) demo to demonstrate BTRFS as a possible backend for images. Docker currently uses AUFS, and was designed to support more fs backends over time. It also demoed a potential path for docker to become more extensible over time. A github issue was created to incorporate BTRFS: https://github.com/dotcloud/docker/issues/443 https://github.com/dotcloud/docker/issues/443 Dotcloud blog post on dockerlite: http://blog.docker.io/2013/05/btrfs-support-for-docker/ http://blog.docker.io/2013/05/btrfs-support-for-docker/ Source: I saw the author's presentation at the docker meetup held at dotcloud a couple months ago.
ballard 13y ago
Docker is great. For those that don't know, it's basically a sandboxing infrastructure for linux running many apps on one physical or virtual server. The future is only having one app per server container. This means mounting a read-only opaque deployment image with an overlay so has a live-cd inspired filesystem (AUFS or dmsetup overlay). Install just one app per a container, do that, and do it well. Combined with an approach of just enough operating system (JEOS), it's possible to have lean-and-mean server instances minus all the default crapware (i.e., bluetooth and pptp). The only feedback is that BTRFS == fsck, and is also sponsored by Oracle, an organization that is killing off the superior, but N.I.H., ZFS. Let's not forget all of their other community fails (Berkeley DB, Java, MySQL, OpenOffice). FreeBSD ZFS and ZFSonLinux are good-to-go.
- derefr 13y ago
  > The only feedback is that BTRFS == fsck, and is also sponsored by Oracle, an organization that is killing off the superior, but N.I.H., ZFS. Let's not forget all of their other community fails (Berkeley DB, Java, MySQL, OpenOffice). Personally I always read the situation as Linux not supporting ZFS because it was born under the Solaris banner, while Btrfs, though being developed "in cooperation with" third parties, is fundamentally a shiny open generic FOSS technology (the name itself eschews branding to make it sound like "look, it's all just B-trees, you can understand this.") Heck, until this conversation, I thought Btrfs was something Linus himself had a stake in, like Git. What wonders a different brand-image can do to people who have incomplete information :)
  - pja 13y ago
    The ZFS sources are licensed in a way that makes it impossible to merge them into the Linux kernel. You could run ZFS as a user space daemon communicating with the kernel over the FUSE interface but it's never going to be part of the kernel itself.
    - otterley 13y ago
      You can also load it as a module with dkms - much better performance than under FUSE.

86 points by oellegaard 13y ago

Where is INSTALL.md and MANUAL.md ?

Nice. Seems like a great project to learn bash. Also, I wonder .. how efficient/correct can be the included JSON parser. Anyone has an idea?

jpetazzo 13y ago
If you want to know more about the JSON parser, it comes from https://github.com/dominictarr/JSON.sh https://github.com/dominictarr/JSON.sh.

zobzu 13y ago

so docker is mainly a json parser to command lxc i suppose. the bash json parser is the main part of this project - the rests are just wrapping lxc/btrs functions.

jpetazzo 13y ago
The JSON parser actually comes from https://github.com/dominictarr/JSON.sh https://github.com/dominictarr/JSON.sh (as mentioned in dockerlite's README file). The JSON parser is not "the main part of this project"; it was included just in case we needed to interact with the registry (but it wasn't deemed necessary).

zrail 13y ago

I don't have anything invested in either project, but I just want to point out that dockerlite's last commit was two months ago and the Docker project has had twelve releases in that same time frame[1]. Docker has moved quite a bit in the last two months and is getting close to production-ready. [1]: https://github.com/dotcloud/docker/releases https://github.com/dotcloud/docker/releases

rschmitty 13y ago
Truly production ready or "wait for version 2" production ready? I'm always afraid of 1.0 release stuff for actual production environments, despite who releases it
- zrail 13y ago
  Well, that depends on your perspective, I guess. I would say "ready to try in a production-like environment".
- jared314 13y ago
  This is a good point. There are two definitions of "production ready": 1) Works for my application. 2) Won't get my head chopped off. The second is what most developers working inside a hierarchy really want. I am always torn in trying to keep up with new things, because reliability is generally not the focus of new products.
- darkarmani 13y ago
  I don't know anything about Docker being production ready, but both LXC (used by docker) and OpenVZ are production ready. OpenVZ is a mostly equivalent technology that has been around for a number of years, but IMHO hasn't gotten the management features and updates that docker is getting. You can use OpenVZ right now for similar things, but it isn't as easy to create small single-use containers. The nice thing OpenVZ has that docker doesn't currently support is mounting a Host directory (read-only or read-write) into the OpenVZ container so you can easily share lots of data to many containers with one copy. Right now, docker supports sharing volumes between containers, but not with the Host system. For example, you can make very small OpenVZ containers by using common /usr, /lib, /lib64 directories and mounting them read-only in all of your containers. It's easy to bring up OpenVZ on a Centos6 machine and you can run Ubuntu containers in it if you like. I have nothing against Docker and hope they keep adding features, but my current experience is with OpenVZ.
  - jpetazzo 13y ago
    Actually, Docker just got the "bind mounting" feature that you are requesting. If you want the long and boring details, it's in this pull request: https://github.com/dotcloud/docker/pull/602 https://github.com/dotcloud/docker/pull/602 Dockerlite doesn't have those features, though (keep in mind that dockerlite is just a playground for funky features, nothing else!)

ryandotsmith 13y ago

Both docker and dockerlite belong to Dotcloud's github organization. Perhaps dockerlite is just a proof of concept.

oellegaard 13y ago
The way I understand it, its indeed a proof of concept - still cool though :-)
- jpetazzo 13y ago
  Yes, dockerlite is mainly a proof-of-concept, to experiment with a couple of important things. 1. Use BTRFS instead of AUFS, and see if any specific problem arises, or if we hit any corner case when doing that. 2. Setup the network without using LXC default userland tools, in a race-condition free way. This is not as obvious as it sounds. dockerlite was a success on both accounts. It paved the way for BTRFS support, and gave us some insights about how to make the network setup more flexible.

Ixiaus 13y ago

FreeBSD has had this for quite some time with zfs jails and ezjails.

cypher543 13y ago

Serious question: Is this proper English? > Where the name dockerlite comes from? I keep seeing that sort of arrangement in headings like "How it works?" where I expect it to say "How does it work?" and I'm starting to wonder if that's actually valid English or just a common error.

klapinat0r 13y ago
It depends on the phrasing. "This is [how it works]", "How does it work?" I was wondering [where the name dickerlite comes from]? Where does the name dickerlite come from?
- jameswyse 13y ago
  "I was wondering [where the name dickerlite comes from]? Where does the name dickerlite come from?" I'm also wondering where 'dickerlite' came from..
  - klapinat0r 13y ago
    Freudian slip?
nkurz 13y ago
I can't speak to other areas, but it's not standard for American English. To my ear it sounds Slavic.
ffk 13y ago
This comes from the dotcloud team, who also makes docker. I believe this particular person speaks French as his main language. Cut him some slack :)
- cypher543 13y ago
  I wasn't trying to be rude. I was genuinely curious, since I've seen that phrasing so often.
13y ago

ffk 13y ago

I believe this was created and presented as simple (contrast to easy) demo to demonstrate BTRFS as a possible backend for images. Docker currently uses AUFS, and was designed to support more fs backends over time. It also demoed a potential path for docker to become more extensible over time. A github issue was created to incorporate BTRFS: https://github.com/dotcloud/docker/issues/443 https://github.com/dotcloud/docker/issues/443 Dotcloud blog post on dockerlite: http://blog.docker.io/2013/05/btrfs-support-for-docker/ http://blog.docker.io/2013/05/btrfs-support-for-docker/ Source: I saw the author's presentation at the docker meetup held at dotcloud a couple months ago.

ballard 13y ago

Docker is great. For those that don't know, it's basically a sandboxing infrastructure for linux running many apps on one physical or virtual server. The future is only having one app per server container. This means mounting a read-only opaque deployment image with an overlay so has a live-cd inspired filesystem (AUFS or dmsetup overlay). Install just one app per a container, do that, and do it well. Combined with an approach of just enough operating system (JEOS), it's possible to have lean-and-mean server instances minus all the default crapware (i.e., bluetooth and pptp). The only feedback is that BTRFS == fsck, and is also sponsored by Oracle, an organization that is killing off the superior, but N.I.H., ZFS. Let's not forget all of their other community fails (Berkeley DB, Java, MySQL, OpenOffice). FreeBSD ZFS and ZFSonLinux are good-to-go.

derefr 13y ago
> The only feedback is that BTRFS == fsck, and is also sponsored by Oracle, an organization that is killing off the superior, but N.I.H., ZFS. Let's not forget all of their other community fails (Berkeley DB, Java, MySQL, OpenOffice). Personally I always read the situation as Linux not supporting ZFS because it was born under the Solaris banner, while Btrfs, though being developed "in cooperation with" third parties, is fundamentally a shiny open generic FOSS technology (the name itself eschews branding to make it sound like "look, it's all just B-trees, you can understand this.") Heck, until this conversation, I thought Btrfs was something Linus himself had a stake in, like Git. What wonders a different brand-image can do to people who have incomplete information :)
- pja 13y ago
  The ZFS sources are licensed in a way that makes it impossible to merge them into the Linux kernel. You could run ZFS as a user space daemon communicating with the kernel over the FUSE interface but it's never going to be part of the kernel itself.
  - otterley 13y ago
    You can also load it as a module with dkms - much better performance than under FUSE.