11 ms·

A Workshop on Linux Containers: Rebuild Docker from Scratch

568 points by mastabadtomm 9y ago

baconomatic 9y ago
I've been looking for something like this for a while now. I find building things from the bottom up is a great way to learn how things work (naturally).
- mandeepj 9y ago
  Agree 100% on this. Genuine ask - Could you please share what you have been building?
- arca_vorago 9y ago
  Also taking things apart is a great way to know how to start a bottom up build.
markbnj 9y ago
There is also http://chimeracoder.github.io/docker-without-docker/#1 http://chimeracoder.github.io/docker-without-docker/#1
hardwaresofton 9y ago
In the same vein: Building a container from scratch in Go (Liz Rice) @ Container Camp 2016 -> https://www.youtube.com/watch?v=Utf-A4rODH8 https://www.youtube.com/watch?v=Utf-A4rODH8 What's a container really let's write one in go from sctach (Liz Rice) @ Golang UK Conference -> https://www.youtube.com/watch?v=HPuvDm8IC-4 https://www.youtube.com/watch?v=HPuvDm8IC-4 Cgroups, Namespaces and beyond: What are containers made from (Jerome Petazzoni) @ DockerCon 2015 - https://www.youtube.com/watch?v=sK5i-N34im8 https://www.youtube.com/watch?v=sK5i-N34im8 Building Containers in Pure Bash and C (Jessica Frazelle) @ ContainerSummit 2016: https://containersummit.io/events/nyc-2016/videos/building-containers-in-pure-bash-and-c https://containersummit.io/events/nyc-2016/videos/building-c... First two are basically the same talk, but it doesn't hurt to hear the same ideas more than once.
- gt5050 9y ago
  I found this introduction to Linux namespaces very good https://medium.com/@teddyking/linux-namespaces-850489d3ccf https://medium.com/@teddyking/linux-namespaces-850489d3ccf
- tyingq 9y ago
  Docker in ~100 lines of bash: https://github.com/p8952/bocker https://github.com/p8952/bocker
- zerr 9y ago
  So basically it is just the usage of particular API the underlying OS provides.
  - DannyB2 9y ago
    A container is launching a process in a way that the kernel lets it believe it can be root, have any illusion of a filesystem and network connectivity that you want it to believe, and be fooled into thinking it is the only process running on this kernel. Therefore, it is as light weight as launching a process? Is that about right?
    - eftychis 9y ago
      Pretty much. Usually you run your own init that starts the desired process. If you as a process have id (pid) 1 then you need to manage orphaned processes (and zombies) and general signal handling (e.g. SIGTERM). There is a lot of cgroups and capabilities tweaking involved also. You can also use bpf (e.g. seccomp-bpf to restrict system calls etc. In general you i) add the illusion of king in your little own world ii) provide necessary handling that the kernel expects from a king iii) restrict resources (e.g. memory, system calls,...) and facilitate communication with the rest of the worlds.
      - zenlikethat 9y ago
        There is also now a `--init` flag for docker run and for dockerd that will run a Pretty Reasonable PID1 (tini) automatically.
kompiuter 9y ago
Great, something to pass the time tonight. Many thanks!
acobster 9y ago
I read the OCI runtime/container specs when I was first starting to move all my work projects' dev environments to docker (Lando, to be precise - in case you haven't used it, it's awesome!). That was very illuminating, but there was a lot to it that I didn't understand because of the relatively dense language. Wish I'd had this! Thanks for posting.
- cyphar 9y ago
  Yeah, we really could've done a better job with explaining how to write your own configurations. I tried to make the conversion from images to runtime specified in some manner[1], but really the trade-off between granularity and ease-of-use was tipped to the "extreme granularity" side of the scale. Though if you're writing low-level container configurations you really should already know enough to write a (simple) container runtime yourself (because the security pitfalls with a slightly misconfigured container can be pretty bad -- even though runc does quite a lot of things to keep you safe that are outside of the spec). [1]: https://github.com/opencontainers/image-spec/blob/v1.0.0/conversion.md https://github.com/opencontainers/image-spec/blob/v1.0.0/con...
amq 9y ago
How would you describe a linux container in one sentence? I'm working with them for some years already, but I'm still struggling with a consceise description aimed for e.g. fresh CS undergrads.
- kiliankoe 9y ago
  I've tried through a joke: > This doesn't build. > Well it works on _my_ machine. > Fine, then we ship _your_ machine.
- cyphar 9y ago
  "A Linux container is just a process that runs in a context where it thinks it exists in isolation, usually with some resource limits applied." If you wanted the more "fundamental" explanation, rather than a "how to use them" explanation.
- kilburn 9y ago
  "Container" means two related but different things nowadays: (1) an abstraction to group some processes and give them their own pid, fs, and network spaces; and (2) an archive that contains some application(s) and everything that's required to run them (minus the kernel), plus metadata on what it is and what it requires from the host kernel to actually run.
  - voidlogic 9y ago
    A container has to be 1, but can be 2
- erikb 9y ago
  There is the idea of "high level concepts", concepts that build on top of other concepts. So it's probably impossible to explain without some insight knowledge into operating systems, programming, multi processing, security, packaging, CS history, etc. Explaining it to undergrads would therefore require a "model", i.e. an abstraction of the reality that simplifies what is really there, but is still apllicable to make predictions about outcomes in the real world. So I would start with analysing how I would explain unix user management to a newby who didn't learn filesystems yet. Then transfer some of that thinking to processes instead of files. E.g. this process doing X and that process doing Y shouldn't conflict with each other but might get problems when they both try Z. Therefore they need to be separated in an abstract way. And that's what namespaces are for. yadda yadda. Another approach might be taking ideas from explaining virtual memory to newbies. I.e. each process has their own virtual filesystem, their own virtual network, and the underlying operating system will figure out how to make things work out without conflicts (if possible).
simonw 9y ago
I like how this includes a very simple, readable C extension for Python that exposes some missing syscalls: https://github.com/Fewbytes/rubber-docker/blob/master/linux.c https://github.com/Fewbytes/rubber-docker/blob/master/linux....
infodroid 9y ago
A similar project is 'mocker', a "crappy imitation of Docker, written in 100% Python": https://github.com/tonybaloney/mocker https://github.com/tonybaloney/mocker
- tzakrajs 9y ago
  > I keep hearing statements like "Docker is basically > just cgroups", "Docker is just chroot on steroids", > which is totally incorrect. But this project basically proves that the characterization is pretty accurate.
  - erikb 9y ago
    go back to the top level of this discussion and ctrl-f for "runc" then it becomes more clear. Docker is more than we usually think it is, because it makes all the stuff around containers so easy to use.
    - hodgesrm 9y ago
      Agree. I'm working through the rubber docker lab now. The fact that docker (a) handles all the details of cgroups, overlays, namespaces; (b) is super easy to use; and (c) runs cross-platform is impressive.
derefr 9y ago
Or rather, rebuild runc from scratch. These days, Docker isn't really "about" the running of containers (all the logic for which is encapsulated outside of the Docker project itself, in runc.) Docker—the thing that Docker Inc produces—is the tooling that gets container images fed into a daemonized instance of runc. So: the Docker Registry daemon (the thing you can push/pull images to/from, such as runs on Docker Hub); the `Dockerfile` format, and the build logic that uses it, and the CI bots that use that build logic; the local daemon that holds a mini-registry that builds and pulls write to; and the tooling to move and create and dump images between all those places. If there's a tutorial that replicates that stuff, I'd love to read it.
- dozzie 9y ago
  Do you realize that "that stuff" is merely a binary packages system?
  - twblalock 9y ago
    People use Docker because it's easy to use and easy to get support, and it's easy to hire people who have experience with it. Similarly, you could roll your own Dropbox solution with rsync, but good luck teaching your Mom how to do that when she calls you on the phone and she just wants to know how to sync her photos. By the way, it's pretty difficult to write a binary package system that is reliable, distributed, easy to use, easy to troubleshoot, well supported, and easy to sell to both the engineers and the execs at a company. Docker has done that. It's not a small thing.
  - zbentley 9y ago
    Eh, if it's a binary package system it's a very nice one--perhaps an historically nice one. Unlike most package systems, most people that develop in docker containers are writing the equivalent of a package manifest (Dockerfile), and they're doing it for things that weren't previously packaged. While there are plenty of exceptions, the standard for deployment of webapps has long been piecemeal deployments; not RPMs or whatnot. Docker's convenience features changed that. Also, the layering system/virtual filesystems are integrated into Docker such that it yields a ton of convenience (in "extending" others images, speeding up deployments/caching, and making huge/arbitrary changes to the filesystem highly reversible) without making users manually manage most of it. There are times when the abstraction leaks (looking at you, hard limit on number of layers), and all of the component technologies that went into it existed before, but as I've posted elsewhere in the comments here, the big advantage of docker is that it integrates those technologies in a way that provides a simple mental model for reasoning about containers, and a very convenient/beginner-accessible toolset for packaging really complex dependencies. Even the nicest "typical" packaging tools out there (rpm/dpkg are pretty crufty, but stuff like Nix and FPM are getting quite nice) still have much rougher edges.

568 points by mastabadtomm 9y ago

I've been looking for something like this for a while now. I find building things from the bottom up is a great way to learn how things work (naturally).

mandeepj 9y ago
Agree 100% on this. Genuine ask - Could you please share what you have been building?
arca_vorago 9y ago
Also taking things apart is a great way to know how to start a bottom up build.

markbnj 9y ago

There is also http://chimeracoder.github.io/docker-without-docker/#1 http://chimeracoder.github.io/docker-without-docker/#1

hardwaresofton 9y ago

In the same vein: Building a container from scratch in Go (Liz Rice) @ Container Camp 2016 -> https://www.youtube.com/watch?v=Utf-A4rODH8 https://www.youtube.com/watch?v=Utf-A4rODH8 What's a container really let's write one in go from sctach (Liz Rice) @ Golang UK Conference -> https://www.youtube.com/watch?v=HPuvDm8IC-4 https://www.youtube.com/watch?v=HPuvDm8IC-4 Cgroups, Namespaces and beyond: What are containers made from (Jerome Petazzoni) @ DockerCon 2015 - https://www.youtube.com/watch?v=sK5i-N34im8 https://www.youtube.com/watch?v=sK5i-N34im8 Building Containers in Pure Bash and C (Jessica Frazelle) @ ContainerSummit 2016: https://containersummit.io/events/nyc-2016/videos/building-containers-in-pure-bash-and-c https://containersummit.io/events/nyc-2016/videos/building-c... First two are basically the same talk, but it doesn't hurt to hear the same ideas more than once.

gt5050 9y ago
I found this introduction to Linux namespaces very good https://medium.com/@teddyking/linux-namespaces-850489d3ccf https://medium.com/@teddyking/linux-namespaces-850489d3ccf
tyingq 9y ago
Docker in ~100 lines of bash: https://github.com/p8952/bocker https://github.com/p8952/bocker
zerr 9y ago
So basically it is just the usage of particular API the underlying OS provides.
- DannyB2 9y ago
  A container is launching a process in a way that the kernel lets it believe it can be root, have any illusion of a filesystem and network connectivity that you want it to believe, and be fooled into thinking it is the only process running on this kernel. Therefore, it is as light weight as launching a process? Is that about right?
  - eftychis 9y ago
    Pretty much. Usually you run your own init that starts the desired process. If you as a process have id (pid) 1 then you need to manage orphaned processes (and zombies) and general signal handling (e.g. SIGTERM). There is a lot of cgroups and capabilities tweaking involved also. You can also use bpf (e.g. seccomp-bpf to restrict system calls etc. In general you i) add the illusion of king in your little own world ii) provide necessary handling that the kernel expects from a king iii) restrict resources (e.g. memory, system calls,...) and facilitate communication with the rest of the worlds.
    - zenlikethat 9y ago
      There is also now a `--init` flag for docker run and for dockerd that will run a Pretty Reasonable PID1 (tini) automatically.

kompiuter 9y ago

Great, something to pass the time tonight. Many thanks!

acobster 9y ago

I read the OCI runtime/container specs when I was first starting to move all my work projects' dev environments to docker (Lando, to be precise - in case you haven't used it, it's awesome!). That was very illuminating, but there was a lot to it that I didn't understand because of the relatively dense language. Wish I'd had this! Thanks for posting.

cyphar 9y ago
Yeah, we really could've done a better job with explaining how to write your own configurations. I tried to make the conversion from images to runtime specified in some manner[1], but really the trade-off between granularity and ease-of-use was tipped to the "extreme granularity" side of the scale. Though if you're writing low-level container configurations you really should already know enough to write a (simple) container runtime yourself (because the security pitfalls with a slightly misconfigured container can be pretty bad -- even though runc does quite a lot of things to keep you safe that are outside of the spec). [1]: https://github.com/opencontainers/image-spec/blob/v1.0.0/conversion.md https://github.com/opencontainers/image-spec/blob/v1.0.0/con...

amq 9y ago

How would you describe a linux container in one sentence? I'm working with them for some years already, but I'm still struggling with a consceise description aimed for e.g. fresh CS undergrads.

kiliankoe 9y ago
I've tried through a joke: > This doesn't build. > Well it works on _my_ machine. > Fine, then we ship _your_ machine.
cyphar 9y ago
"A Linux container is just a process that runs in a context where it thinks it exists in isolation, usually with some resource limits applied." If you wanted the more "fundamental" explanation, rather than a "how to use them" explanation.
kilburn 9y ago
"Container" means two related but different things nowadays: (1) an abstraction to group some processes and give them their own pid, fs, and network spaces; and (2) an archive that contains some application(s) and everything that's required to run them (minus the kernel), plus metadata on what it is and what it requires from the host kernel to actually run.
- voidlogic 9y ago
  A container has to be 1, but can be 2
erikb 9y ago
There is the idea of "high level concepts", concepts that build on top of other concepts. So it's probably impossible to explain without some insight knowledge into operating systems, programming, multi processing, security, packaging, CS history, etc. Explaining it to undergrads would therefore require a "model", i.e. an abstraction of the reality that simplifies what is really there, but is still apllicable to make predictions about outcomes in the real world. So I would start with analysing how I would explain unix user management to a newby who didn't learn filesystems yet. Then transfer some of that thinking to processes instead of files. E.g. this process doing X and that process doing Y shouldn't conflict with each other but might get problems when they both try Z. Therefore they need to be separated in an abstract way. And that's what namespaces are for. yadda yadda. Another approach might be taking ideas from explaining virtual memory to newbies. I.e. each process has their own virtual filesystem, their own virtual network, and the underlying operating system will figure out how to make things work out without conflicts (if possible).

simonw 9y ago

I like how this includes a very simple, readable C extension for Python that exposes some missing syscalls: https://github.com/Fewbytes/rubber-docker/blob/master/linux.c https://github.com/Fewbytes/rubber-docker/blob/master/linux....

infodroid 9y ago

A similar project is 'mocker', a "crappy imitation of Docker, written in 100% Python": https://github.com/tonybaloney/mocker https://github.com/tonybaloney/mocker

tzakrajs 9y ago
> I keep hearing statements like "Docker is basically > just cgroups", "Docker is just chroot on steroids", > which is totally incorrect. But this project basically proves that the characterization is pretty accurate.
- erikb 9y ago
  go back to the top level of this discussion and ctrl-f for "runc" then it becomes more clear. Docker is more than we usually think it is, because it makes all the stuff around containers so easy to use.
  - hodgesrm 9y ago
    Agree. I'm working through the rubber docker lab now. The fact that docker (a) handles all the details of cgroups, overlays, namespaces; (b) is super easy to use; and (c) runs cross-platform is impressive.

derefr 9y ago

Or rather, rebuild runc from scratch. These days, Docker isn't really "about" the running of containers (all the logic for which is encapsulated outside of the Docker project itself, in runc.) Docker—the thing that Docker Inc produces—is the tooling that gets container images fed into a daemonized instance of runc. So: the Docker Registry daemon (the thing you can push/pull images to/from, such as runs on Docker Hub); the `Dockerfile` format, and the build logic that uses it, and the CI bots that use that build logic; the local daemon that holds a mini-registry that builds and pulls write to; and the tooling to move and create and dump images between all those places. If there's a tutorial that replicates that stuff, I'd love to read it.

dozzie 9y ago
Do you realize that "that stuff" is merely a binary packages system?
- twblalock 9y ago
  People use Docker because it's easy to use and easy to get support, and it's easy to hire people who have experience with it. Similarly, you could roll your own Dropbox solution with rsync, but good luck teaching your Mom how to do that when she calls you on the phone and she just wants to know how to sync her photos. By the way, it's pretty difficult to write a binary package system that is reliable, distributed, easy to use, easy to troubleshoot, well supported, and easy to sell to both the engineers and the execs at a company. Docker has done that. It's not a small thing.
- zbentley 9y ago
  Eh, if it's a binary package system it's a very nice one--perhaps an historically nice one. Unlike most package systems, most people that develop in docker containers are writing the equivalent of a package manifest (Dockerfile), and they're doing it for things that weren't previously packaged. While there are plenty of exceptions, the standard for deployment of webapps has long been piecemeal deployments; not RPMs or whatnot. Docker's convenience features changed that. Also, the layering system/virtual filesystems are integrated into Docker such that it yields a ton of convenience (in "extending" others images, speeding up deployments/caching, and making huge/arbitrary changes to the filesystem highly reversible) without making users manually manage most of it. There are times when the abstraction leaks (looking at you, hard limit on number of layers), and all of the component technologies that went into it existed before, but as I've posted elsewhere in the comments here, the big advantage of docker is that it integrates those technologies in a way that provides a simple mental model for reasoning about containers, and a very convenient/beginner-accessible toolset for packaging really complex dependencies. Even the nicest "typical" packaging tools out there (rpm/dpkg are pretty crufty, but stuff like Nix and FPM are getting quite nice) still have much rougher edges.