14 ms·
Build your own Docker with Linux namespaces, cgroups, and chroot
- osrec 3y agoWhat are the additional features that docker provides over the basic containers in the article?
- thwarted 3y agoThis is mentioned near the top. Docker have features such as layered images, networking, container orchestration, and extensive tooling that make it a powerful and versatile solution for deploying applications.
- dtauzell 3y agoThe article says “Docker have features such as layered images, networking, container orchestration, and extensive tooling that make it a powerful and versatile “
- deleted 3y ago[deleted]
- dhx 3y agoDocker by default also applies a seccomp system call whitelist per [1] and restricts capabilities per [2], amongst numerous other default hardening practices that are applied. If a Docker container really had a need to call the "reboot" system call, this permission could be explicitly added. More complex sandboxing techniques include opening handles for sockets, pipes, files, etc and then hardening seccomp filters on top to prevent any new handles being opened. In this way, some containers can read/write defined files on a volume without having any ability to otherwise interact with file systems such as opening new files (all file system related system calls could be disabled). [1] https://github.com/moby/moby/blob/master/profiles/seccomp/default.json https://github.com/moby/moby/blob/master/profiles/seccomp/de... [2] https://docs.docker.com/engine/security/#linux-kernel-capabilities https://docs.docker.com/engine/security/#linux-kernel-capabi...
- raesene9 3y agoIn addition to the other features that siblings have mentioned, what Docker offers is simplicity, in that you don't need to understand the details of namespaces, capabilities, cgroups etc to get an application running in a container :).
- 8organicbits 3y agoDocumentation, third-party tooling, third-party training, large number of software engineers already experienced, extensive real-world usage and testing, plus the items others have mentioned.
- fulafel 3y agoIt's hard to say as "Docker" as a term or product name has diluted pretty much as they variously use it for various products with various functionalitites, and they don't have anything just called "Docker" any more. There's a lot of virtualization stuff for various operating systems, orchestration stuff, etc. (Note that the article also doesn't to link to any specific product, just to the Docker company front page. For me the first link the front page offers is "Docker Desktop for Linux" which is I guess a virtualization based system)
- exploderate 3y agoDockerfile and "market place" (hub) were the big ones in my opinion. Even though Dockerfile syntax was a mess in the beginning, being able to specify a base and a few commands was a huge improvement in usability. Then running build and push to make your image widely available. Collaboration was so easy compared to the alternatives, Linux jails, jailer, debootstrap, lxc and such.
- v7n 3y agoNeat, was studying just this yesterday for safely exposing some system resources to a WebAssembly module. Good read, but short.
- ghostfoxgod 3y agoThanks and agreed, this one is a bit short because I skipped writing about the namespaces, cgroups and chroot in details and kept it outside into separate articles, Sadly I've seen in past that when I post relatively long articles then those don't do good because of short attention span (and I don't blame anyone for this, this is just the fact that I noticed). Hence, for such large topics I tend to break it down into multiple articles. Sharing the individual links if you want to go through the individual topics in bit more detail. [1] Namespaces: https://akashrajpurohit.com/blog/linux-namespaces-isolating-your-system-for-enhanced-security-and-performance/ https://akashrajpurohit.com/blog/linux-namespaces-isolating-... [2] Cgroups: https://akashrajpurohit.com/blog/linux-control-groups-finetuning-resource-allocation-for-optimal-system-performance/ https://akashrajpurohit.com/blog/linux-control-groups-finetu... [3] Chroot: https://akashrajpurohit.com/blog/how-to-create-a-restricted-environment-with-the-linux-chroot-command/ https://akashrajpurohit.com/blog/how-to-create-a-restricted-...
- mudiadamz 3y agoThe article is like Chat GPT generated
- bestcoder69 3y agoYes, this seems like ChatGPT’s style of writing. Another post has a conclusion that matches ChatGPT’s conclusion style: > Remember, namespaces are a powerful feature that requires careful configuration and management. With proper knowledge and implementation, you can harness the full potential of Linux namespaces to create robust and secure systems. A clue from this post itself is that all the links were added to the intro because GPT won’t intersperse links throughout. e:Softened my language since there’s no way to know, and w/e ChatGPT is smart anyway. Better to judge content on the merits anyway imo
- sim7c00 3y agochatGPT is amazing if you have not mastered the language you are writing in. its what it is for. give it some text, have it rewrite it. that its generated by it doesnt mean any content was produced by it. often ppl just use it to rephrase. imho thats what its for. (hard to tell tho which is which :D)
- fwungy 3y agoLikely generated and then edited.
- YetAnotherNick 3y agoI don't think author even tried running the command himself.
- mudiadamz 3y agoYes that's the problem, Chat GPT isn't always right
- ghostfoxgod 3y agoYep, I do use GPT as one of the tools in my workflow. I write these blogs in markdown locally and have a helper script which takes the raw content and with a prompt it helps me generate a title, summary, Intro and conclusion (personal preference to keep these consistent on all blogs) and proofread the whole raw content for any mistakes (replaces grammarly completely now). Quite happy with this workflow because it helps me publish articles more frequently where I don't have to worry about stuff other than just dumping my thoughts in raw format. Its similar to how I use Astro as a tool to generate static pages from these markdown files to easily deploy on web or TailwindCSS etc etc you get the point. https://media.tenor.com/1PMq-CFZno4AAAAC/avengers-endgame-hulk.gif https://media.tenor.com/1PMq-CFZno4AAAAC/avengers-endgame-hu...
- deleted 3y ago[deleted]
- lastdong 3y agoBrilliant! I know of pre-docker teams that still use this kind of implementation in their systems (with added functionalities). It’s great to see this guide, as some things are deemed more complex than they really are. I guess tooling is what brings an advantage around existing solutions, and made Docker super successful (minus license discussions).
- deleted 3y ago[deleted]
- bottlepalm 3y agoI read this as ‘Build your own Doctor’ and thought the next words were going to be Llama, medical journals and your health records.
- quickthrower2 3y agoIronically you were being an LLM in that moment by predicting the next words.
- bottlepalm 3y agoHah we are all LLMs, free will is an illusion. Ask BrainGPT a question and an answer prints itself out word by word in your head.
- diego_sandoval 3y agoThe fact that containers are such a simple technology always makes me think this: Why is it still necessary to have whole, full-blown OS filesystems inside of our containers, if their purpose is running a single binary? Dependencies/dynamic libraries are decent reason, sure. But wouldn't it make more sense to do things "bottom-up"? i.e. starting from an empty filesystem, and then progressively adding the files that are absolutely necessary for the binary to work, instead of the "top-down" approach, which starts from a complete OS filesystem and then starts removing the things that are not needed?
- compsciphd 3y agoit isnt and very few people actually think that. plenty of "from scratch" images. ergo any image that is a single go binary
- silisili 3y agoNever made a ton of sense to me. Go crosscompiles easily, ship a binary. Because the second you want to do anything involving https, you need certificates, and that's where having a minimal but existent base image starts shining, and mostly goes up from there...
- speedgoose 3y agoOnce you have a hammer… One advantage of software containers is to have this unique interface that is the same whatever is inside the container. Like shipping containers. If it's one single golang binary, or a weird python container running only on specific version of Debian compiled during a blue moon, or some 8GB java enterprise bloatware, it's the same.
- claytongulick 3y agoSee, this is actually my problem with containers. "or a weird python container running only on specific version of Debian compiled during a blue moon" This. I guess I've been fortunate that I'm able to reject software like this from my stack. I know that everyone isn't so lucky. I mostly do nodejs, and have zero need for containers when a simple npm install gets all deps. Or if I need performance, a single go binary. I tried doing the container thing just to understand how it all works and what the hype is about. It seemed needlessly complex and hard to develop/debug. For more complex situations where you need a bunch of interacting programs and services, I prefer stuff like Ansible and VMs, or just manually setting up a base image. I guess it's a "get off my lawn" kind of thing. It seems like containers are used a lot by folks who don't want to learn ops, like how to install and configure postgres, redis, etc... I think that's a mistake, and just pushes the problem onto others who have to support the software in production.
- nigma1337 3y agoReminds me of Bocker[0] [0]: https://github.com/p8952/bocker https://github.com/p8952/bocker
- appleflaxen 3y agoThis looks a bit like Sandstorm, which is freaking awesome.
- dmpanch 3y agoThis topic is perfectly covered in a presentation by Jérôme Petazzoni from Dockercon 2015, a Docker developer, on how containerization works. After watching the video, I began to clearly understand that this is not virtualization at all, as many at first imagine. https://www.youtube.com/watch?v=sK5i-N34im8 https://www.youtube.com/watch?v=sK5i-N34im8
- bewilderbeast 3y agoIt already exists, though these guides are good to learn: LXC/LXD containers.
- superlupo 3y agoIsn't that quite the same as running debootstrap focal ./ubuntu-rootfs http://archive.ubuntu.com/ubuntu/ systemd-nspawn -D ./ubuntu-rootfs ?
- dxxvi 3y agoYes, "isolated and efficient environments" are what I need. How can I do the port mapping, disk mounting with this approach? Could you talk more about it?
- taha_jahangir 3y agoAdd `--mount-proc` to `unshare` to have separated `ps` output.
- tambourine_man 3y agoI don’t know how I feel about the author admitting to using ChatGPT to write the article. The topic is right in my area of interest, but I don’t think want to be reading ChatGPT articles from here on out if I can avoid it.
- LeonenTheDK 3y agoAt least they're up front about it. Personally I don't think I mind some ChatGPT magic if the creator doesn't think they can write better. As long as it was heavily curated and modified, and not just "write an article about manually using linux namespaces..." and copy-pasted into their blog.
- ghostfoxgod 3y agoI'll expand once again, I use GPT to help me write title, summary (for excerpt) Intro and conclusions. I mostly focus on dumping down my thoughts from the things I am learning into a markdown file and use a script to sprinkle GPT magic on it which makes it much better in terms of phrasing things into a short and crisp article format and proofread the content for any mistakes (syntactic as well as semantics). I am exploring Linux myself for this year and hence more focused for content around that these days. Given said that, completely understand your sentiment here, so feel free to skip it, no hard feelings but I'm gonna continue with this workflow till I find something better to improve upon this as well. :)
- tambourine_man 3y agoIt may be something I'll eventually overcome. It may also be unavoidable or even undetectable very shortly. But I find its wishy-washy tone tiring, lacking personality or spark. I'm not entirely sure it's the AI that's bothersome to me. Reading long Wikipedia articles is not much fun either. The committee process washes all the texture from the piece. I think what we eventually seek in writing is character and that's missing.
- deleted 3y ago[deleted]
- deleted 3y ago
- larata_media 3y agoWhile I do like this concept, I’ve always done this simply by compiling all the dependencies into the same folder root as the application. It never needed to be any more complicated than that. Application developers made it complicated by not wanting to do anthything but use a package manager. You can have nginx, Apache, php, Perl, python, etc… siloed into the same root as your application and have multiple instances for every application simply by compiling with the “path” parameter in the configuration.
- nascarsayan 3y ago[dead]
- ghostfoxgod 3y agoThis is great, thanks for sharing!
- qwertox 3y agoI'm confused. I tried `unshare --uts --pid --net --mount --ipc --fork` but it failed due to permissions. `sudo unshare --uts --pid --net --mount --ipc --fork` left me in an environment where I was still in my home directory, able to see all the files and create new files which would persist after exiting. I guess there are many other tutorials which would explain this in depth, but this blog post did not really teach me anything useful about `unshare`.
- ghostfoxgod 3y agoBest place to learn in depth about any command ⇾ https://man7.org/linux/man-pages/man1/unshare.1.html https://man7.org/linux/man-pages/man1/unshare.1.html Of course, it's not feasible to add everything into a single blog, it's upon the reader's curiosity to explore more. PS: I find man pages a lot helpful now, and would recommend the same for others as well.
- jimmar 3y agoI've seen lots of articles like these over the years. And yet, Docker persists. Perhaps working with the Linux internals is not the hard part of building a container ecosystem.
- flaminHotSpeedo 3y agoIt's not the hard part in the simple case, certainly. It's an interesting "hands on learning" style of exercise, although I'm not sure why we need a new article like this to make the rounds every so often
- lopkeny12ko 3y agoThere isn't any discussion of overlayfs here, which is a pretty important component of Docker.
- throwway120385 3y agoAlso no discussion of how networking is accomplished, which is a big gaping hole that ties people to ecosystems like Docker or Podman.
- giobox 3y agoThese articles where someone uses Linux kernel level features to replicate Docker isolation are as old as Docker itself, and in my experience they always miss one of the most critical parts of the Docker ecosystem - the easily hackable and extensible container image format. It's the ease of extension of the container image format that is as much responsible for the popularity of container based architectures as it is clever use of namespaces, cgroups and chroot for "robust isolation, resource management, and security". Without the image format, Docker is way less interesting and you arguably haven't "built your own Docker".
- deleted 3y ago[deleted]
- rewmie 3y ago> (...) they always miss one of the most critical parts of the Docker ecosystem - the easily hackable and extensible container image format. Exactly this. Docker might have a ton of nifty features, but it's killer feature is undoubtedly app packaging and deploying. Features like chroot are already as old as time, but they never became nearly as popular as Docker for a good reason: they don't solve the problem most people need to get out of the way before being able to containerize apps.
- arnaudperalta 3y agoReplace every occurrence of 'Docker' in this article by 'containerd' and it's a match !