10 ms·
Hi I'm a developer advocate in the container engineering org at AWS. I think there are a few misunderstandings here that I may be able to explain better. First
by NathanKP 6y ago
Hi I'm a developer advocate in the container engineering org at AWS. I think there are a few misunderstandings here that I may be able to explain better.
First Bottlerocket is not Amazon Linux 2, it is its own minimal operating system, with most components built from the ground up in Rust. This is totally different than the Amazon Linux 2 you may be familiar with (and most other operating systems for that matter). Bottlerocket is optimized for running containers with high security isolation. The host OS is extremely minimal, it does not come with bash, an interpreter, ssh, or anything beyond the system basics needed to run containers. In fact it uses an immutable root filesystem. You aren't intended to run or install things directly on the host at all.
Everything that is installed and runs on Bottlerocket runs as containers, which are kept isolated from each other and the host with best practice security features such as SELinux. For example you can connect to a container on the host via AWS Systems Manager, or you can optionally enable a container that lets you connect to it via SSH. Once again the thing you are connecting to is the container on the host though, not directly to the host.
For this initial release of Bottlerocket we are focusing on providing image variants that are prepackaged with the requirements to serve as underlying container hosts for an ECS or EKS cluster. However we also intend Bottlerocket to eventually be something that anyone can use, anywhere, even outside of AWS, if they wanted to benefit from the secure by default, container first design of Bottlerocket.
You can read more about the security features of Bottlerocket here: https://github.com/bottlerocket-os/bottlerocket/blob/develop/SECURITY_FEATURES.md https://github.com/bottlerocket-os/bottlerocket/blob/develop...
And you can find a bit more of the charter / goals for the project here: https://github.com/bottlerocket-os/bottlerocket/blob/develop/CHARTER.md https://github.com/bottlerocket-os/bottlerocket/blob/develop...
- mnd999 6y agoSo you saw CoreOS and thought “Let’s re:invent that wheel”?
- 013a 6y agoYou mean, the CoreOS that was end-of-lifed four months ago?
- k__ 6y agoAt least "Fedora CoreOS" still seems to be a thing.
- freedomben 6y agoAnd Red Hat Core OS (RHCOS) for Red Hat flavored shops.
- rosywoozlechan 6y agoRed Hat bought the company that created CoreOS so it's not surprising they have Red Hat/Fedora supported versions right?
- freedomben 6y agoYep, very much agree. Container Linux was still very young and growing, but had some great promise. Incorporating the tech into RHEL and Fedora was a great idea IMHO. I think CoreOS was acquired for that specific purpose personally.
- merb 6y agoI don't think. Fedora CoreOS is more coupled to be an OpenShift node than a plain Container/K8s node. it's sad what they did. And I was a Fedora user for my whole life, but that made me reconsider.
- wmf 6y agoSo did every other vendor. We have two CoreOSes, Flatcar, Container-Optimized OS from Google, Bottlerocket, k3OS, etc. Fortunately these aren't just different in name; there's a lot of experimentation going on around different ways to do updates, security, etc. I hope we'll eventually see some convergence after a few years.
- branzo 6y agoTwo CoreOSes? The one being Fedora CoreOS and the other..?
- Bombthecat 6y agoDead :)
- wmf 6y agoFedora CoreOS and RHEL CoreOS.
- e12e 6y agoWould it be safe to say that bottlerocket is a container host for running containers under a hypervisor? Or is it intended to do full hw interfacing and run bare metal?
- NathanKP 6y agoBottlerocket runs containers using containerd, so containers are visible as processes on the host, from its perspective, not currently isolated from each other via a hypervisor. Bottlerocket limits containers ability to interact with the host or each other via SELinux, among other things. We do have firecracker-containerd (https://github.com/firecracker-microvm/firecracker-containerd https://github.com/firecracker-microvm/firecracker-container...) which is designed to allow the containerd runtime to launch containers as microVMs via Firecracker, with that additional layer of isolation via the KVM hypervisor. This stack is not currently fully compatible with K8s or ECS though, so it is not implemented using that approach yet. Rather Bottlerocket is built as a progressive improvement on the current state of container hosts, which is that many people are running all their containers on their hosts without any strong security hardening at all. I think from the similar naming scheme of Firecracker and Bottlerocket you can already see the pieces of the puzzle that are in progress and the future potential though.
- e12e 6y agoThanks, I was more wondering what the relationship was between something like Debian GNU/linux and bottlerocket. From gp description it sounds like there's no "GNU userland", just the Linux kernel and some utility functions in rust - enough to launch containerd. So if I have a heterogeneous collection of servers - I could install Debian, and run docker on Debian. It sounds like bottlerocket would more comfortably run on top of a hypervisor abstracting away the actual hw a bit? Eg on top of xen, kvm or VMware? Obviously the Linux kernel can be made to run on a toaster, but maybe bottlerocket isn't ideal for that purpose?
- jhaynes 6y agoAnswering your initial question and this one: Bottlerocket today only runs in EC2, but we've tried to make it flexible enough to run outside of a hypervisor on bare metal in the future (in fact, a few engineers on the team are really excited to get it running on their RaspberryPi's at home; toasters haven't been added to our roadmap yet ;) ). Bottlerocket has a GNU userland like many other distros. It is just one that is stripped down of many things including removal of interpreters, shells, and package managers. If you want to explore more deeply, you can enable the admin container and jump into a shell on the host[1] to look at the filesystem and see what Bottlerocket's userspace looks like up close and personal. You can also see a bit more of this debugging/exploration tooling explained in an AWS Partner Blog[2]. [1] https://github.com/bottlerocket-os/bottlerocket#admin-container https://github.com/bottlerocket-os/bottlerocket#admin-contai... [2] https://aws.amazon.com/blogs/apn/getting-started-with-bottlerocket-and-certified-aws-partners/ https://aws.amazon.com/blogs/apn/getting-started-with-bottle...
- azinman2 6y agoI just installed Proxmox on a home server, and I’m using its CT containers (LXC) to run various services. Could I use this as a replacement for Proxmox?
- geoffeg 6y agoI read that this is more intended as the VM you run inside Proxmox, that you then run your (docker/k8) containers inside.
- rubyn00bie 6y agoNope. Proxmox is mostly just an administrative UI, containers are a Linux feature. You don't need Proxmox for anything, you can just run containers "natively" in Linux or VMs with KVM + QEMU. The linked above is mostly just a Linux distro geared towards a pretty specific set of use-cases.
- cle 6y agoI wish the blog post was just this. So much clearer. Thank you.
- simonebrunozzi 6y agoAmazon's PR would have not allowed that.
- srameshc 6y agoIsn't it somewhat similar in concept to the Container-Optimized OS that is being offered by GCP ?
- WrtCdEvrydy 6y agoQuestion: Will this be an AMI that can be deployed as an ECS cluster recipient on EC2? End Question.
- NathanKP 6y agoYes! It already is in fact. The ECS documentation provides the list of Bottlerocket AMI's you can launch into your ECS clusters: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-bottlerocket.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/... I'd also recommend reading the ECS quickstart in the Bottlerocket repo: https://github.com/bottlerocket-os/bottlerocket/blob/develop/QUICKSTART-ECS.md https://github.com/bottlerocket-os/bottlerocket/blob/develop...
- WrtCdEvrydy 6y agoWell... i'll be modifying some launch configurations then.
- robomaker2 6y agolooks like this a beta
- boojing 6y agoDoes Fargate use Bottlerocket?
- mchusma 6y agoI tried to see any performance characteristics of running bottlerocket, just to understand what the expectations are there. I assume since it was not mentioned it is either similar to Amazon Linux 2 or worse (but with security advantages). Can you request a follow on post on the aws blog that describes the performance impact of Bottlerocket? If performance is better, would be nice to know that as well of course.
- robomaker2 6y agoAWS-Linux is so slow because of old packages that the new OS is guaranteed to be faster. It has a recent kernel & containerd. Podman would have made it even faster.
- fnord77 6y agoHi - Does bottlerocket use glibc or something else?
- sanxiyn 6y agoAs you can see here https://github.com/bottlerocket-os/bottlerocket/tree/develop/packages https://github.com/bottlerocket-os/bottlerocket/tree/develop..., Bottlerocket uses glibc.
- hakre 6y agoIn the intended anyone, anywhere (specifically you wrote outside AWS) scenario you describe, is AWS offering professional support for Bottlerocket?
- fsflover 6y ago> Bottlerocket is optimized for running containers with high security isolation. The host OS is extremely minimal, it does not come with bash, an interpreter, ssh, or anything beyond the system basics needed to run containers. In fact it uses an immutable root filesystem. You aren't intended to run or install things directly on the host at all.Bottlerocket is optimized for running containers with high security isolation. The host OS is extremely minimal, it does not come with bash, an interpreter, ssh, or anything beyond the system basics needed to run containers. In fact it uses an immutable root filesystem. You aren't intended to run or install things directly on the host at all. Sounds exactly like Qubes OS [0], but the latter is probably much more secure (based on Xen and VT-d). So in comparison Bottlerocket seems like trading some of security for performance. [0] https://qubes-os.org https://qubes-os.org
- cslarson 6y agoCould bottlerocket be the basis for desktop system where it would be easy to switch between isolated environments? It would be very handy to have a maximally secure environment, at least from software perspective, for certain operations.
- _underfl0w_ 6y agoYou may want to look into Qubes. It's not specifically designed for running containers like this is, but it does provide a single OS with multiple isolated environments for different programs.
- __MatrixMan__ 6y agoAs _underfl0w_ said, you're pretty much describing Qubes. Using it can be eye opening--nothing works together unless you let it, which is a good way to get a feel for your attack surface / unknown dependencies. Qubes uses VM's. Subgraph OS is another option. It tries to achieve the same thing with containers, but their latest release is pretty old--not sure how alive that project is.