5 ms·

Should I be using CoreOS for my containers? Generally I've been using either Ubuntu or Alpine as bases. Also it looks to be Red Hat based? Is there a Debian e

by Jonnax 7y ago

Should I be using CoreOS for my containers? Generally I've been using either Ubuntu or Alpine as bases. Also it looks to be Red Hat based? Is there a Debian equivalent?

GordonS 7y ago
I think CoreOS is more intended for use as a container host - so you'd run CoreOS on a VM or bare metal, and that machine would run your Ubuntu/Alpine containers.
- Jonnax 7y ago
  Ah I see. I totally missed that. So that's quite interesting. Because on the host I generally run Ubuntu 18.04. Because I know how it works and can ensure it's secured and up to date. Same with someone with familiarity with Redhat/Centos etc. But installing a whole different OS? Like I get it on niche Linux distros where people like packaging their desktop environments whilst pulling in Debian packages for everything else, for example. But what's the advantage here?
  - sekh60 7y ago
    In the OpenStack world you can use the old Fedora atomic and container Linux to automatically spin up k8s clusters using OpenStack heat and OpenStack magnum.
  - GordonS 7y ago
    There are a few advantages. First is minimalism - it's designed to run containers. Next is automatic, atomic updates - layers that comprise the entire OS are "pulled" and updated when the host boots, similar to how updates work with containers. Because updates are atomic, they can also be rolled back. I think there is also built-in support for rolling updates across a fleet of host machines. I find CoreOS a very interesting proposition, but because it's so different from other distros, there is going to be a learning curve.
    - GordonS 7y ago
      Oh, I forgot one! The OS is read-only while the host is up, which is a big security and stability benefit.
      - usr1106 7y ago
        Fedora CoreOS is read-only? CoreOS Container Linux is not. But changes are not persistent over redeployment. That's nice for quick testing and development, but of course fatal should you do it in a production system. And of course not ideal for security. Disclaimer: Edited for false claims. Hope, it's correct now...
        robszumski 7y ago
        Go look at /usr on Container Linux, where the actual OS lives. It’s literally mounted R/O. Binaries are symlinked from there, so it’s impossible to modify OpenSSL, etc.
        fulafel 7y ago
        How is root prevented from modifying it? (Readonly mount alone is trivial to bypass)
        robszumski 7y ago
        The entire partition is signed so the machine will detect it
        pa7ch 7y ago
        dm-verity mainly https://coreos.com/blog/verifying-os-at-runtime.html https://coreos.com/blog/verifying-os-at-runtime.html
        cmurf 7y ago
        Bit more on dm-verity as it relates to appliance and non-appliance systems. https://blog.verbum.org/2017/06/12/on-dm-verity-and-operating-systems/ https://blog.verbum.org/2017/06/12/on-dm-verity-and-operatin... Btrfs will support cryptographically secure hash algorithms (sha256 and blake2) starting with kernel 5.5. I wonder if this is a suitable compromise over dm-verity, which can't be updated, and more conventional file system options? When I consider various use cases, Btrfs always results in EIO rather than propagating bad data to user space; compress=zstd:1 reduces writes, saves storage space, and can improve performance of slower storage. There's also two interesting read-only options: read-only snapshots and read-only volume (via the seed flag), root can't write to either. Root would need to unset the flag first. A read-only seed can support writes via a volatile 2nd device, e.g. /dev/zram device, reboot and you get a reset. Or persistence via a partition. Either way a reset also resets filesystem state.
zozbot234 7y ago
The old Emdebian project was an attempt to provide an "atomic", "immutable" model for Debian, usually for embedded workloads as opposed to server container hosting, but that's practically dead and I'm not sure that a modern equivalent is out there.
blixtra 7y ago
Just to reiterate what others have said, because this comes up quite a bit. You should not use CoreOS or Flatcar images as the base for your containers. They are intended to be the host operating system upon which you run the containers. Their key features that make them an awesome host OS for containers (no package manager, for example) make them unsuitable for use as the base image of containers.

by Jonnax 7y ago

Should I be using CoreOS for my containers? Generally I've been using either Ubuntu or Alpine as bases. Also it looks to be Red Hat based? Is there a Debian equivalent?

GordonS 7y ago

I think CoreOS is more intended for use as a container host - so you'd run CoreOS on a VM or bare metal, and that machine would run your Ubuntu/Alpine containers.

Jonnax 7y ago
Ah I see. I totally missed that. So that's quite interesting. Because on the host I generally run Ubuntu 18.04. Because I know how it works and can ensure it's secured and up to date. Same with someone with familiarity with Redhat/Centos etc. But installing a whole different OS? Like I get it on niche Linux distros where people like packaging their desktop environments whilst pulling in Debian packages for everything else, for example. But what's the advantage here?
- sekh60 7y ago
  In the OpenStack world you can use the old Fedora atomic and container Linux to automatically spin up k8s clusters using OpenStack heat and OpenStack magnum.
- GordonS 7y ago
  There are a few advantages. First is minimalism - it's designed to run containers. Next is automatic, atomic updates - layers that comprise the entire OS are "pulled" and updated when the host boots, similar to how updates work with containers. Because updates are atomic, they can also be rolled back. I think there is also built-in support for rolling updates across a fleet of host machines. I find CoreOS a very interesting proposition, but because it's so different from other distros, there is going to be a learning curve.
  - GordonS 7y ago
    Oh, I forgot one! The OS is read-only while the host is up, which is a big security and stability benefit.
    - usr1106 7y ago
      Fedora CoreOS is read-only? CoreOS Container Linux is not. But changes are not persistent over redeployment. That's nice for quick testing and development, but of course fatal should you do it in a production system. And of course not ideal for security. Disclaimer: Edited for false claims. Hope, it's correct now...
      - robszumski 7y ago
        Go look at /usr on Container Linux, where the actual OS lives. It’s literally mounted R/O. Binaries are symlinked from there, so it’s impossible to modify OpenSSL, etc.
        fulafel 7y ago
        How is root prevented from modifying it? (Readonly mount alone is trivial to bypass)
        robszumski 7y ago
        The entire partition is signed so the machine will detect it
        pa7ch 7y ago
        dm-verity mainly https://coreos.com/blog/verifying-os-at-runtime.html https://coreos.com/blog/verifying-os-at-runtime.html
        cmurf 7y ago
        Bit more on dm-verity as it relates to appliance and non-appliance systems. https://blog.verbum.org/2017/06/12/on-dm-verity-and-operating-systems/ https://blog.verbum.org/2017/06/12/on-dm-verity-and-operatin... Btrfs will support cryptographically secure hash algorithms (sha256 and blake2) starting with kernel 5.5. I wonder if this is a suitable compromise over dm-verity, which can't be updated, and more conventional file system options? When I consider various use cases, Btrfs always results in EIO rather than propagating bad data to user space; compress=zstd:1 reduces writes, saves storage space, and can improve performance of slower storage. There's also two interesting read-only options: read-only snapshots and read-only volume (via the seed flag), root can't write to either. Root would need to unset the flag first. A read-only seed can support writes via a volatile 2nd device, e.g. /dev/zram device, reboot and you get a reset. Or persistence via a partition. Either way a reset also resets filesystem state.

zozbot234 7y ago

The old Emdebian project was an attempt to provide an "atomic", "immutable" model for Debian, usually for embedded workloads as opposed to server container hosting, but that's practically dead and I'm not sure that a modern equivalent is out there.

blixtra 7y ago

Just to reiterate what others have said, because this comes up quite a bit. You should not use CoreOS or Flatcar images as the base for your containers. They are intended to be the host operating system upon which you run the containers. Their key features that make them an awesome host OS for containers (no package manager, for example) make them unsuitable for use as the base image of containers.