10 ms·

Fedora CoreOS Out of Preview

121 points by ecliptik 7y ago

usr1106 7y ago
The positive headline hides bad news for those of us who run CoreOS. CoreOS will be out of support soon(?). Migration instructions have been announced, but not written yet... There is a CoreOS fork at https://www.flatcar-linux.org/ https://www.flatcar-linux.org/ promising seamless continuation. I have no experience with or further knowledge about it.
- alexhaber 7y ago
  Are you thinking of Container Linux by CoreOS (The company purchased by Red Hat)? Fedora CoreOS is a continuation of Container Linux. See https://en.wikipedia.org/wiki/Container_Linux#Derivatives https://en.wikipedia.org/wiki/Container_Linux#Derivatives
  - usr1106 7y ago
    Yes, sorry, unprecise terminology. When I say CoreOS I mean CoreOS Container Linux. Fedora CoreOS is marketed to be a continuation. But it's not a seamless one, it requires manual porting. Detailed instructions don't exist yet, but the list is not that short https://github.com/coreos/fedora-coreos-tracker/issues/159 https://github.com/coreos/fedora-coreos-tracker/issues/159. While most people won't be affected by many items, a couple of days are easily burned for understanding, coding and testing. Coding probably the smallest part, but if everything goes without "surprises" in real life, I would be surprised.
    - yebyen 7y ago
      This is generally one of the philosophical differences I see between the RHAT family of distributions and some others (those that I personally favor), that seamless upgrades are generally not well supported from each major version to next. For example, and I'm sure I'm oversimplifying, and there are caveats; for years, I've maintained Debian and Ubuntu machines which I've used mostly staying inside of the bounds of the apt package manager, and since many years I've never had a need to question whether I could "dist-upgrade" from one release to the next. It just works. It is a supported operation, and reliably so. At my current job, we use Amazon Linux, a distro which I understand to be mostly derived from CentOS with Amazon addons, (and I'm open to being corrected if anybody knows better, but...) one of the things we're coping with right now is that Amazon Linux 2 does not have an upgrade path from Amazon Linux 1. We are on the hook to rebuild those machines, it's a fact that has given us pause to seriously re-evaluate whether we want to stay on the same train, or get off and switch to another paradigm broadly. I don't strongly see this as a decision that Amazon made, because I'm fairly sure it's also true that upgrades from eg CentOS 6 to 7 are unsupported. And CentOS apparently got this idea from RedHat Enterprise Linux upstream, which provides this helpful page locked behind paywall[1], but from above the fold you can tell at least one thing without logging in: there are only a limited set of cases where upgrades are supported, even with a paid subscription to RHEL. [1]: https://access.redhat.com/solutions/637583 https://access.redhat.com/solutions/637583
      - emmelaich 7y ago
        I've upgraded multiple Fedora machines from Fedora 19 up to the current Fedora 30 without too much issue. Contrariwise, I've found various issues with Debian and Ubuntu. That said, I rarely do dist upgrades of prod systems at all; I install the base os from scratch (pxeboot etc) and use automation to install the rest.
        d0ugal 7y ago
        I’m the same. I never upgrade cattle, they get replaced and only pets are upgraded.
        deleted 7y ago
        [deleted]
        yebyen 7y ago
        I will concede, Fedora has as good a reputation and it's what I run on my desktop at home! Good point.
      - dralley 7y ago
        Honestly I've had more success migrating between versions of the Fedora family than the Ubuntu family. I don't have enough experience with Debian to judge. In an case I disagree totally with the assertion that Fedora isn't good at upgrades between versions. CentOS 7 upgrade would be tricky to do automatically because of the systemd transition and the fact that any Enterprise user would likely have a ton of services they need to migrate safely.
        hoistbypetard 7y ago
        > In an case I disagree totally with the assertion that Fedora isn't good at upgrades between versions. Same here. I think that changed somewhere around Fedora 15, though. Prior to that it was a minefield. That's probably why it has such a bad reputation among those who've not tried recently. My current desktop workstation started out at Fedora 14 and now sits at 31. It's gone through multiple motherboard and component upgrades over that time, too.
    - jhart99 7y ago
      From what I've seen so far, it is a continuation in name only. Many of the things that made CoreOS nice have disappeared like locksmith, and other stuff has been made incompatible for no good reason, for example dropping docker...
      - cmurf 7y ago
        Fedora has preferred podman for a while, it's not unique to Fedora CoreOS. Much of that has to do with Docker's choices. moby-engine https://koji.fedoraproject.org/koji/packageinfo?packageID=27395 https://koji.fedoraproject.org/koji/packageinfo?packageID=27... docker https://koji.fedoraproject.org/koji/packageinfo?packageID=5145 https://koji.fedoraproject.org/koji/packageinfo?packageID=51... podman https://koji.fedoraproject.org/koji/packageinfo?packageID=26289 https://koji.fedoraproject.org/koji/packageinfo?packageID=26...
      - kaeso 7y ago
        > have disappeared like locksmith Locksmith[0] implementation is tightly coupled to the specific update daemon, so it can't be directly re-used outside of Container Linux or without update-engine[1]. Its logic has been ported over to Zincati[2], which performs reboot management on top of rpm-ostree[3]. [0] https://github.com/coreos/locksmith https://github.com/coreos/locksmith [1] https://github.com/coreos/update_engine https://github.com/coreos/update_engine [2] https://github.com/coreos/zincati https://github.com/coreos/zincati [3] https://github.com/coreos/rpm-ostree https://github.com/coreos/rpm-ostree
      - the_why_of_y 7y ago
        Last I heard Docker still doesn't support cgroups-v2, 4 years after it was merged into Linux. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/Documentation/cgroup-v2.txt?id=v4.5&id2=v4.4 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
    - deogeo 7y ago
      > but if everything goes without "surprises" in real life, I would be surprised. Do you also shave all those, and those only, who do not shave themselves?
- hunta2097 7y ago
  Is there a firm date for this? The original statement was "at least to the end of 2019". My employer uses CoreOS Container Linux extensively. It's really frustrating not knowing when the hammer will drop. I'm not sure Flatcar Linux will make it through our security team. I'm "looking forward" to the migration guide to see how much pain i'm in for!
  - fctorial 7y ago
    And a firecracker is guaranteed to not explode for at least 3 secs after its lit.
  - blixtra 7y ago
    Chris from Kinvolk here. We're happy to talk about what you need on the security front. Some of the folks working on Flatcar Container Linux have a very strong security background; worked on AWS' EC2 security team, do regular pentesting for distributed systems[1], have reported dozens of security issues to upstream projects packaged in Flatcar Container Linux, including the kernel. We've just now started breaking away from the upstream project, and updating packages. Addressing any open security issues is front and center in our efforts. If you have concerns we'd love to hear them. We worked with CoreOS team for years (was our founding project) and they trusted us on the security front. We feel that if you trusted CoreOS and know our team + background, you should have just as much trust in Kinvolk. [1] https://www.youtube.com/watch?v=ze1vgh8sjlE https://www.youtube.com/watch?v=ze1vgh8sjlE
    - hunta2097 7y ago
      Hi Chris, Thanks for the informed response. To be honest I hope Flatcar does well, I notice the Docker version has been updated on your edge release - long overdue from the upstream project! I will let you know if our security team has any issues if and when they look at it. Thanks for giving everyone the option of staying on Container Linux!
      - blixtra 7y ago
        The whole point of CoreOS Container Linux was to deliver a steady stream of security/software updates. We've been eager to update packages for Flatcar Container Linux but have wanted to maintain as much compatibility as possible for as long as possible. Fairly soon, however, we'll be introducing an updated kernel and user space (systemd, Docker, etc.) into the alpha channel. For us, this will mark the point where we feel like we're fully taking the reins from CoreOS and carrying forward the original objectives.
- blixtra 7y ago
  Happy to answer any questions about Flatcar Container Linux (https://www.flatcar-linux.org/ https://www.flatcar-linux.org/). It should be noted that we went a step further than CoreOS by releasing the update service we use to manage updates as an open source project, Nebraska (https://github.com/kinvolk/nebraska https://github.com/kinvolk/nebraska).
Jonnax 7y ago
Should I be using CoreOS for my containers? Generally I've been using either Ubuntu or Alpine as bases. Also it looks to be Red Hat based? Is there a Debian equivalent?
- GordonS 7y ago
  I think CoreOS is more intended for use as a container host - so you'd run CoreOS on a VM or bare metal, and that machine would run your Ubuntu/Alpine containers.
  - Jonnax 7y ago
    Ah I see. I totally missed that. So that's quite interesting. Because on the host I generally run Ubuntu 18.04. Because I know how it works and can ensure it's secured and up to date. Same with someone with familiarity with Redhat/Centos etc. But installing a whole different OS? Like I get it on niche Linux distros where people like packaging their desktop environments whilst pulling in Debian packages for everything else, for example. But what's the advantage here?
    - sekh60 7y ago
      In the OpenStack world you can use the old Fedora atomic and container Linux to automatically spin up k8s clusters using OpenStack heat and OpenStack magnum.
    - GordonS 7y ago
      There are a few advantages. First is minimalism - it's designed to run containers. Next is automatic, atomic updates - layers that comprise the entire OS are "pulled" and updated when the host boots, similar to how updates work with containers. Because updates are atomic, they can also be rolled back. I think there is also built-in support for rolling updates across a fleet of host machines. I find CoreOS a very interesting proposition, but because it's so different from other distros, there is going to be a learning curve.
      - 7y ago
grizzles 7y ago
Is this the best solution if I want a standalone bare metal server that can spin up VMs? I'm looking for the closest analog to for example, an open source digitalocean style platform. Also interested in something that lets me manage CUDA/TF/PyTorch jobs. I know and understand docker and kvm but not kubernetes yet. TBH I find this entire space totally confusing.
- neilalexander 7y ago
  If you want a standalone bare metal server that can spin up full-fat VMs that will live for a long time with some expectation of persistence, you are probably better looking at something like VMware ESXi (free for individual use on a single host), Joyent SmartOS or something based on KVM, Xen etc. If you only want to run lightweight containers for comparably shorter periods of time and worry about data persistence through some other means, then things like Fedora CoreOS, VMware Photon are designed for that purpose.
- BossingAround 7y ago
  > Is this the best solution if I want a standalone bare metal server that can spin up VMs? No; Fedora CoreOS focuses on containers, not VMs. What you want is OpenStack, it seems.
  - jlgaddis 7y ago
    OpenStack? For a standalone bare metal server?
    - Conan_Kudo 7y ago
      No. oVirt makes much more sense for that case: https://ovirt.org/ https://ovirt.org/ OpenStack is a lot more complexity for a single box...
      - BossingAround 7y ago
        To be completely honest, for a single server, why not just use libvirt's commandline utilities?
deleted 7y ago
[deleted]
kim0 7y ago
Still no okd 4.3 though. Waiting
- pepemon 7y ago
  This. I was waiting for new CoreOS just because of being the blocker for OKD release.
  - smarterclayton 7y ago
    Still trying to land the remaining changes to 4.4. It’s working, just a lot of workarounds still waiting to be replaced with real changes. It has definitely taken longer than anticipated to line up all of the impact from Fedora CoreOS cleanup and changes.
- vrutkovs 7y ago
  OKD Preview has been released in November (uses Fedora CoreOS btw) - https://github.com/openshift/okd#getting-started https://github.com/openshift/okd#getting-started
rossmohax 7y ago
Ignition is a killer feature, so much better than cloud-init, glad to see it surving and being actively worked on.

121 points by ecliptik 7y ago

The positive headline hides bad news for those of us who run CoreOS. CoreOS will be out of support soon(?). Migration instructions have been announced, but not written yet... There is a CoreOS fork at https://www.flatcar-linux.org/ https://www.flatcar-linux.org/ promising seamless continuation. I have no experience with or further knowledge about it.

alexhaber 7y ago
Are you thinking of Container Linux by CoreOS (The company purchased by Red Hat)? Fedora CoreOS is a continuation of Container Linux. See https://en.wikipedia.org/wiki/Container_Linux#Derivatives https://en.wikipedia.org/wiki/Container_Linux#Derivatives
- usr1106 7y ago
  Yes, sorry, unprecise terminology. When I say CoreOS I mean CoreOS Container Linux. Fedora CoreOS is marketed to be a continuation. But it's not a seamless one, it requires manual porting. Detailed instructions don't exist yet, but the list is not that short https://github.com/coreos/fedora-coreos-tracker/issues/159 https://github.com/coreos/fedora-coreos-tracker/issues/159. While most people won't be affected by many items, a couple of days are easily burned for understanding, coding and testing. Coding probably the smallest part, but if everything goes without "surprises" in real life, I would be surprised.
  - yebyen 7y ago
    This is generally one of the philosophical differences I see between the RHAT family of distributions and some others (those that I personally favor), that seamless upgrades are generally not well supported from each major version to next. For example, and I'm sure I'm oversimplifying, and there are caveats; for years, I've maintained Debian and Ubuntu machines which I've used mostly staying inside of the bounds of the apt package manager, and since many years I've never had a need to question whether I could "dist-upgrade" from one release to the next. It just works. It is a supported operation, and reliably so. At my current job, we use Amazon Linux, a distro which I understand to be mostly derived from CentOS with Amazon addons, (and I'm open to being corrected if anybody knows better, but...) one of the things we're coping with right now is that Amazon Linux 2 does not have an upgrade path from Amazon Linux 1. We are on the hook to rebuild those machines, it's a fact that has given us pause to seriously re-evaluate whether we want to stay on the same train, or get off and switch to another paradigm broadly. I don't strongly see this as a decision that Amazon made, because I'm fairly sure it's also true that upgrades from eg CentOS 6 to 7 are unsupported. And CentOS apparently got this idea from RedHat Enterprise Linux upstream, which provides this helpful page locked behind paywall[1], but from above the fold you can tell at least one thing without logging in: there are only a limited set of cases where upgrades are supported, even with a paid subscription to RHEL. [1]: https://access.redhat.com/solutions/637583 https://access.redhat.com/solutions/637583
    - emmelaich 7y ago
      I've upgraded multiple Fedora machines from Fedora 19 up to the current Fedora 30 without too much issue. Contrariwise, I've found various issues with Debian and Ubuntu. That said, I rarely do dist upgrades of prod systems at all; I install the base os from scratch (pxeboot etc) and use automation to install the rest.
      - d0ugal 7y ago
        I’m the same. I never upgrade cattle, they get replaced and only pets are upgraded.
        deleted 7y ago
        [deleted]
      - yebyen 7y ago
        I will concede, Fedora has as good a reputation and it's what I run on my desktop at home! Good point.
    - dralley 7y ago
      Honestly I've had more success migrating between versions of the Fedora family than the Ubuntu family. I don't have enough experience with Debian to judge. In an case I disagree totally with the assertion that Fedora isn't good at upgrades between versions. CentOS 7 upgrade would be tricky to do automatically because of the systemd transition and the fact that any Enterprise user would likely have a ton of services they need to migrate safely.
      - hoistbypetard 7y ago
        > In an case I disagree totally with the assertion that Fedora isn't good at upgrades between versions. Same here. I think that changed somewhere around Fedora 15, though. Prior to that it was a minefield. That's probably why it has such a bad reputation among those who've not tried recently. My current desktop workstation started out at Fedora 14 and now sits at 31. It's gone through multiple motherboard and component upgrades over that time, too.
  - jhart99 7y ago
    From what I've seen so far, it is a continuation in name only. Many of the things that made CoreOS nice have disappeared like locksmith, and other stuff has been made incompatible for no good reason, for example dropping docker...
    - cmurf 7y ago
      Fedora has preferred podman for a while, it's not unique to Fedora CoreOS. Much of that has to do with Docker's choices. moby-engine https://koji.fedoraproject.org/koji/packageinfo?packageID=27395 https://koji.fedoraproject.org/koji/packageinfo?packageID=27... docker https://koji.fedoraproject.org/koji/packageinfo?packageID=5145 https://koji.fedoraproject.org/koji/packageinfo?packageID=51... podman https://koji.fedoraproject.org/koji/packageinfo?packageID=26289 https://koji.fedoraproject.org/koji/packageinfo?packageID=26...
    - kaeso 7y ago
      > have disappeared like locksmith Locksmith[0] implementation is tightly coupled to the specific update daemon, so it can't be directly re-used outside of Container Linux or without update-engine[1]. Its logic has been ported over to Zincati[2], which performs reboot management on top of rpm-ostree[3]. [0] https://github.com/coreos/locksmith https://github.com/coreos/locksmith [1] https://github.com/coreos/update_engine https://github.com/coreos/update_engine [2] https://github.com/coreos/zincati https://github.com/coreos/zincati [3] https://github.com/coreos/rpm-ostree https://github.com/coreos/rpm-ostree
    - the_why_of_y 7y ago
      Last I heard Docker still doesn't support cgroups-v2, 4 years after it was merged into Linux. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/Documentation/cgroup-v2.txt?id=v4.5&id2=v4.4 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
  - deogeo 7y ago
    > but if everything goes without "surprises" in real life, I would be surprised. Do you also shave all those, and those only, who do not shave themselves?
hunta2097 7y ago
Is there a firm date for this? The original statement was "at least to the end of 2019". My employer uses CoreOS Container Linux extensively. It's really frustrating not knowing when the hammer will drop. I'm not sure Flatcar Linux will make it through our security team. I'm "looking forward" to the migration guide to see how much pain i'm in for!
- fctorial 7y ago
  And a firecracker is guaranteed to not explode for at least 3 secs after its lit.
- blixtra 7y ago
  Chris from Kinvolk here. We're happy to talk about what you need on the security front. Some of the folks working on Flatcar Container Linux have a very strong security background; worked on AWS' EC2 security team, do regular pentesting for distributed systems[1], have reported dozens of security issues to upstream projects packaged in Flatcar Container Linux, including the kernel. We've just now started breaking away from the upstream project, and updating packages. Addressing any open security issues is front and center in our efforts. If you have concerns we'd love to hear them. We worked with CoreOS team for years (was our founding project) and they trusted us on the security front. We feel that if you trusted CoreOS and know our team + background, you should have just as much trust in Kinvolk. [1] https://www.youtube.com/watch?v=ze1vgh8sjlE https://www.youtube.com/watch?v=ze1vgh8sjlE
  - hunta2097 7y ago
    Hi Chris, Thanks for the informed response. To be honest I hope Flatcar does well, I notice the Docker version has been updated on your edge release - long overdue from the upstream project! I will let you know if our security team has any issues if and when they look at it. Thanks for giving everyone the option of staying on Container Linux!
    - blixtra 7y ago
      The whole point of CoreOS Container Linux was to deliver a steady stream of security/software updates. We've been eager to update packages for Flatcar Container Linux but have wanted to maintain as much compatibility as possible for as long as possible. Fairly soon, however, we'll be introducing an updated kernel and user space (systemd, Docker, etc.) into the alpha channel. For us, this will mark the point where we feel like we're fully taking the reins from CoreOS and carrying forward the original objectives.
blixtra 7y ago
Happy to answer any questions about Flatcar Container Linux (https://www.flatcar-linux.org/ https://www.flatcar-linux.org/). It should be noted that we went a step further than CoreOS by releasing the update service we use to manage updates as an open source project, Nebraska (https://github.com/kinvolk/nebraska https://github.com/kinvolk/nebraska).

Jonnax 7y ago

Should I be using CoreOS for my containers? Generally I've been using either Ubuntu or Alpine as bases. Also it looks to be Red Hat based? Is there a Debian equivalent?

GordonS 7y ago
I think CoreOS is more intended for use as a container host - so you'd run CoreOS on a VM or bare metal, and that machine would run your Ubuntu/Alpine containers.
- Jonnax 7y ago
  Ah I see. I totally missed that. So that's quite interesting. Because on the host I generally run Ubuntu 18.04. Because I know how it works and can ensure it's secured and up to date. Same with someone with familiarity with Redhat/Centos etc. But installing a whole different OS? Like I get it on niche Linux distros where people like packaging their desktop environments whilst pulling in Debian packages for everything else, for example. But what's the advantage here?
  - sekh60 7y ago
    In the OpenStack world you can use the old Fedora atomic and container Linux to automatically spin up k8s clusters using OpenStack heat and OpenStack magnum.
  - GordonS 7y ago
    There are a few advantages. First is minimalism - it's designed to run containers. Next is automatic, atomic updates - layers that comprise the entire OS are "pulled" and updated when the host boots, similar to how updates work with containers. Because updates are atomic, they can also be rolled back. I think there is also built-in support for rolling updates across a fleet of host machines. I find CoreOS a very interesting proposition, but because it's so different from other distros, there is going to be a learning curve.
    - 7y ago

grizzles 7y ago

Is this the best solution if I want a standalone bare metal server that can spin up VMs? I'm looking for the closest analog to for example, an open source digitalocean style platform. Also interested in something that lets me manage CUDA/TF/PyTorch jobs. I know and understand docker and kvm but not kubernetes yet. TBH I find this entire space totally confusing.

neilalexander 7y ago
If you want a standalone bare metal server that can spin up full-fat VMs that will live for a long time with some expectation of persistence, you are probably better looking at something like VMware ESXi (free for individual use on a single host), Joyent SmartOS or something based on KVM, Xen etc. If you only want to run lightweight containers for comparably shorter periods of time and worry about data persistence through some other means, then things like Fedora CoreOS, VMware Photon are designed for that purpose.
BossingAround 7y ago
> Is this the best solution if I want a standalone bare metal server that can spin up VMs? No; Fedora CoreOS focuses on containers, not VMs. What you want is OpenStack, it seems.
- jlgaddis 7y ago
  OpenStack? For a standalone bare metal server?
  - Conan_Kudo 7y ago
    No. oVirt makes much more sense for that case: https://ovirt.org/ https://ovirt.org/ OpenStack is a lot more complexity for a single box...
    - BossingAround 7y ago
      To be completely honest, for a single server, why not just use libvirt's commandline utilities?

deleted 7y ago

[deleted]

kim0 7y ago

Still no okd 4.3 though. Waiting

pepemon 7y ago
This. I was waiting for new CoreOS just because of being the blocker for OKD release.
- smarterclayton 7y ago
  Still trying to land the remaining changes to 4.4. It’s working, just a lot of workarounds still waiting to be replaced with real changes. It has definitely taken longer than anticipated to line up all of the impact from Fedora CoreOS cleanup and changes.
vrutkovs 7y ago
OKD Preview has been released in November (uses Fedora CoreOS btw) - https://github.com/openshift/okd#getting-started https://github.com/openshift/okd#getting-started

rossmohax 7y ago

Ignition is a killer feature, so much better than cloud-init, glad to see it surving and being actively worked on.