5 ms·

I think there was a very large knee-jerk reaction to the CentOS announcement, myself included. I immediately began looking into alternatives, completely discoun

I think there was a very large knee-jerk reaction to the CentOS announcement, myself included. I immediately began looking into alternatives, completely discounting Stream as a viable production OS, a perception shared by many commentators around the web. But then I actually tried out Stream, and thought deeper about it. I don’t really need to exactly match RHEL’s point releases. I don’t need 100% binary compatibility with RHEL. My installs don’t break between point releases or errata updates, why would a rolling release change that? I just need a rock-solid OS to run my applications on, and Stream seems to fill that need. Just ‘cause the update model was changed doesn’t mean the OS is unstable. I agree with the authors claim that 95% of CentOS users will be perfectly happy with Stream. Of course, there is that remaining 5% gap, where I understand the frustration of those in that situation. I already roll several other Red Hat upstream projects in production, including AWX (Ansible Tower) and oVirt (RHV). I’m sure RH would love for me to pay for support, but I don’t really need it. The upstreams serve me well enough. The one thing I can’t get over though, is the decision to roll out this change in the middle of CentOS 8’s lifecycle, trimming out what, 8 years or so of support. That seriously stung. This decision should have been made a year or two ago, before CentOS 8 was released.

tw04 6y ago
Based on my experience working for a hardware vendor and now in another role where I work with clients of all sizes in the enterprise, I think saying this works for 95% of users is an intentionally gross mischaracterization of the user base. Literally every enterprise user I work with that has CentOS uses it specifically because it directly tracks RHEL. They don't want to pay for RHEL licenses in test/dev but need a binary compatible version to ensure they can get enterprise support in production should they hit an issue. Having different drivers/kernels/libraries is a non-starter. There's absolutely no way that Redhat is so blind to their user base that they don't know this is happening. This is a pure money grab, if they can get even 5% of the users doing what I've seen paying for licenses, it's a net-win for them. I'm 100% onboard with Redhat doing things to make lives difficult for companies like Oracle that are trying to make money off their hard work. I have no time for user-hostile moves like this, every justification they've given is hollow IMO which is why they're still trying to do damage control and persuade the public that it's not what it looks like.
- invokestatic 6y ago
  Yes, I think that use case falls in the 5% gap that had the rug pulled under them. But I really do believe 95% of CentOS users don’t really need that, hosting stuff like HTTP servers or containers. Stuff that really doesn’t need to match RHEL.
  - NathanOsullivan 6y ago
    Web hosters and the like need stability and don't really care about the underlying distribution at all. Those types of users will leave CentOS and never look back.
- lmeyerov 6y ago
  Bingo. RHEL is a subscription mess when working across its zoo ("Is our dev mirror the Azure's managed RHEL repository with mismatching kernel headers? Or pass through to RHEL's with epol disabled? Or RHEL 8 with docker repos disconnected so they can push their podman alternative?"). Losing centos hurts. As soon as you get into basics like drivers or build tools, RHEL quickly goes off the rails, and centos gave a semi-predictable escape hatch. GPU land is directly on this path, so I feel awful for enterprise IT admins we work with who just want to do a good job for their client hospitals, schools, businesses, etc. We help shield them most of the way by reducing the install to just a gpu driver + docker, but those deps still need basic pinned version ranges. RHEL makes getting those hard, and worse, as life happens, make it hard to get the right diagnostic tool versions (gcc 7, ...). It's a bad look for a 2020 infra vendor to charge an enterprise for reliable and predictable rollout sw/support when RHEL makes it a marathon to get the right major release of base layers like docker/gcc/drivers. Contrast this to our cloud users, who one-click launch, or our self-managed cloud/ubuntu/etc. ones, who can generally get the right drivers + docker stack up in 10-15 provided commands. Watching the chaos spread into centos makes me even more nervous about critical software relying on RHEL, and we already had to have The Talk with enterprises wanting HPC/ML on RHEL before this..
- freedomben 6y ago
  You're saying it's a "gross mischaracterization." Does that mean you have more/better data than Red Hat/CentOS about CentOS usage? > Literally every enterprise user I work with that has CentOS uses it specifically because it directly tracks RHEL. They don't want to pay for RHEL licenses in test/dev but need a binary compatible version to ensure they can get enterprise support in production should they hit an issue. Having different drivers/kernels/libraries is a non-starter. Unless you are doing kernelly things then the small difference in kernel version will not be that big a deal. Also for most applications, dev/test is better on Stream since it represent RHEL next. If the next version of RHEL will break your app, you find out in dev/test instead of in production like you used to. I think it's worth considering if this isn't better for your use case than previously. It's also not very hard to compile your own kernel if you really need an exact version. I do that sometimes on Fedora when I'm testing a driver I maintain. You can also just keep the RPMs from either previous releases as well (that's not a bad idea so even if you're building your own kernel you can extract the config file from it).
indolering 6y ago
> I don’t really need to exactly match RHEL’s point releases. I don’t need 100% binary compatibility with RHEL. That's good, because regular old CentOS is not bit-for-bit identical either. CentOS might strip out all the RHEL trademarks, but they had to reverse engineer RHEL code drops just like Oracle, Amazon, Google, Facebook, SuSE, etc. > The upstreams serve me well enough. What you don't appreciate is that the distinction between upstream/downstream is a broken metaphor across small time scales. Bug fixes and backports will show up in any number of RHEL clones from Oracle|Amazon|Facebook|Google|Baidu|Samsung before making their way into Fedora|CentOS and finally into RHEL. In the future, RHEL will release their own bugfixes to subscription customers before those fixes find their way into CentOS Stream. That's the meat grinder of a Linux distribution: uptime "guarantees" are just refunds you get to cash in when a large devops team can't fix a problem fast enough. > The one thing I can’t get over though, is the decision to roll out this change in the middle of CentOS 8’s lifecycle, trimming out what, 8 years or so of support. That seriously stung. This decision should have been made a year or two ago, before CentOS 8 was released. If you have a kernel that hasn't been updated in over three years and you need very low downtime, you either have a dedicated operations crew or you don't let it touch the internet. But if you are so cheap that you don't want to pay for bandwidth/build servers yet aren't willing to violate the "developer" license agreement ... then use a RHEL clone from Amazon|Oracle. They are already undercutting Red Hat support packages, why not make them foot the bandwidth bill too? Everyone else should be celebrating: Red Hat is basically turning CentOS Stream into a spot where they can collaborate with their frenemies and ship more software faster. Qubes developers for example, have been spread very thin by Fedora's six month release cycle. Switching away from an RPM distro would be too difficult but CentOS lags too far behind mainline (5 years!) to be useful. If Red Hat can step up their game and push out a major release very 3 years and get the "pipeline" between Fedora and CentOS working ... maybe they can win some market share back from Debian and Ubuntu.
- rombert 6y ago
  > That's good, because regular old CentOS is not bit-for-bit identical either. CentOS might strip out all the RHEL trademarks, but they had to reverse engineer RHEL code drops just like Oracle, Amazon, Google, Facebook, SuSE, etc. Do you have a source for SUSE reverse engineering RHEL code drops? Last I checked they were unrelated.

tw04 6y ago

Based on my experience working for a hardware vendor and now in another role where I work with clients of all sizes in the enterprise, I think saying this works for 95% of users is an intentionally gross mischaracterization of the user base. Literally every enterprise user I work with that has CentOS uses it specifically because it directly tracks RHEL. They don't want to pay for RHEL licenses in test/dev but need a binary compatible version to ensure they can get enterprise support in production should they hit an issue. Having different drivers/kernels/libraries is a non-starter. There's absolutely no way that Redhat is so blind to their user base that they don't know this is happening. This is a pure money grab, if they can get even 5% of the users doing what I've seen paying for licenses, it's a net-win for them. I'm 100% onboard with Redhat doing things to make lives difficult for companies like Oracle that are trying to make money off their hard work. I have no time for user-hostile moves like this, every justification they've given is hollow IMO which is why they're still trying to do damage control and persuade the public that it's not what it looks like.

invokestatic 6y ago
Yes, I think that use case falls in the 5% gap that had the rug pulled under them. But I really do believe 95% of CentOS users don’t really need that, hosting stuff like HTTP servers or containers. Stuff that really doesn’t need to match RHEL.
- NathanOsullivan 6y ago
  Web hosters and the like need stability and don't really care about the underlying distribution at all. Those types of users will leave CentOS and never look back.
lmeyerov 6y ago
Bingo. RHEL is a subscription mess when working across its zoo ("Is our dev mirror the Azure's managed RHEL repository with mismatching kernel headers? Or pass through to RHEL's with epol disabled? Or RHEL 8 with docker repos disconnected so they can push their podman alternative?"). Losing centos hurts. As soon as you get into basics like drivers or build tools, RHEL quickly goes off the rails, and centos gave a semi-predictable escape hatch. GPU land is directly on this path, so I feel awful for enterprise IT admins we work with who just want to do a good job for their client hospitals, schools, businesses, etc. We help shield them most of the way by reducing the install to just a gpu driver + docker, but those deps still need basic pinned version ranges. RHEL makes getting those hard, and worse, as life happens, make it hard to get the right diagnostic tool versions (gcc 7, ...). It's a bad look for a 2020 infra vendor to charge an enterprise for reliable and predictable rollout sw/support when RHEL makes it a marathon to get the right major release of base layers like docker/gcc/drivers. Contrast this to our cloud users, who one-click launch, or our self-managed cloud/ubuntu/etc. ones, who can generally get the right drivers + docker stack up in 10-15 provided commands. Watching the chaos spread into centos makes me even more nervous about critical software relying on RHEL, and we already had to have The Talk with enterprises wanting HPC/ML on RHEL before this..
freedomben 6y ago
You're saying it's a "gross mischaracterization." Does that mean you have more/better data than Red Hat/CentOS about CentOS usage? > Literally every enterprise user I work with that has CentOS uses it specifically because it directly tracks RHEL. They don't want to pay for RHEL licenses in test/dev but need a binary compatible version to ensure they can get enterprise support in production should they hit an issue. Having different drivers/kernels/libraries is a non-starter. Unless you are doing kernelly things then the small difference in kernel version will not be that big a deal. Also for most applications, dev/test is better on Stream since it represent RHEL next. If the next version of RHEL will break your app, you find out in dev/test instead of in production like you used to. I think it's worth considering if this isn't better for your use case than previously. It's also not very hard to compile your own kernel if you really need an exact version. I do that sometimes on Fedora when I'm testing a driver I maintain. You can also just keep the RPMs from either previous releases as well (that's not a bad idea so even if you're building your own kernel you can extract the config file from it).

indolering 6y ago

> I don’t really need to exactly match RHEL’s point releases. I don’t need 100% binary compatibility with RHEL. That's good, because regular old CentOS is not bit-for-bit identical either. CentOS might strip out all the RHEL trademarks, but they had to reverse engineer RHEL code drops just like Oracle, Amazon, Google, Facebook, SuSE, etc. > The upstreams serve me well enough. What you don't appreciate is that the distinction between upstream/downstream is a broken metaphor across small time scales. Bug fixes and backports will show up in any number of RHEL clones from Oracle|Amazon|Facebook|Google|Baidu|Samsung before making their way into Fedora|CentOS and finally into RHEL. In the future, RHEL will release their own bugfixes to subscription customers before those fixes find their way into CentOS Stream. That's the meat grinder of a Linux distribution: uptime "guarantees" are just refunds you get to cash in when a large devops team can't fix a problem fast enough. > The one thing I can’t get over though, is the decision to roll out this change in the middle of CentOS 8’s lifecycle, trimming out what, 8 years or so of support. That seriously stung. This decision should have been made a year or two ago, before CentOS 8 was released. If you have a kernel that hasn't been updated in over three years and you need very low downtime, you either have a dedicated operations crew or you don't let it touch the internet. But if you are so cheap that you don't want to pay for bandwidth/build servers yet aren't willing to violate the "developer" license agreement ... then use a RHEL clone from Amazon|Oracle. They are already undercutting Red Hat support packages, why not make them foot the bandwidth bill too? Everyone else should be celebrating: Red Hat is basically turning CentOS Stream into a spot where they can collaborate with their frenemies and ship more software faster. Qubes developers for example, have been spread very thin by Fedora's six month release cycle. Switching away from an RPM distro would be too difficult but CentOS lags too far behind mainline (5 years!) to be useful. If Red Hat can step up their game and push out a major release very 3 years and get the "pipeline" between Fedora and CentOS working ... maybe they can win some market share back from Debian and Ubuntu.

rombert 6y ago
> That's good, because regular old CentOS is not bit-for-bit identical either. CentOS might strip out all the RHEL trademarks, but they had to reverse engineer RHEL code drops just like Oracle, Amazon, Google, Facebook, SuSE, etc. Do you have a source for SUSE reverse engineering RHEL code drops? Last I checked they were unrelated.