6 ms·

I really hate the fact that RHEL updates kernels so slowly. This becomes very painful when your product cannot take advantage of some advanced kernel features j

by mikevm 6y ago

I really hate the fact that RHEL updates kernels so slowly. This becomes very painful when your product cannot take advantage of some advanced kernel features just because you have customers running RHEL :(.

bluedays 6y ago
Is there anyway you can use a different kernel? The kernel is really easily replaceable in other distros. I don't know much about RHEL.
- pnutjam 6y ago
  No, you can't do that in RHEL. It would invalidate support.
  - sam_lowry_ 6y ago
    O'rly?
  - bluedays 6y ago
    Seems sad that the modularity of Linux is not allowed due to issues with support. That, in my opinion, is the best part of Linux - which is finding solutions to problems that exists outside of the scope of what is designed by the vendors.
    - azinman2 6y ago
      A big part of RHEL IS their kernel. They spend a lot of time to optimize/stabilize their releases. Why would they support any other kernel?
    - yjftsjthsd-h 6y ago
      It's a tradeoff. There's nothing to stop you from taking a RHEL system and installing the very latest kernel, and there's a good chance that the resulting system would work fine. However, RH the company only supports their own versions, which seems reasonable. Also, and not unrelated, I believe RHEL kernels hold a stable ABI to the point that you can use vendor-provided (out-of-tree) drivers and not have them break on every release, which is something that normal kernels intentionally don't support. So you can use RHEL like any other Linux system, but you lose some of the advantages that made RHEL useful in the first place.
    - umanwizard 6y ago
      Nothing stops people from using whatever distro they like, instead of RHEL. If they are using RHEL, presuming they’re rational actors, it’s because the RHEL model works for their use case.
    - jcrawfordor 6y ago
      You're free to do whatever you want with your Linux distribution, but if you're not using the thing that the vendor supports, they're not going to support it. There's not really any way around it. Unfortunately the kernel has significant and widespread implications on the system, so it's very hard to figure out how to support a system that is running a kernel version you haven't validated. As a RH customer, the way I handle this kind of thing is usually by replicating the problem on a "stock" system and working with RH support on fixing it on that system --- this is often an important step anyway since, for my own purposes, I need to figure out whether or not I introduced the problem by something I did.
- juangacovas 6y ago
  I update some CentOS servers with ML kernel from ElRepo.org, and no problems for four years, but my stack could be minimalist (webservers) and do just fine
- curt15 6y ago
  Facebook runs CentOS with mainline kernels: https://www.youtube.com/watch?v=cA_Nd3crBuA&feature=youtu.be https://www.youtube.com/watch?v=cA_Nd3crBuA&feature=youtu.be
pnutjam 6y ago
Try SLES; enterprise support, recent kernels.
Twirrim 6y ago
Suggest Oracle Linux. Their UEK gets refreshed on a regular basis and it's way closer to upstream. UEK6 just released for OL7 and OL8, based off the 5.4 kernel. (disclaimer: I work for the Compute team in Oracle Cloud Infrastructure, and work with the OL team regularly on a number of things. It has been one of the more interesting aspects of the job)
kbenson 6y ago
If Red Hat (and any of the other major distros) didn't do it, someone else would as a service you could pay for. That lack of updating is exactly why they are used.
awill 6y ago
It can be frustrating, but it's almost always worth it in a serious production environment. I have my own anecdote: I have serveral home servers running CentOS 7 (set up nearly 6 years ago). I update all the time and I've had zero issues. Literally nothing has ever broken. My previous servers ran CentOS 6, and were equally reliable. My desktop runs Arch Linux, and gets ever kernel dot release. I've had an unbootable system at least 5 times over the last 5 years, requiring me to boot from a flash drive to downgrade a package, or edit a grub config. As you can imagine, it's bitten me at the worst possible time (when I really don't have time to troubleshoot). I know some people advocate for Arch on the server, but to me that seems crazy. Nothing against Arch. It's great for a desktop, and I knowingly trade off some reliability/stability for the latest stuff. I wouldn't make that trade on a server that hosted important stuff.
- ryanisnan 6y ago
  I second this. I was running antergos on my desktop for a while and it is a fantastic desktop OS, if you have time to deal with the inevitable boot failures.
- gbrown 6y ago
  Fedora seems like a sensible middle ground.
  - deleted 6y ago
    [deleted]
- hnra 6y ago
  I would recommend keeping an lts kernel around when running arch. It’s not much harder than pacman-S linux-lts.
  - awill 6y ago
    That probably helps, though many of my issues have been other problems, like xorg, the nvidia driver, grub (or now systemd-boot)
brozaman 6y ago
If it's any consolation, I'm a software engineer working for Red Hat on the userspace part of one of our most bleeding edge products, and it's a pain for us too :). And it's a much bigger pain for our kernel guys It would be way cheaper and easier for us to ship a newer kernel, we do it because it allows RHEL to have significantly less regressions than upstream kernels (and I'm talking about what the kernel engineers say, not some marketing bullshit).
- bonzini 6y ago
  Yes (kernel engineer here), we can mix and match what we pick and when. Almost every major subsystem is updated only a few months after the corresponding patches hit Linus's tree. I would guess that most of RHEL 7 is on par with 4.14 or 4.19 (which is newer than the RHEL 8 kernel!) and RHEL8.3 will have subsystems updated up to 5.7-5.8. Of course it's not much fun backporting 50 patches for Intel vulnerability mitigations to RHEL7.2 that some mission critical apps still use. But I guess it pays my bills so who am I to complain, I love everything else about my job. ;)

by mikevm 6y ago

bluedays 6y ago

Is there anyway you can use a different kernel? The kernel is really easily replaceable in other distros. I don't know much about RHEL.

pnutjam 6y ago
No, you can't do that in RHEL. It would invalidate support.
- sam_lowry_ 6y ago
  O'rly?
- bluedays 6y ago
  Seems sad that the modularity of Linux is not allowed due to issues with support. That, in my opinion, is the best part of Linux - which is finding solutions to problems that exists outside of the scope of what is designed by the vendors.
  - azinman2 6y ago
    A big part of RHEL IS their kernel. They spend a lot of time to optimize/stabilize their releases. Why would they support any other kernel?
  - yjftsjthsd-h 6y ago
    It's a tradeoff. There's nothing to stop you from taking a RHEL system and installing the very latest kernel, and there's a good chance that the resulting system would work fine. However, RH the company only supports their own versions, which seems reasonable. Also, and not unrelated, I believe RHEL kernels hold a stable ABI to the point that you can use vendor-provided (out-of-tree) drivers and not have them break on every release, which is something that normal kernels intentionally don't support. So you can use RHEL like any other Linux system, but you lose some of the advantages that made RHEL useful in the first place.
  - umanwizard 6y ago
    Nothing stops people from using whatever distro they like, instead of RHEL. If they are using RHEL, presuming they’re rational actors, it’s because the RHEL model works for their use case.
  - jcrawfordor 6y ago
    You're free to do whatever you want with your Linux distribution, but if you're not using the thing that the vendor supports, they're not going to support it. There's not really any way around it. Unfortunately the kernel has significant and widespread implications on the system, so it's very hard to figure out how to support a system that is running a kernel version you haven't validated. As a RH customer, the way I handle this kind of thing is usually by replicating the problem on a "stock" system and working with RH support on fixing it on that system --- this is often an important step anyway since, for my own purposes, I need to figure out whether or not I introduced the problem by something I did.
juangacovas 6y ago
I update some CentOS servers with ML kernel from ElRepo.org, and no problems for four years, but my stack could be minimalist (webservers) and do just fine
curt15 6y ago
Facebook runs CentOS with mainline kernels: https://www.youtube.com/watch?v=cA_Nd3crBuA&feature=youtu.be https://www.youtube.com/watch?v=cA_Nd3crBuA&feature=youtu.be

pnutjam 6y ago

Try SLES; enterprise support, recent kernels.

Twirrim 6y ago

Suggest Oracle Linux. Their UEK gets refreshed on a regular basis and it's way closer to upstream. UEK6 just released for OL7 and OL8, based off the 5.4 kernel. (disclaimer: I work for the Compute team in Oracle Cloud Infrastructure, and work with the OL team regularly on a number of things. It has been one of the more interesting aspects of the job)

kbenson 6y ago

If Red Hat (and any of the other major distros) didn't do it, someone else would as a service you could pay for. That lack of updating is exactly why they are used.

awill 6y ago

It can be frustrating, but it's almost always worth it in a serious production environment. I have my own anecdote: I have serveral home servers running CentOS 7 (set up nearly 6 years ago). I update all the time and I've had zero issues. Literally nothing has ever broken. My previous servers ran CentOS 6, and were equally reliable. My desktop runs Arch Linux, and gets ever kernel dot release. I've had an unbootable system at least 5 times over the last 5 years, requiring me to boot from a flash drive to downgrade a package, or edit a grub config. As you can imagine, it's bitten me at the worst possible time (when I really don't have time to troubleshoot). I know some people advocate for Arch on the server, but to me that seems crazy. Nothing against Arch. It's great for a desktop, and I knowingly trade off some reliability/stability for the latest stuff. I wouldn't make that trade on a server that hosted important stuff.

ryanisnan 6y ago
I second this. I was running antergos on my desktop for a while and it is a fantastic desktop OS, if you have time to deal with the inevitable boot failures.
gbrown 6y ago
Fedora seems like a sensible middle ground.
- deleted 6y ago
  [deleted]
hnra 6y ago
I would recommend keeping an lts kernel around when running arch. It’s not much harder than pacman-S linux-lts.
- awill 6y ago
  That probably helps, though many of my issues have been other problems, like xorg, the nvidia driver, grub (or now systemd-boot)

brozaman 6y ago

If it's any consolation, I'm a software engineer working for Red Hat on the userspace part of one of our most bleeding edge products, and it's a pain for us too :). And it's a much bigger pain for our kernel guys It would be way cheaper and easier for us to ship a newer kernel, we do it because it allows RHEL to have significantly less regressions than upstream kernels (and I'm talking about what the kernel engineers say, not some marketing bullshit).

bonzini 6y ago
Yes (kernel engineer here), we can mix and match what we pick and when. Almost every major subsystem is updated only a few months after the corresponding patches hit Linus's tree. I would guess that most of RHEL 7 is on par with 4.14 or 4.19 (which is newer than the RHEL 8 kernel!) and RHEL8.3 will have subsystems updated up to 5.7-5.8. Of course it's not much fun backporting 50 patches for Intel vulnerability mitigations to RHEL7.2 that some mission critical apps still use. But I guess it pays my bills so who am I to complain, I love everything else about my job. ;)