3 ms·
> The version and ABI guarantee is not for everyone. As an aside, that kABI guarantee only goes so far. I work in HPC/AI, and the out-of-tree drivers we use li
by jabl 1y ago
> The version and ABI guarantee is not for everyone.
As an aside, that kABI guarantee only goes so far. I work in HPC/AI, and the out-of-tree drivers we use like MOFED and Lustre drivers would break with EVERY SINGLE RHEL minor update (like RHEL X.Y -> X.(Y+1) ). Using past form here because I haven't been using RHEL for this purpose for the past ~5 years, so maybe it has changed since although I doubt it.
- pabs3 1y agoIs anyone trying to get those drivers upstreamed?
- globalc 1y agoThat, and if not possible one can try to get the used kABI symbols graylisted at Red Hat, to get informed when they change.
- jabl 1y agoYes, and yes. The kernel parts of MOFED are largely backports of the latest and greatest upstream kernel drivers to the various distro kernels their customers actually run. (The non-kernel parts of MOFED is mostly open source but does contain some proprietary special sauce on top, like IIRC SHARP support isn't available in FOSS.). The HPC community does tend to want to use the latest RDMA drivers as those are critical for at scale performance. For Lustre, the client driver was upstreamed into staging, where it sat AFAIU largely unused for a few years until it was ripped out again. The problem was that Lustre developers didn't adopt an upstream-first development approach, and thus the in-kernel driver was basically a throw over the fence fork that nobody cared about. I think there is an effort to try again and hopefully adopt an upstream-first approach, remains to be seen whether it'll succeed.
- pabs3 1y agoFor MOFED, why not just wholesale use a newer Linux kernel version?
- jabl 1y agoPerhaps the cure is worse than the disease? There are several reasons to stay with a distro kernel: - Lustre releases target distro kernels, upstream would likely break. - Distro stays on top of CVE's etc. and provide updates when needed. - HW likely certified for a few supported distros only, use anything else and you're on your own.
- aragilar 1y agoLustre got dropped from the kernel, so it seems unlikely?
- lustre-fan 1y ago[dead]
- kj4ips 1y agoIn that boat now, weak-modules means you sometimes get lucky, and can reuse. However, since it's more effort to determine if a rebuild is needed than just slap the "build all the vendor kmods and slurm" button, we tend to build for each kernel. IIRC el8 added kernel symbol signature hashes as Provide/Requires, with automation to extract them at build time, so kmods got a lot easier to deal with.
- jabl 1y agoSorry, I was being imprecise. Rebuilds per se are no problem, as both MOFED and Lustre provide sources, and DKMS nicely automates rebuilding when installing a new kernel image. The actual problem is that RHEL minor releases would also break kernel internal API's, and thus building the kernel modules would fail.