3 ms·

If Linux would add a stable kernel module API this wouldn't be a huge a problem and it would be easy for bcachefs to ship as a kernel module with his own indepe

If Linux would add a stable kernel module API this wouldn't be a huge a problem and it would be easy for bcachefs to ship as a kernel module with his own independent release schedule.

josephcsible 1y ago
The slight benefit for out-of-tree module authors wouldn't be worth the negative effects on the rest of the kernel to everyone else.
- charcircuit 1y ago
  "slight benefit"? Having a working system after upgrading your kernel is not just a slight benefit. It's table stakes. Especially for something critical like a filesystem it should never break. >negative effects on the rest of the kernel Needing to design and support an API is not purely negative for kernel developers. It also gives a change to have a proper interface for drivers to use and follow. Take a look at the Rust for Linux which keeps running into undocumented APIs that make little sense and are just whatever <insert most popular driver> does.
  - josephcsible 1y ago
    > Having a working system after upgrading your kernel is not just a slight benefit. It's table stakes. We already have that, with the "don't break userspace" policy combined with all of the modules being in-tree. > Needing to design and support an API is not purely negative for kernel developers. Sure, it's not purely negative, but it's overall a big net negative. > Take a look at the Rust for Linux which keeps running into undocumented APIs that make little sense and are just whatever <insert most popular driver> does. That's an argument against a stable module API! Those things are getting fixed as they get found, but if we had a stable module API, we'd be stuck with them forever. I recommend reading https://docs.kernel.org/process/stable-api-nonsense.html https://docs.kernel.org/process/stable-api-nonsense.html
    - charcircuit 1y ago
      >We already have that, with the "don't break userspace" Bcachefs is not user space. >with all of the modules being in-tree. That is not true. There are out of tree modules such as ZFS. >That's an argument against a stable module API! My point was that there was 0 thought put into creating a good API. Additionally API could be evolved over time and have a support period if you care about being able to evolve it and deprecate the old one. And likely even with a better interface there is probably a way to make the old API still function.
      - josephcsible 1y ago
        > Bcachefs is not user space. bcachefs is still in-tree. > That is not true. There are out of tree modules such as ZFS. ZFS could be in-tree in no time at all if Oracle would fix its license. And until they do that, it's not safe to use ZFS-on-Linux anyway, since Oracle could sue you for it. > My point was that there was 0 thought put into creating a good API. There is thought put into it: it's exactly what we need right now, because if what we need ever changes, we'll change the API too, thus avoiding YAGNI and similar problems. > Additionally API could be evolved over time and have a support period if you care about being able to evolve it. If a temporary "support period" is what you want, then just use the LTS kernels. That's already exactly what they give you. > And likely even with a better interface there is probably a way to make the old API still function. That's the big net negative I was mentioning and that https://docs.kernel.org/process/stable-api-nonsense.html https://docs.kernel.org/process/stable-api-nonsense.html talks about too. Sometimes there isn't a feasible way to support part of an old API anymore, and it's not worth holding the whole kernel back just for the out-of-tree modules.
        deleted 1y ago
        [deleted]
        yjftsjthsd-h 1y ago
        > ZFS could be in-tree in no time at all if Oracle would fix its license. And until they do that, it's not safe to use ZFS-on-Linux anyway, since Oracle could sue you for it. IANAL, but I don't believe either of these things are true. OpenZFS contains enough code not authored by Sun/Oracle that relicensing it now is effectively impossible. OTOH, it is under the CDDL, which is a perfectly good open source license; AFAICT the problem, if one exists at all[0], only manifests when distributing the combination of CDDL (OpenZFS) and GPL (Linux) software. If you download CDDL software and compile it into GPL software yourself (say, with DKMS) then it should be fine because you aren't distributing it. [0] This is a case where I'm going to really emphasize that I'm really not a lawyer and merely point out that ex. Canonical's lawyers do seem to think CDDL+GPL is okay.
  - msgodel 1y ago
    Does your system have some critical out of tree driver? That should have been recompiled with the new kernel, that sounds like a failure of whoever maintains the driver/kernel/distro (which may be you if you're building it yourself.)
homebrewer 1y ago
It would also have a lot less FOSS drivers, neither we nor FreeBSD (which is often invoked in these complaints) would have amdgpu for example.
- charcircuit 1y ago
  I would actually posture that making it easier to make drivers would actually have the opposite effect and result in more FOSS drivers. >FreeBSD (which is often invoked in these complaints) would have amdgpu for example. In such a hypothetical FreeBSD could reimplement the stable API of Linux.
  - throw0101d 1y ago
    > In such a hypothetical FreeBSD could reimplement the stable API of Linux. Like it does with the userland API of Linux, which is stable: * https://wiki.freebsd.org/Linuxulator https://wiki.freebsd.org/Linuxulator
  - smcameron 1y ago
    No, every gpu vendor out there would prefer proprietary drivers and with a stable ABI, they could do it, and would do, there is no question about it. I worked for HP on storage drivers for a decade or so, and had their been a stable ABI, HP would have shipped proprietary storage drivers for everything. Even without a stable ABI, they shipped proprietary drivers at considerable effort, compiling for myriad different distro kernels. It was a nightmare, and good thing too, or there wouldn't be any open source drivers.
    - charcircuit 1y ago
      I never said they wouldn't. Having more and better drivers is a good thing for Linux users. It's okay for proprietary drivers to exist. The kernel isn't meant to be a vehicle to push the free software agenda.
      - deleted 1y ago
        [deleted]
heavyset_go 1y ago
The unstable interface is Linux's moat, and IMO, is the reason we're able to enjoy such a large ecosystem of hardware via open source operating systems.
- zahlman 1y ago
  I'm afraid I don't follow your reasoning.
  - rcxdude 1y ago
    The interface churn in linux adds a strong incentive (on top of the GPL) to upstream drivers, i.e. publish them as open source. Not doing so tends to mean you get stuck on old versions. If it had a stable interface, hardware vendors would just release crappy binary blobs and they'd only be usable on linux, and not maintainable by anyone else (and hardware vendors don't generally maintain their drivers for long)
  - heavyset_go 1y ago
    With a stable driver API/ABI, vendors will just dump closed source drivers once and call it a day, or pull a Apple/Sony/Nintendo with FreeBSD, where you effectively get a closed source OS that supports your hardware. An unstable interface means the driver source needs to be updated frequently, you can't just dump a .ko file online and expect it to work for however long the hardware lasts. Easiest way to approach it is to attempt to upstream drivers, and potentially take advantage of free labor and maintenance in virtual perpetuity, which is good for all Linux users. If vendors don't want to spend the effort upstreaming drivers, but they need to support Linux, by necessity the drivers must be open source so they can be compiled against users' changing kernels. That's at least a step in the right direction, and should anyone want to make the effort, they're free to upstream drivers themselves.
    - shtripok 1y ago
      It is very difficult to get the driver included in the upstream. This is why almost none of the Chinese equipment manufacturers do this. And those that do do it for one or two models out of 2-3 dozen produced, and this process rarely takes less than 2-3 years. That is, by the time the device is included in the kernel, it is usually already out of production for a year or more. So don't repeat these legends from 20 years ago. However, this may not have been true even 20 years ago.
stefantalpalaru 1y ago
[dead]

If Linux would add a stable kernel module API this wouldn't be a huge a problem and it would be easy for bcachefs to ship as a kernel module with his own independent release schedule.

josephcsible 1y ago

The slight benefit for out-of-tree module authors wouldn't be worth the negative effects on the rest of the kernel to everyone else.

charcircuit 1y ago
"slight benefit"? Having a working system after upgrading your kernel is not just a slight benefit. It's table stakes. Especially for something critical like a filesystem it should never break. >negative effects on the rest of the kernel Needing to design and support an API is not purely negative for kernel developers. It also gives a change to have a proper interface for drivers to use and follow. Take a look at the Rust for Linux which keeps running into undocumented APIs that make little sense and are just whatever <insert most popular driver> does.
- josephcsible 1y ago
  > Having a working system after upgrading your kernel is not just a slight benefit. It's table stakes. We already have that, with the "don't break userspace" policy combined with all of the modules being in-tree. > Needing to design and support an API is not purely negative for kernel developers. Sure, it's not purely negative, but it's overall a big net negative. > Take a look at the Rust for Linux which keeps running into undocumented APIs that make little sense and are just whatever <insert most popular driver> does. That's an argument against a stable module API! Those things are getting fixed as they get found, but if we had a stable module API, we'd be stuck with them forever. I recommend reading https://docs.kernel.org/process/stable-api-nonsense.html https://docs.kernel.org/process/stable-api-nonsense.html
  - charcircuit 1y ago
    >We already have that, with the "don't break userspace" Bcachefs is not user space. >with all of the modules being in-tree. That is not true. There are out of tree modules such as ZFS. >That's an argument against a stable module API! My point was that there was 0 thought put into creating a good API. Additionally API could be evolved over time and have a support period if you care about being able to evolve it and deprecate the old one. And likely even with a better interface there is probably a way to make the old API still function.
    - josephcsible 1y ago
      > Bcachefs is not user space. bcachefs is still in-tree. > That is not true. There are out of tree modules such as ZFS. ZFS could be in-tree in no time at all if Oracle would fix its license. And until they do that, it's not safe to use ZFS-on-Linux anyway, since Oracle could sue you for it. > My point was that there was 0 thought put into creating a good API. There is thought put into it: it's exactly what we need right now, because if what we need ever changes, we'll change the API too, thus avoiding YAGNI and similar problems. > Additionally API could be evolved over time and have a support period if you care about being able to evolve it. If a temporary "support period" is what you want, then just use the LTS kernels. That's already exactly what they give you. > And likely even with a better interface there is probably a way to make the old API still function. That's the big net negative I was mentioning and that https://docs.kernel.org/process/stable-api-nonsense.html https://docs.kernel.org/process/stable-api-nonsense.html talks about too. Sometimes there isn't a feasible way to support part of an old API anymore, and it's not worth holding the whole kernel back just for the out-of-tree modules.
      - deleted 1y ago
        [deleted]
      - yjftsjthsd-h 1y ago
        > ZFS could be in-tree in no time at all if Oracle would fix its license. And until they do that, it's not safe to use ZFS-on-Linux anyway, since Oracle could sue you for it. IANAL, but I don't believe either of these things are true. OpenZFS contains enough code not authored by Sun/Oracle that relicensing it now is effectively impossible. OTOH, it is under the CDDL, which is a perfectly good open source license; AFAICT the problem, if one exists at all[0], only manifests when distributing the combination of CDDL (OpenZFS) and GPL (Linux) software. If you download CDDL software and compile it into GPL software yourself (say, with DKMS) then it should be fine because you aren't distributing it. [0] This is a case where I'm going to really emphasize that I'm really not a lawyer and merely point out that ex. Canonical's lawyers do seem to think CDDL+GPL is okay.
- msgodel 1y ago
  Does your system have some critical out of tree driver? That should have been recompiled with the new kernel, that sounds like a failure of whoever maintains the driver/kernel/distro (which may be you if you're building it yourself.)

homebrewer 1y ago

It would also have a lot less FOSS drivers, neither we nor FreeBSD (which is often invoked in these complaints) would have amdgpu for example.

charcircuit 1y ago
I would actually posture that making it easier to make drivers would actually have the opposite effect and result in more FOSS drivers. >FreeBSD (which is often invoked in these complaints) would have amdgpu for example. In such a hypothetical FreeBSD could reimplement the stable API of Linux.
- throw0101d 1y ago
  > In such a hypothetical FreeBSD could reimplement the stable API of Linux. Like it does with the userland API of Linux, which is stable: * https://wiki.freebsd.org/Linuxulator https://wiki.freebsd.org/Linuxulator
- smcameron 1y ago
  No, every gpu vendor out there would prefer proprietary drivers and with a stable ABI, they could do it, and would do, there is no question about it. I worked for HP on storage drivers for a decade or so, and had their been a stable ABI, HP would have shipped proprietary storage drivers for everything. Even without a stable ABI, they shipped proprietary drivers at considerable effort, compiling for myriad different distro kernels. It was a nightmare, and good thing too, or there wouldn't be any open source drivers.
  - charcircuit 1y ago
    I never said they wouldn't. Having more and better drivers is a good thing for Linux users. It's okay for proprietary drivers to exist. The kernel isn't meant to be a vehicle to push the free software agenda.
    - deleted 1y ago
      [deleted]

heavyset_go 1y ago

The unstable interface is Linux's moat, and IMO, is the reason we're able to enjoy such a large ecosystem of hardware via open source operating systems.

zahlman 1y ago
I'm afraid I don't follow your reasoning.
- rcxdude 1y ago
  The interface churn in linux adds a strong incentive (on top of the GPL) to upstream drivers, i.e. publish them as open source. Not doing so tends to mean you get stuck on old versions. If it had a stable interface, hardware vendors would just release crappy binary blobs and they'd only be usable on linux, and not maintainable by anyone else (and hardware vendors don't generally maintain their drivers for long)
- heavyset_go 1y ago
  With a stable driver API/ABI, vendors will just dump closed source drivers once and call it a day, or pull a Apple/Sony/Nintendo with FreeBSD, where you effectively get a closed source OS that supports your hardware. An unstable interface means the driver source needs to be updated frequently, you can't just dump a .ko file online and expect it to work for however long the hardware lasts. Easiest way to approach it is to attempt to upstream drivers, and potentially take advantage of free labor and maintenance in virtual perpetuity, which is good for all Linux users. If vendors don't want to spend the effort upstreaming drivers, but they need to support Linux, by necessity the drivers must be open source so they can be compiled against users' changing kernels. That's at least a step in the right direction, and should anyone want to make the effort, they're free to upstream drivers themselves.
  - shtripok 1y ago
    It is very difficult to get the driver included in the upstream. This is why almost none of the Chinese equipment manufacturers do this. And those that do do it for one or two models out of 2-3 dozen produced, and this process rarely takes less than 2-3 years. That is, by the time the device is included in the kernel, it is usually already out of production for a year or more. So don't repeat these legends from 20 years ago. However, this may not have been true even 20 years ago.

stefantalpalaru 1y ago

[dead]