6 ms·

I would expect page cache to be one of those platform-dependent things that prevent OpenZFS from doing that. Especially on Linux, and especially because AFAIK t

I would expect page cache to be one of those platform-dependent things that prevent OpenZFS from doing that. Especially on Linux, and especially because AFAIK the Linux version has the most eyes on it.

rincebrain 3y ago
My understanding, not having dug into it, is that it's possible but just work nobody has done yet, though I'm not sure what the relevant interfaces are in the Linux kernel. One thing that makes the interfaces in Linux much messier than the FreeBSD ones is that a lot of the core functionality you might like to leverage in Linux (workqueues, basically anything more complicated than just calling kmalloc, and of course any SIMD save/restore state, to name three examples I've stumbled over recently) are marked EXPORT_SYMBOL_GPL or just entirely not exported in newer releases, so you get to reimplement the wheel for those, whereas on FreeBSD it's trivial to just use their implementations of such things and shim them to the Solaris-ish interfaces the non-platform-specific code expects. So that makes the Linux-specific code a lot heavier, because upstream is actively hostile.
- Dylan16807 3y ago
  > SIMD save/restore state I wish someone would come in and convince the kernel devs that "hey, if you want EXPORT_SYMBOL_GPL to have legal weight in a copyleft sense then you can't just slap it onto interfaces for political reasons"
  - rincebrain 3y ago
    I don't think they care about it having legal weight, that ship sailed long ago when they started advocating for just slapping SYMBOL_GPL on things out of spite; I think they care about excluding people from using their software. IMO Linus should stop being half-and-half about it and either mark everything SYMBOL_GPL and see how well that goes or stop this nonsense.
    - Filligree 3y ago
      I just don't understand why they're so anti-ZFS. I want my data to survive, please...
      - rincebrain 3y ago
        My impression is that some of the Linux kernel devs are anti-anything that's not GPL-compatible, of any sort, regardless of the particulars. Linus himself also made remarks about ZFS at one point that were pretty...hostile. [1] [2] > The fact is, the whole point of the GPL is that you're being "paid" in terms of tit-for-tat: we give source code to you for free, but we want source code improvements back. If you don't do that but instead say "I think this is _legal_, but I'm not going to help you" you certainly don't get any help from us. > So things that are outside the kernel tree simply do not matter to us. They get absolutely zero attention. We simply don't care. It's that simple. > And things that don't do that "give back" have no business talking about us being assholes when we don't care about them. > See? Note that there's at least one unfixed Linux kernel bug that was found by OpenZFS users, reproducible without using OpenZFS in any way, reported with a patch, and ignored. [3] So "not giving back" is a dubious claim. [1] - https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-statements-arent-right-heres-the-straight-dope/ https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-s... [2] - https://www.realworldtech.com/forum/?threadid=189711&curpostid=189959 https://www.realworldtech.com/forum/?threadid=189711&curpost... [3] - https://bugzilla.kernel.org/show_bug.cgi?id=212295 https://bugzilla.kernel.org/show_bug.cgi?id=212295
      - zokula 3y ago
        ZFS was made to be Linux hostile, the kernel devs are just returning the favor.
        p_l 3y ago
        That's frankly speaking untrue - we have one off-hand comment by one person from Sun for this, and multiple claims opposite. Also, ZFS goes with the same licensing interpretation as Linus used with AFS.
        3y ago
  - colonwqbang 3y ago
    Why don't you think it has legal weight? Or did you mean something else? As far as know the point of EXPORT_SYMBOL_GPL was to push back on companies like Nvidia who wanted to exploit loopholes in the GPL. That seems to me like a reasonable objective. Relevant Torvalds quote: https://yarchive.net/comp/linux/export_symbol_gpl.html https://yarchive.net/comp/linux/export_symbol_gpl.html
    - rincebrain 3y ago
      Sure, and that alone isn't an unreasonable premise - as he says, intent matters. But if you're marking interfaces as GPL-only, or implementing taint detection that means if you use a non-SYMBOL_GPL kernel symbol which calls a GPL-only function it treats the non-SYMBOL_GPL symbol as GPL-only and blocks your linking, it gets a bit out of hand. Building the kernel with certain kernel options makes modules like OpenZFS or OpenAFS not link because of that taint propagation - because things like the lockdep checker turn uninfringing calls into infringing ones. Or a little while ago, there was a change which broke building on PPC because a change made a non-SYMBOL_GPL call on POWER into a SYMBOL_GPL one indirectly, and when the original author was contacted, he sent a patch reverting the changed symbol, and GregKH refused to pull it into stable, suggesting distros could carry it if they wanted to. (Of course, he had happily merged a change into -stable earlier that just implemented more aggressive GPL tainting and thereby broke things like the aforementioned...)
      - PlutoIsAPlanet 3y ago
        The Linux kernel has never supported out of tree modules like how ZFS works out of tree. All ZFS needs to do is just have one of Oracles many lawyers say "CDDL is compatible with GPL". Yet, they Oracle don't.
        rincebrain 3y ago
        "All." It's explicitly not compatible with GPL, though. It has clauses that are more restrictive than GPL, and IIRC some people who contributed to the OpenZFS project did so explicitly without allowing later CDDL license revisions, which removes Oracle's ability to say CDDL-2 or whatever is GPL-compatible. So even if someone rolled up dumptrucks of cash and convinced Oracle that everything was great, they don't have all the control needed to do that.
- bakul 3y ago
  I suspect the underlying issues for not unifying the two have more to do with the ZFS design than anything to do with Linux. It may be the codebase is far too large at this stage to make such a fundamental change.
  - rincebrain 3y ago
    I don't think so. The memory management stuff is pretty well abstracted; on FBSD it just glues into UMA pretty transparently, it's just on Linux there's a lot of machinery for implementing our own little cache allocating because Linux's kernel cache allocator is very limited in what sizes it will give you, and sometimes ZFS wants 16M (not necessarily contiguous) regions because someone said they wanted 16M records. The ZoL project lead said at one point there were a variety of reasons this wasn't initially done for the Linux integration [1], but that it was worth taking another look at since that was a decade ago now. Having looked at the Linux memory subsystems recently for various reasons, I would suspect the limiting factor is that almost all the Linux memory management functions that involve details beyond "give me X pages" are SYMBOL_GPL, so I suspect we couldn't access whatever functionality would be needed to do this. I could be wrong, though, as I wasn't looking at the code for that specific purpose, so I might have missed functionality that would provide this. [1] - https://github.com/openzfs/zfs/issues/10255#issuecomment-620886713 https://github.com/openzfs/zfs/issues/10255#issuecomment-620...
    - bakul 3y ago
      Behlendorf's comment in that thread seems to be talking about linux integration. My point was this is an older issue, going back to the Sun days. See for instance this thread in where McVoy complains about the same issue! https://www.tuhs.org/pipermail/tuhs/2021-February/023013.html https://www.tuhs.org/pipermail/tuhs/2021-February/023013.htm...
      - rincebrain 3y ago
        That seems more like it's complaining about it not being the actual page cache, not it not being counted as "cache", which is a larger set in at least Linux than just the page cache itself. But sure, it's certainly an older issue, and given that the ABD rework happened, I wouldn't put anything past being "feasible" if the benefits were great enough. (Look at the O_DIRECT zvol rework stuff that's pending (I believe not merged) for how a more cut-through memory model could be done, though that has all the tradeoffs you might expect of skipping the abstractions ZFS uses to minimize the ability of applications to poke holes in the abstraction model and violate consistency, I believe...)
- 5e92cb50239222b 3y ago
  Sun has picked the license specifically to make the project incompatible with the Linux kernel. If you start out with hostility, what treatment do you expect in return?
  - depr 3y ago
    Surely you have a source for that that isn't someone else's comment.
the8472 3y ago
Could the linux integration use dax[0] to bypass the page cache and go straight to ARC? [0] https://www.kernel.org/doc/Documentation/filesystems/dax.txt https://www.kernel.org/doc/Documentation/filesystems/dax.txt

rincebrain 3y ago

My understanding, not having dug into it, is that it's possible but just work nobody has done yet, though I'm not sure what the relevant interfaces are in the Linux kernel. One thing that makes the interfaces in Linux much messier than the FreeBSD ones is that a lot of the core functionality you might like to leverage in Linux (workqueues, basically anything more complicated than just calling kmalloc, and of course any SIMD save/restore state, to name three examples I've stumbled over recently) are marked EXPORT_SYMBOL_GPL or just entirely not exported in newer releases, so you get to reimplement the wheel for those, whereas on FreeBSD it's trivial to just use their implementations of such things and shim them to the Solaris-ish interfaces the non-platform-specific code expects. So that makes the Linux-specific code a lot heavier, because upstream is actively hostile.

Dylan16807 3y ago
> SIMD save/restore state I wish someone would come in and convince the kernel devs that "hey, if you want EXPORT_SYMBOL_GPL to have legal weight in a copyleft sense then you can't just slap it onto interfaces for political reasons"
- rincebrain 3y ago
  I don't think they care about it having legal weight, that ship sailed long ago when they started advocating for just slapping SYMBOL_GPL on things out of spite; I think they care about excluding people from using their software. IMO Linus should stop being half-and-half about it and either mark everything SYMBOL_GPL and see how well that goes or stop this nonsense.
  - Filligree 3y ago
    I just don't understand why they're so anti-ZFS. I want my data to survive, please...
    - rincebrain 3y ago
      My impression is that some of the Linux kernel devs are anti-anything that's not GPL-compatible, of any sort, regardless of the particulars. Linus himself also made remarks about ZFS at one point that were pretty...hostile. [1] [2] > The fact is, the whole point of the GPL is that you're being "paid" in terms of tit-for-tat: we give source code to you for free, but we want source code improvements back. If you don't do that but instead say "I think this is _legal_, but I'm not going to help you" you certainly don't get any help from us. > So things that are outside the kernel tree simply do not matter to us. They get absolutely zero attention. We simply don't care. It's that simple. > And things that don't do that "give back" have no business talking about us being assholes when we don't care about them. > See? Note that there's at least one unfixed Linux kernel bug that was found by OpenZFS users, reproducible without using OpenZFS in any way, reported with a patch, and ignored. [3] So "not giving back" is a dubious claim. [1] - https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-statements-arent-right-heres-the-straight-dope/ https://arstechnica.com/gadgets/2020/01/linus-torvalds-zfs-s... [2] - https://www.realworldtech.com/forum/?threadid=189711&curpostid=189959 https://www.realworldtech.com/forum/?threadid=189711&curpost... [3] - https://bugzilla.kernel.org/show_bug.cgi?id=212295 https://bugzilla.kernel.org/show_bug.cgi?id=212295
    - zokula 3y ago
      ZFS was made to be Linux hostile, the kernel devs are just returning the favor.
      - p_l 3y ago
        That's frankly speaking untrue - we have one off-hand comment by one person from Sun for this, and multiple claims opposite. Also, ZFS goes with the same licensing interpretation as Linus used with AFS.
      - 3y ago
- colonwqbang 3y ago
  Why don't you think it has legal weight? Or did you mean something else? As far as know the point of EXPORT_SYMBOL_GPL was to push back on companies like Nvidia who wanted to exploit loopholes in the GPL. That seems to me like a reasonable objective. Relevant Torvalds quote: https://yarchive.net/comp/linux/export_symbol_gpl.html https://yarchive.net/comp/linux/export_symbol_gpl.html
  - rincebrain 3y ago
    Sure, and that alone isn't an unreasonable premise - as he says, intent matters. But if you're marking interfaces as GPL-only, or implementing taint detection that means if you use a non-SYMBOL_GPL kernel symbol which calls a GPL-only function it treats the non-SYMBOL_GPL symbol as GPL-only and blocks your linking, it gets a bit out of hand. Building the kernel with certain kernel options makes modules like OpenZFS or OpenAFS not link because of that taint propagation - because things like the lockdep checker turn uninfringing calls into infringing ones. Or a little while ago, there was a change which broke building on PPC because a change made a non-SYMBOL_GPL call on POWER into a SYMBOL_GPL one indirectly, and when the original author was contacted, he sent a patch reverting the changed symbol, and GregKH refused to pull it into stable, suggesting distros could carry it if they wanted to. (Of course, he had happily merged a change into -stable earlier that just implemented more aggressive GPL tainting and thereby broke things like the aforementioned...)
    - PlutoIsAPlanet 3y ago
      The Linux kernel has never supported out of tree modules like how ZFS works out of tree. All ZFS needs to do is just have one of Oracles many lawyers say "CDDL is compatible with GPL". Yet, they Oracle don't.
      - rincebrain 3y ago
        "All." It's explicitly not compatible with GPL, though. It has clauses that are more restrictive than GPL, and IIRC some people who contributed to the OpenZFS project did so explicitly without allowing later CDDL license revisions, which removes Oracle's ability to say CDDL-2 or whatever is GPL-compatible. So even if someone rolled up dumptrucks of cash and convinced Oracle that everything was great, they don't have all the control needed to do that.
bakul 3y ago
I suspect the underlying issues for not unifying the two have more to do with the ZFS design than anything to do with Linux. It may be the codebase is far too large at this stage to make such a fundamental change.
- rincebrain 3y ago
  I don't think so. The memory management stuff is pretty well abstracted; on FBSD it just glues into UMA pretty transparently, it's just on Linux there's a lot of machinery for implementing our own little cache allocating because Linux's kernel cache allocator is very limited in what sizes it will give you, and sometimes ZFS wants 16M (not necessarily contiguous) regions because someone said they wanted 16M records. The ZoL project lead said at one point there were a variety of reasons this wasn't initially done for the Linux integration [1], but that it was worth taking another look at since that was a decade ago now. Having looked at the Linux memory subsystems recently for various reasons, I would suspect the limiting factor is that almost all the Linux memory management functions that involve details beyond "give me X pages" are SYMBOL_GPL, so I suspect we couldn't access whatever functionality would be needed to do this. I could be wrong, though, as I wasn't looking at the code for that specific purpose, so I might have missed functionality that would provide this. [1] - https://github.com/openzfs/zfs/issues/10255#issuecomment-620886713 https://github.com/openzfs/zfs/issues/10255#issuecomment-620...
  - bakul 3y ago
    Behlendorf's comment in that thread seems to be talking about linux integration. My point was this is an older issue, going back to the Sun days. See for instance this thread in where McVoy complains about the same issue! https://www.tuhs.org/pipermail/tuhs/2021-February/023013.html https://www.tuhs.org/pipermail/tuhs/2021-February/023013.htm...
    - rincebrain 3y ago
      That seems more like it's complaining about it not being the actual page cache, not it not being counted as "cache", which is a larger set in at least Linux than just the page cache itself. But sure, it's certainly an older issue, and given that the ABD rework happened, I wouldn't put anything past being "feasible" if the benefits were great enough. (Look at the O_DIRECT zvol rework stuff that's pending (I believe not merged) for how a more cut-through memory model could be done, though that has all the tradeoffs you might expect of skipping the abstractions ZFS uses to minimize the ability of applications to poke holes in the abstraction model and violate consistency, I believe...)
5e92cb50239222b 3y ago
Sun has picked the license specifically to make the project incompatible with the Linux kernel. If you start out with hostility, what treatment do you expect in return?
- depr 3y ago
  Surely you have a source for that that isn't someone else's comment.

the8472 3y ago

Could the linux integration use dax[0] to bypass the page cache and go straight to ARC? [0] https://www.kernel.org/doc/Documentation/filesystems/dax.txt https://www.kernel.org/doc/Documentation/filesystems/dax.txt