5 ms·

Kudos for Microsoft for not going the Linux NIH way.

by lelf 8y ago

josteink 8y ago
NIH would imply they had a real choice. Sadly due to licensing, they don’t. Edit: I stand corrected. GPL should be fine.
- lelf 8y ago
  It's pure NIH. dtrace is GPLv2. (Also ZFS (CDDL) run fine on Linux.)
  - josteink 8y ago
    Thanks for the correction. I thought/assumed dtrace was CDDL too. I know ZFS works fine on Linux, because I use it everywhere, but there’s no way around the fact that the incompatible licenses are causing issues.
    - johannkokos 8y ago
      You're not wrong. Dtrace is gpl only in 2017.
  - kapilvt 8y ago
    dtrace is now gplv2 (2018, ~10 years after the acquisition), it wasn't for a long time (and its still a separate license around userspace), because oracle was holding it as a competitive advantage I presume for unbreakable linux distro (they ported dtrace over to unbreakable linux ~2012), the rest of the linux community continued with other projects (system tap, ktap, lttng, ftrace, etc), before settling in over the last 24 months onto building onto of bpf (see https://github.com/iovisor/bpftrace https://github.com/iovisor/bpftrace for a high level dsl interface), at which point oracle GPL'd dtrace, ie when it held no value, because everyone had moved on.
    - lelf 8y ago
      Again, even when it was CDDL it wasn't a real problem. See ZFS. > system tap, ktap, lttng, ftrace, etc That's why I'm talking about NIH. Even with ebfp, they intentionally choose different tracing syntax (just look at it, it's comical). Didn't use ready-to-use CTF. Etc etc etc.
      - yellowapple 8y ago
        > See ZFS ZFS is still problematic, specifically because of the GPL-incompatible license. Various distros have found ways around that (or are deliberately ignoring those issues at their own peril), but it's still very much "a real problem", and it took years (if not decades) to get to the point where ZFS on Linux is actually as viable an option as it is today.
      - aseipp 8y ago
        You're deluding yourself if you think the CDDL hasn't been a major hurdle against ZFS success in the Linux world. ZFS isn't successful in Linux because the CDDL doesn't matter, it's successful despite the fact it does matter because people put a lot more work into it than otherwise needed and work with the licensing and packaging issues involved with that. But saying "it isn't a real problem" is just factually false, no matter how much you don't like that. If the CDDL simply didn't matter, after all, they could just add it to the source tree and be done with it. I wonder why they haven't done that...
      - BurningCycles 8y ago
        >Again, even when it was CDDL it wasn't a real problem. See ZFS. ZFS is not in the Linux kernel, and Linus has stated that he will never merge CDDL licensed code due to license incompability. Those Linux distros that distribute ZFS do so as a separate module, not part of the Linux kernel. DTrace was GPL licensed too late (2018), something even Brendan Gregg stated as he finds eBPF more capable (and he is a leading expert on DTrace).
    - nullnix 8y ago
      DTrace for Linux CTF hacker here. The BPF people cannot be blamed for not using CTF. Even if it had been relicensed to the GPLv2 way back in 2005, until recently (https://github.com/oracle/libdtrace-ctf https://github.com/oracle/libdtrace-ctf release 1.0 or 1.1) it was impractical to use CTF on larger projects because the file format could only encode a strictly limited number of types (2^15 in each of a parent and child container): it had a lot of related limitations as well, but that was the big one. This is not enough for a largeish enterprise kernel, even assuming that you share types used by multiple modules to reduce the overall type count. Also, BTF and CTF serve rather different purposes: CTF specifically encodes knowledge about C types, while BTF is specialized for encoding information about BPF maps. You can't use CTF for that: C doesn't even have a map type, nor anything like one, and the bits CTF spends on things like the details of floating-point formats are wasted on BPF. As for the other part... obviously, as someone working on DTrace and using it ever more, I think it does hold value in its own right. It operates at a different level of the stack from BPF, in any case: it's a user-facing tool like a kernel-level awk, which is nothing like BPF. In my opinion, saying that DTrace doesn't hold value because of BPF is like saying that C doesn't hold value because ARM assembly language exists as well as x86 (in this metaphor, C == DTrace, ARM assembly language == BPF, x86 assembly language == DIF, the DTrace intermediate format). It certainly seems possible to replace the DIF portion of DTrace with BPF, but this will not obsolete BPF nor DTrace: instead, DTrace will drop DIF and the DIF interpreter and build on BPF, generating BPF instruction streams the way it now generates DIF instruction streams. We get to improve BPF if needed and drop a redundant interpreter and BPF tracing gets a hopefully-nicer user interface in the shape of DTrace, and wider usage. Win-win! (I'm not doing most of the work on this, so my opinion is far from authoritative, but I did do a preliminary experimental conversion of the hand-rolled DTrace code generator to emit BPF instructions instead, and it seemed perfectly practical: the two encodings are remarkably similar, and BPF is pleasant to generate, as such things go. That's only a small part of what needs doing, of course...)
  - deleted 8y ago
    [deleted]
  - simcop2387 8y ago
    It didn't used to be available under GPLv2, it was originally under the CDDL until late 2017 when Oracle relicensed it. This was years after the Linux kernel community had been working on other solutions that didn't have that restriction.
grumpydba 8y ago
eBPF was created before dtrace was GPL'd. Oracle is to blame there. 100%.
- pjmlp 8y ago
  HN loves to hate Oracle, however CDDL license was created by Sun.
  - grumpydba 8y ago
    Indeed. But sun is oracle now. So yeah oracle is to blame. They have literally fired an employee working on zfs for saying oracle should gpl it. There, one more reason to hate oracle.
    - lawnchair_larry 8y ago
      No, they didn’t. Many reasons to hate them, but that didn’t happen.
      - grumpydba 8y ago
        Well I guess Brian Cantrill is not well informed about oracle and ex sun employees. Or is he?
        pjmlp 8y ago
        He also has an agenda against them, had an agenda against the Java division at Sun as well, so I always take his opinions with a big grain of salt.
        lawnchair_larry 8y ago
        It doesn’t matter. Just think about it for a second. It’s ridiculous. Oracle has plenty of open source code. Merely suggesting that something be open source could not possibly lead to termination. There is obviously way more to that. Whatever reason he was fired for might be unjustified, I have no idea. But it was absolutely not due to suggesting that they open source something. If you’re basing that on this tweet, not only does it come well short of the claim you’re making, but it clearly indicates that Cantrill is going on third party hearsay. https://mobile.twitter.com/bcantrill/status/931342500446351360 https://mobile.twitter.com/bcantrill/status/9313425004463513...
benmmurphy 8y ago
i think eBPF is theoretically faster. eBPF is verified statically while dtrace loads/writes are all checked dynamically. i guess eBPF verifier is a bit scary because a small bug anywhere in the verifier creates a big hole. i think project zero found a lot of bugs in the verifier. its pretty scary putting an interpreter in the kernel. i'm not surprised linux didn't want to have two completely different interpreters in there.

Kudos for Microsoft for not going the Linux NIH way.

by lelf 8y ago

Kudos for Microsoft for not going the Linux NIH way.

NIH would imply they had a real choice. Sadly due to licensing, they don’t. Edit: I stand corrected. GPL should be fine.

lelf 8y ago
It's pure NIH. dtrace is GPLv2. (Also ZFS (CDDL) run fine on Linux.)
- josteink 8y ago
  Thanks for the correction. I thought/assumed dtrace was CDDL too. I know ZFS works fine on Linux, because I use it everywhere, but there’s no way around the fact that the incompatible licenses are causing issues.
  - johannkokos 8y ago
    You're not wrong. Dtrace is gpl only in 2017.
- kapilvt 8y ago
  dtrace is now gplv2 (2018, ~10 years after the acquisition), it wasn't for a long time (and its still a separate license around userspace), because oracle was holding it as a competitive advantage I presume for unbreakable linux distro (they ported dtrace over to unbreakable linux ~2012), the rest of the linux community continued with other projects (system tap, ktap, lttng, ftrace, etc), before settling in over the last 24 months onto building onto of bpf (see https://github.com/iovisor/bpftrace https://github.com/iovisor/bpftrace for a high level dsl interface), at which point oracle GPL'd dtrace, ie when it held no value, because everyone had moved on.
  - lelf 8y ago
    Again, even when it was CDDL it wasn't a real problem. See ZFS. > system tap, ktap, lttng, ftrace, etc That's why I'm talking about NIH. Even with ebfp, they intentionally choose different tracing syntax (just look at it, it's comical). Didn't use ready-to-use CTF. Etc etc etc.
    - yellowapple 8y ago
      > See ZFS ZFS is still problematic, specifically because of the GPL-incompatible license. Various distros have found ways around that (or are deliberately ignoring those issues at their own peril), but it's still very much "a real problem", and it took years (if not decades) to get to the point where ZFS on Linux is actually as viable an option as it is today.
    - aseipp 8y ago
      You're deluding yourself if you think the CDDL hasn't been a major hurdle against ZFS success in the Linux world. ZFS isn't successful in Linux because the CDDL doesn't matter, it's successful despite the fact it does matter because people put a lot more work into it than otherwise needed and work with the licensing and packaging issues involved with that. But saying "it isn't a real problem" is just factually false, no matter how much you don't like that. If the CDDL simply didn't matter, after all, they could just add it to the source tree and be done with it. I wonder why they haven't done that...
    - BurningCycles 8y ago
      >Again, even when it was CDDL it wasn't a real problem. See ZFS. ZFS is not in the Linux kernel, and Linus has stated that he will never merge CDDL licensed code due to license incompability. Those Linux distros that distribute ZFS do so as a separate module, not part of the Linux kernel. DTrace was GPL licensed too late (2018), something even Brendan Gregg stated as he finds eBPF more capable (and he is a leading expert on DTrace).
  - nullnix 8y ago
    DTrace for Linux CTF hacker here. The BPF people cannot be blamed for not using CTF. Even if it had been relicensed to the GPLv2 way back in 2005, until recently (https://github.com/oracle/libdtrace-ctf https://github.com/oracle/libdtrace-ctf release 1.0 or 1.1) it was impractical to use CTF on larger projects because the file format could only encode a strictly limited number of types (2^15 in each of a parent and child container): it had a lot of related limitations as well, but that was the big one. This is not enough for a largeish enterprise kernel, even assuming that you share types used by multiple modules to reduce the overall type count. Also, BTF and CTF serve rather different purposes: CTF specifically encodes knowledge about C types, while BTF is specialized for encoding information about BPF maps. You can't use CTF for that: C doesn't even have a map type, nor anything like one, and the bits CTF spends on things like the details of floating-point formats are wasted on BPF. As for the other part... obviously, as someone working on DTrace and using it ever more, I think it does hold value in its own right. It operates at a different level of the stack from BPF, in any case: it's a user-facing tool like a kernel-level awk, which is nothing like BPF. In my opinion, saying that DTrace doesn't hold value because of BPF is like saying that C doesn't hold value because ARM assembly language exists as well as x86 (in this metaphor, C == DTrace, ARM assembly language == BPF, x86 assembly language == DIF, the DTrace intermediate format). It certainly seems possible to replace the DIF portion of DTrace with BPF, but this will not obsolete BPF nor DTrace: instead, DTrace will drop DIF and the DIF interpreter and build on BPF, generating BPF instruction streams the way it now generates DIF instruction streams. We get to improve BPF if needed and drop a redundant interpreter and BPF tracing gets a hopefully-nicer user interface in the shape of DTrace, and wider usage. Win-win! (I'm not doing most of the work on this, so my opinion is far from authoritative, but I did do a preliminary experimental conversion of the hand-rolled DTrace code generator to emit BPF instructions instead, and it seemed perfectly practical: the two encodings are remarkably similar, and BPF is pleasant to generate, as such things go. That's only a small part of what needs doing, of course...)
- deleted 8y ago
  [deleted]
- simcop2387 8y ago
  It didn't used to be available under GPLv2, it was originally under the CDDL until late 2017 when Oracle relicensed it. This was years after the Linux kernel community had been working on other solutions that didn't have that restriction.

grumpydba 8y ago

eBPF was created before dtrace was GPL'd. Oracle is to blame there. 100%.

pjmlp 8y ago
HN loves to hate Oracle, however CDDL license was created by Sun.
- grumpydba 8y ago
  Indeed. But sun is oracle now. So yeah oracle is to blame. They have literally fired an employee working on zfs for saying oracle should gpl it. There, one more reason to hate oracle.
  - lawnchair_larry 8y ago
    No, they didn’t. Many reasons to hate them, but that didn’t happen.
    - grumpydba 8y ago
      Well I guess Brian Cantrill is not well informed about oracle and ex sun employees. Or is he?
      - pjmlp 8y ago
        He also has an agenda against them, had an agenda against the Java division at Sun as well, so I always take his opinions with a big grain of salt.
      - lawnchair_larry 8y ago
        It doesn’t matter. Just think about it for a second. It’s ridiculous. Oracle has plenty of open source code. Merely suggesting that something be open source could not possibly lead to termination. There is obviously way more to that. Whatever reason he was fired for might be unjustified, I have no idea. But it was absolutely not due to suggesting that they open source something. If you’re basing that on this tweet, not only does it come well short of the claim you’re making, but it clearly indicates that Cantrill is going on third party hearsay. https://mobile.twitter.com/bcantrill/status/931342500446351360 https://mobile.twitter.com/bcantrill/status/9313425004463513...

benmmurphy 8y ago

i think eBPF is theoretically faster. eBPF is verified statically while dtrace loads/writes are all checked dynamically. i guess eBPF verifier is a bit scary because a small bug anywhere in the verifier creates a big hole. i think project zero found a lot of bugs in the verifier. its pretty scary putting an interpreter in the kernel. i'm not surprised linux didn't want to have two completely different interpreters in there.