14 ms·

Dtrace for Linux

354 points by espadrine 9y ago

bcantrill 9y ago
I had somehow missed this; this is excellent news! The Oracle port of DTrace to Linux is (far and away) the most complete port; Kris van Hees and team did a very thorough job[1], and I'm elated (and honestly, surprised) that they somehow prevailed on Oracle to do the right thing here! [1] https://www.youtube.com/watch?v=NElog3MvUC8 https://www.youtube.com/watch?v=NElog3MvUC8
- rhinoceraptor 9y ago
  If ZFS were also relicensed, would Illumos be able to relicense OpenZFS as well, even with the divergence?
  - Conan_Kudo 9y ago
    If they can get in touch with the developers or their employers, then yes. OpenZFS could be fully relicensed as the main reason for not doing it is because of the substantial Oracle ZFS base code not being licensed appropriately.
    - mjg59 9y ago
      The majority of OpenZFS is under CDDL 1.0 or later, so Oracle can relicense it by releasing a new version of CDDL that permits relicensing any ZFS-related code to GPLv2. Wikipedia and the FSF did something similar to relicense Wikipedia from GFDL to CC.
  - jsiepkes 9y ago
    If Oracle's sources would be GPL'ed all OpenZFS changes would also have to be put under the GPL (or dual licensed) if you want to use those. OpenZFS doesn't have a CLA I think so that might be somewhat of a problem. Then again most ZFS changes were made by people for their employers (Joyent, Delphix, etc.) so it might be that only a few companies (...and Allan Jude ;-) need to give their blessing.
    - takeda 9y ago
      I don't think the license can be done retroactively. I mean Oracle owns copyright to major part of ZFS and they can change license to that. But the contributions done to OpenZFS each developer owns copyright to their own work. I believe the original code could be treated as either CDDL or GPL, but the code that was contributed before license change would still be under CDDL. I think OpenZFS project owners would need to track down every contributor and get their permission to change the license. Perhaps it wouldn't be terribly hard with git/svn blame command. The only problem would be if the original authors were no longer available, but hopefully that code could be just rewritten by someone else. Things would be much easier if every author would surrender their copyright to OpenZFS project, but some people might have problem with it since then you surrender all claims to the code.
      - Conan_Kudo 9y ago
        > Things would be much easier if every author would surrender their copyright to OpenZFS project, but some people might have problem with it since then you surrender all claims to the code. This is super-dangerous. There's a reason that copyright assignment agreements are rare. In a lot of jurisdictions, it's not even possible. Most copyright license agreements instead force unlimited sublicensing grants, which provides de facto ownership of the copyrighted code, rather than de jure ownership.
        nordsieck 9y ago
        > There's a reason that copyright assignment agreements are rare. In a lot of jurisdictions, it's not even possible. How is a software development business possible under those circumstances? In the US one of the core pieces of every employment contract (for software developers) is IP assignment.
        vetinari 9y ago
        IP assignment itself is a bit of hand-wavy term, that hides a lot. Those jurisdictions, where you cannot assign authorship, make distinction between being an author and being able to distribute a copyrighted work. It you wrote something, you will be forever the author and you cannot transfer the authorship. What you can transfer is the rights to distribution.
  - 4ad 9y ago
    OpenZFS can't be relicensed as GPL since FreeBSD won't be able to use it. Perhaps it could be dual-licensed, not sure, but that would be bad for FreeBSD as well. If it would be integrated into the mainline Linux kernel, I don't believe FreeBSD could merge back subsequent ZFS changes done on the Linux side. If OpenZFS were relincensed and developed outside the mainline Linux tree (as it is now), FreeBSD would be able to get all changes, but it would still be annoying to actually use ZFS in Linux for the same reason it is annoying to use it now, even though the legal uncertainties would disappear.
    - insertnickname 9y ago
      FreeBSD is OK with CDDL but not GPL?
      - boomboomsubban 9y ago
        Yeah, the CDDL is weak copyleft, meaning derived works don't need to be licensed CDDL. The GPL is strong copyleft, and could force all of FreeBSD to adopt the GPL.
        LukeShu 9y ago
        To clarify that a bit: The CDDL is a per-file copyleft; you must provide the sources of the CDDL-licensed source files. The GPL is a per-program copyleft; you must provide the sources of the entire program if it contains GPL-licensed code; the source of the entire program effectively becomes GPL-licensed if you incorporate GPL-licensed code. If I made a closed derivative of the FreeBSD kernel, I'd have to provide the (CDDL) ZFS sources, but nothing else. If ZFS were GPL, I'd have to provide the sources to the entire kernel.
      - deleted 9y ago
        [deleted]
    - simcop2387 9y ago
      Instead of relicensing to gpl it could be relicensed to bsd or MIT both of which can be included in mainline Linux kernel and in bsd land too, without compromising future development edit: stupid phone autocorrect on the train this morning. s/not/MIT/
paride5745 9y ago
Time for ZFS now! Even better yet, a new OpenSolaris under GPL instead of CDDL.
- Annatar 9y ago
  That day better never come; If the last thing I dearly love, illumos (there is no OpenSolaris any more) ever gets relicensed under that fascist GPL, I’ll never write another line of code again. That would be the last straw.
  - gribbly 9y ago
    I can understand why someone does not like GPL, but calling a license which says you have to pass on the same rights you were given a 'fascist' license makes absolutely zero sense.
    - _ph_ 9y ago
      The issue with the GPL is, that it does not only requires to pass on the source any (modified) GPL component, but also any other program part which happens to be linked together with the GPL code, but is in no way derived from it. This is pretty violent towards code not strictly related to the GPL code - whether the strong term used is appropriate, is another question.
- snw 9y ago
  The new OpenSolaris is called illumos. And the CDDL is actually fine. Give it a try :)
  - paride5745 9y ago
    I use OpenIndiana since its beginning, it's fine but IMHO not there for serious work. FreeBSD with ZFS is miles ahead. And the CDDL is not fine, as it is not compatible with GPL, which makes it impossible to have it official in the Linux kernel.
jsiepkes 9y ago
Really cool! However isn't it a bit too late for DTrace to gain market share on Linux now SystemTap and all have gotten such a head start? For me personally I like the idea of being able to use my DTrace skills (which I gained on FreeBSD and SmartOS) on Linux!
- loeg 9y ago
  It would be very nice to be able to use the same Dtrace tracing skills and interface across a variety of modern Unix systems. I hope it isn't too late.
- severino 9y ago
  Well, it's never too late! I mean, when reading articles like this one: http://www.brendangregg.com/blog/2015-07-08/choosing-a-linux-tracer.html http://www.brendangregg.com/blog/2015-07-08/choosing-a-linux... Linux tracing looks like a mess, where there're dozens of incompatible alternatives, each one with its gotchas. But instead, dtrace is a mature standard framework which also works under BSD or macOS. You got one programming language, D, and tons of resources out there. So... my best wishes for this.
  - lscotte 9y ago
    It was a mess back then - in 2015. Instead, it's probably more useful to look at at where things are today rather than assuming things are the same as they were a few years ago. Things move fast in this space - for example, just a year later Brendan wrote this on eBPF: http://brendangregg.com/blog/2016-10-27/dtrace-for-linux-2016.html http://brendangregg.com/blog/2016-10-27/dtrace-for-linux-201...
brendangregg 9y ago
Unfortunately for DTrace, this is too late. Oracle should have done this years ago. Now Linux has a more powerful tracer builtin, eBPF, and it would be a backwards step to switch the kernel code to DTrace (assuming the DTrace port is completed, which it is not). I'm sure this will not be lost on the maintainers, who have the ultimate say as to what is included in Linux mainline. The only hope for DTrace is to have the frontend emit BPF bytecode. The bulk of this GPL DTrace code is no longer needed, only the user-level front end.
- burke 9y ago
  As a non-expert, I still find this exciting. While eBPF may be more powerful, I find it far less approachable than DTrace. Perhaps this is just a failure to discover the right documentation. I use DTrace now and then on macOS and have resented that I can't really use it on Linux as well.
  - byefruit 9y ago
    With regards to tracing eBPF has actually moved at a fairly brisk pace the last year or two and it will probably take a while for the documentation and knowledge to permeate. The usability side you mention is what Brendan's suggesting is worth porting from DTrace, just the front-end that the users sees. That would just back onto eBPF and the user would know little different.
  - brendangregg 9y ago
    eBPF lacks a high level front end, in part because that's optional. Engineering effort has gone into the essentials first: kernel development of eBPF, basic user-level frameworks to use it (bcc), and lots and lots of testing and bug fixes to make that rock solid. bcc is not as easy to code in, but I've been able to port over many of my DTraceToolkit scripts already. We're only more recently looking at higher level languages, as a nice to have. Here's one project: https://github.com/ajor/bpftrace https://github.com/ajor/bpftrace We can also look at taking the /usr/sbin/dtrace code (licenses permitting), and calling into libbcc and emitting BPF bytecode. That'd give us a D language front-end to BPF, so I can run my old DTrace scripts directly (plus sell more copies of my DTrace book -- I joke ;-) Help on either of those projects would be appreciated (bpftrace or DTrace-BPF).
benmmurphy 9y ago
it looks like the dtrace port can be used by non-privileged users (i'm just guessing because there is some uid checks in there). just be aware there has been a bunch of fixes applied to the illumos version of dtrace and i'm pretty sure at least 1 (maybe more) have not been applied to this version.
alberth 9y ago
What does this mean for DTrace on FreeBSD? Given that DTrace is now GPL'ed, does that kill DTrace on FreeBSD? https://wiki.freebsd.org/DTrace https://wiki.freebsd.org/DTrace
- floatboth 9y ago
  You can't retroactively revoke CDDL :) And new DTrace userspace is actually under the Universal Permissive License now (which is, well, permissive). The Linux kernel module is GPL. I don't think FreeBSD needs anything from that module at all.
  - stock_toaster 9y ago
    The UPL license also has some somewhat "unique" patent grant wording with regard to a "Larger Works" file, which can even reference external projects by name, and its supposed use as a CLA document. Not being a lawyer (and similarly not a licensing or IP subject matter expert), and know the license was drafted by Oracle, I am somewhat suspicious of it.
rurban 9y ago
Thanks God, eBPF is a security nightmare as we saw with Spectre, they even got arrays now. and there is no infrastructure as with dtrace. Will compile it into my kernel for sure for proper dev work.
- detaro 9y ago
  Why is the dtrace language not vulnerable in the same way?
  - rurban 9y ago
    Because dtrace does not have the security holes eBPF has right now. dtrace only enables a bit (e.g. only a hash, no arrays), eBPF on the other hand tries to disable the dangerous stuff, which is always a lost cause.
- chatmasta 9y ago
  Doesn't this criticism apply to any software that opens a communications channel between the kernel and user space? Also, how does eBPF relate to Spectre?
  - rurban 9y ago
    Yes, but syscalls are properly designed. drivers are a huge problem, always have been. eBPF on the other hand lacks proper security design. They added it afterwards, to some extent. > how does eBPF relate to Spectre? Please read any spectre paper. Besides the known javascript attacks, eBPF is the easiest way to bypass kernel ASLR. google is your friend.
luvpreet33 9y ago
Have you tried Sysdig? https://github.com/draios/sysdig https://github.com/draios/sysdig
krylon 9y ago
This is good news, I think. I have heard so many people praise DTrace, but it not being available on Linux has so far kept me from looking at it. I have one FreeBSD box at home, one (ancient) laptop running OpenBSD, but all my other machines run various Linux distros. So there was little point for me to spend any time learning it. Now there is. :D I think I know what I will be doing with my next vacation.

354 points by espadrine 9y ago

I had somehow missed this; this is excellent news! The Oracle port of DTrace to Linux is (far and away) the most complete port; Kris van Hees and team did a very thorough job[1], and I'm elated (and honestly, surprised) that they somehow prevailed on Oracle to do the right thing here! [1] https://www.youtube.com/watch?v=NElog3MvUC8 https://www.youtube.com/watch?v=NElog3MvUC8

rhinoceraptor 9y ago
If ZFS were also relicensed, would Illumos be able to relicense OpenZFS as well, even with the divergence?
- Conan_Kudo 9y ago
  If they can get in touch with the developers or their employers, then yes. OpenZFS could be fully relicensed as the main reason for not doing it is because of the substantial Oracle ZFS base code not being licensed appropriately.
  - mjg59 9y ago
    The majority of OpenZFS is under CDDL 1.0 or later, so Oracle can relicense it by releasing a new version of CDDL that permits relicensing any ZFS-related code to GPLv2. Wikipedia and the FSF did something similar to relicense Wikipedia from GFDL to CC.
- jsiepkes 9y ago
  If Oracle's sources would be GPL'ed all OpenZFS changes would also have to be put under the GPL (or dual licensed) if you want to use those. OpenZFS doesn't have a CLA I think so that might be somewhat of a problem. Then again most ZFS changes were made by people for their employers (Joyent, Delphix, etc.) so it might be that only a few companies (...and Allan Jude ;-) need to give their blessing.
  - takeda 9y ago
    I don't think the license can be done retroactively. I mean Oracle owns copyright to major part of ZFS and they can change license to that. But the contributions done to OpenZFS each developer owns copyright to their own work. I believe the original code could be treated as either CDDL or GPL, but the code that was contributed before license change would still be under CDDL. I think OpenZFS project owners would need to track down every contributor and get their permission to change the license. Perhaps it wouldn't be terribly hard with git/svn blame command. The only problem would be if the original authors were no longer available, but hopefully that code could be just rewritten by someone else. Things would be much easier if every author would surrender their copyright to OpenZFS project, but some people might have problem with it since then you surrender all claims to the code.
    - Conan_Kudo 9y ago
      > Things would be much easier if every author would surrender their copyright to OpenZFS project, but some people might have problem with it since then you surrender all claims to the code. This is super-dangerous. There's a reason that copyright assignment agreements are rare. In a lot of jurisdictions, it's not even possible. Most copyright license agreements instead force unlimited sublicensing grants, which provides de facto ownership of the copyrighted code, rather than de jure ownership.
      - nordsieck 9y ago
        > There's a reason that copyright assignment agreements are rare. In a lot of jurisdictions, it's not even possible. How is a software development business possible under those circumstances? In the US one of the core pieces of every employment contract (for software developers) is IP assignment.
        vetinari 9y ago
        IP assignment itself is a bit of hand-wavy term, that hides a lot. Those jurisdictions, where you cannot assign authorship, make distinction between being an author and being able to distribute a copyrighted work. It you wrote something, you will be forever the author and you cannot transfer the authorship. What you can transfer is the rights to distribution.
- 4ad 9y ago
  OpenZFS can't be relicensed as GPL since FreeBSD won't be able to use it. Perhaps it could be dual-licensed, not sure, but that would be bad for FreeBSD as well. If it would be integrated into the mainline Linux kernel, I don't believe FreeBSD could merge back subsequent ZFS changes done on the Linux side. If OpenZFS were relincensed and developed outside the mainline Linux tree (as it is now), FreeBSD would be able to get all changes, but it would still be annoying to actually use ZFS in Linux for the same reason it is annoying to use it now, even though the legal uncertainties would disappear.
  - insertnickname 9y ago
    FreeBSD is OK with CDDL but not GPL?
    - boomboomsubban 9y ago
      Yeah, the CDDL is weak copyleft, meaning derived works don't need to be licensed CDDL. The GPL is strong copyleft, and could force all of FreeBSD to adopt the GPL.
      - LukeShu 9y ago
        To clarify that a bit: The CDDL is a per-file copyleft; you must provide the sources of the CDDL-licensed source files. The GPL is a per-program copyleft; you must provide the sources of the entire program if it contains GPL-licensed code; the source of the entire program effectively becomes GPL-licensed if you incorporate GPL-licensed code. If I made a closed derivative of the FreeBSD kernel, I'd have to provide the (CDDL) ZFS sources, but nothing else. If ZFS were GPL, I'd have to provide the sources to the entire kernel.
    - deleted 9y ago
      [deleted]
  - simcop2387 9y ago
    Instead of relicensing to gpl it could be relicensed to bsd or MIT both of which can be included in mainline Linux kernel and in bsd land too, without compromising future development edit: stupid phone autocorrect on the train this morning. s/not/MIT/

paride5745 9y ago

Time for ZFS now! Even better yet, a new OpenSolaris under GPL instead of CDDL.

Annatar 9y ago
That day better never come; If the last thing I dearly love, illumos (there is no OpenSolaris any more) ever gets relicensed under that fascist GPL, I’ll never write another line of code again. That would be the last straw.
- gribbly 9y ago
  I can understand why someone does not like GPL, but calling a license which says you have to pass on the same rights you were given a 'fascist' license makes absolutely zero sense.
  - _ph_ 9y ago
    The issue with the GPL is, that it does not only requires to pass on the source any (modified) GPL component, but also any other program part which happens to be linked together with the GPL code, but is in no way derived from it. This is pretty violent towards code not strictly related to the GPL code - whether the strong term used is appropriate, is another question.
snw 9y ago
The new OpenSolaris is called illumos. And the CDDL is actually fine. Give it a try :)
- paride5745 9y ago
  I use OpenIndiana since its beginning, it's fine but IMHO not there for serious work. FreeBSD with ZFS is miles ahead. And the CDDL is not fine, as it is not compatible with GPL, which makes it impossible to have it official in the Linux kernel.

jsiepkes 9y ago

Really cool! However isn't it a bit too late for DTrace to gain market share on Linux now SystemTap and all have gotten such a head start? For me personally I like the idea of being able to use my DTrace skills (which I gained on FreeBSD and SmartOS) on Linux!

loeg 9y ago
It would be very nice to be able to use the same Dtrace tracing skills and interface across a variety of modern Unix systems. I hope it isn't too late.
severino 9y ago
Well, it's never too late! I mean, when reading articles like this one: http://www.brendangregg.com/blog/2015-07-08/choosing-a-linux-tracer.html http://www.brendangregg.com/blog/2015-07-08/choosing-a-linux... Linux tracing looks like a mess, where there're dozens of incompatible alternatives, each one with its gotchas. But instead, dtrace is a mature standard framework which also works under BSD or macOS. You got one programming language, D, and tons of resources out there. So... my best wishes for this.
- lscotte 9y ago
  It was a mess back then - in 2015. Instead, it's probably more useful to look at at where things are today rather than assuming things are the same as they were a few years ago. Things move fast in this space - for example, just a year later Brendan wrote this on eBPF: http://brendangregg.com/blog/2016-10-27/dtrace-for-linux-2016.html http://brendangregg.com/blog/2016-10-27/dtrace-for-linux-201...

brendangregg 9y ago

Unfortunately for DTrace, this is too late. Oracle should have done this years ago. Now Linux has a more powerful tracer builtin, eBPF, and it would be a backwards step to switch the kernel code to DTrace (assuming the DTrace port is completed, which it is not). I'm sure this will not be lost on the maintainers, who have the ultimate say as to what is included in Linux mainline. The only hope for DTrace is to have the frontend emit BPF bytecode. The bulk of this GPL DTrace code is no longer needed, only the user-level front end.

burke 9y ago
As a non-expert, I still find this exciting. While eBPF may be more powerful, I find it far less approachable than DTrace. Perhaps this is just a failure to discover the right documentation. I use DTrace now and then on macOS and have resented that I can't really use it on Linux as well.
- byefruit 9y ago
  With regards to tracing eBPF has actually moved at a fairly brisk pace the last year or two and it will probably take a while for the documentation and knowledge to permeate. The usability side you mention is what Brendan's suggesting is worth porting from DTrace, just the front-end that the users sees. That would just back onto eBPF and the user would know little different.
- brendangregg 9y ago
  eBPF lacks a high level front end, in part because that's optional. Engineering effort has gone into the essentials first: kernel development of eBPF, basic user-level frameworks to use it (bcc), and lots and lots of testing and bug fixes to make that rock solid. bcc is not as easy to code in, but I've been able to port over many of my DTraceToolkit scripts already. We're only more recently looking at higher level languages, as a nice to have. Here's one project: https://github.com/ajor/bpftrace https://github.com/ajor/bpftrace We can also look at taking the /usr/sbin/dtrace code (licenses permitting), and calling into libbcc and emitting BPF bytecode. That'd give us a D language front-end to BPF, so I can run my old DTrace scripts directly (plus sell more copies of my DTrace book -- I joke ;-) Help on either of those projects would be appreciated (bpftrace or DTrace-BPF).

benmmurphy 9y ago

it looks like the dtrace port can be used by non-privileged users (i'm just guessing because there is some uid checks in there). just be aware there has been a bunch of fixes applied to the illumos version of dtrace and i'm pretty sure at least 1 (maybe more) have not been applied to this version.

alberth 9y ago

What does this mean for DTrace on FreeBSD? Given that DTrace is now GPL'ed, does that kill DTrace on FreeBSD? https://wiki.freebsd.org/DTrace https://wiki.freebsd.org/DTrace

floatboth 9y ago
You can't retroactively revoke CDDL :) And new DTrace userspace is actually under the Universal Permissive License now (which is, well, permissive). The Linux kernel module is GPL. I don't think FreeBSD needs anything from that module at all.
- stock_toaster 9y ago
  The UPL license also has some somewhat "unique" patent grant wording with regard to a "Larger Works" file, which can even reference external projects by name, and its supposed use as a CLA document. Not being a lawyer (and similarly not a licensing or IP subject matter expert), and know the license was drafted by Oracle, I am somewhat suspicious of it.

rurban 9y ago

Thanks God, eBPF is a security nightmare as we saw with Spectre, they even got arrays now. and there is no infrastructure as with dtrace. Will compile it into my kernel for sure for proper dev work.

detaro 9y ago
Why is the dtrace language not vulnerable in the same way?
- rurban 9y ago
  Because dtrace does not have the security holes eBPF has right now. dtrace only enables a bit (e.g. only a hash, no arrays), eBPF on the other hand tries to disable the dangerous stuff, which is always a lost cause.
chatmasta 9y ago
Doesn't this criticism apply to any software that opens a communications channel between the kernel and user space? Also, how does eBPF relate to Spectre?
- rurban 9y ago
  Yes, but syscalls are properly designed. drivers are a huge problem, always have been. eBPF on the other hand lacks proper security design. They added it afterwards, to some extent. > how does eBPF relate to Spectre? Please read any spectre paper. Besides the known javascript attacks, eBPF is the easiest way to bypass kernel ASLR. google is your friend.

luvpreet33 9y ago

Have you tried Sysdig? https://github.com/draios/sysdig https://github.com/draios/sysdig

krylon 9y ago

This is good news, I think. I have heard so many people praise DTrace, but it not being available on Linux has so far kept me from looking at it. I have one FreeBSD box at home, one (ancient) laptop running OpenBSD, but all my other machines run various Linux distros. So there was little point for me to spend any time learning it. Now there is. :D I think I know what I will be doing with my next vacation.