3 ms·

Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel

Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Seems like right now you need to compile a kernel module which will taint your standard kernel in order to use it.

gbrown_ 10y ago
> Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Never. Due to dtrace being licensed under the CDDL it will never be mainlined into the upstream kernel. I would be keen to point out this does not mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS kernel module in recent releases of Ubuntu. I shall say no more on the matter as a refuse to be drawn into a licensing debate... Getting back on topic, those seeking more on user space providers should check out this video from Bryan Cantrill. https://www.youtube.com/watch?v=6chLw2aodYQ https://www.youtube.com/watch?v=6chLw2aodYQ Full disclosure I never had to run Solaris in production so I can look back on Sun with rose tinted glasses. That said I still long for many of the features it pioneered. Seriously ZFS would eliminate half of my production problems and dtrace would make the remaining half solvable.
- e12e 10y ago
  > I would be keen to point out this does not mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS kernel module in recent releases of Ubuntu. AFAIK it's only packaged as a dkms-package: Package: zfs-dkms Version: 0.6.5.6-0ubuntu8 Priority: extra Section: universe/kernel Source: zfs-linux Origin: Ubuntu Maintainer: Ubuntu Developers... Original-Maintainer: Darik Horn ... Bugs: https://bugs.launchpad... Installed-Size: 8 325 kB Provides: lustre-backend-fs, lzfs, lzfs-dkms Pre-Depends: spl-dkms (>= 0.6.5.6) Depends: dkms (>= 2.2.0.2), zfs-doc (= 0.6.5.6-0ubuntu8) Conflicts: lzfs, lzfs-dkms Replaces: lzfs, lzfs-dkms (...) xenial/universe amd64 Packages Note that this is in "universe" not Ubuntu proper. From DKMS(8): dkms is a framework which allows kernel modules to be dynamically built for each kernel on your system in a simplified and organized fashion. > I shall say no more on the matter as a refuse to be drawn into a licensing debate... I really don't want to start that debate, but I also think the "Ubuntu distributes (binary) ZFS" isn't quite accurate. Now, how this all shakes out in terms of whether or not ZFS is bundled with Ubuntu ("on the ISO so to speak"), and if it then really makes a difference that it's distributed as source-code that can be automatically compiled -- I don't know. I wouldn't think shipping pythons source (to be later compiled to byte code, pyc) is enough to dodge GPL restrictions on bundling other code/binaries as part of a distribution. But, yeah, whichever it is, CDDL code isn't likely to ever be upstreamed to the kernel proper.
  - lathiat 10y ago
    That's actually only half the story and mostly not relevant... the dkms package is still there, but ZFS is now shipped with the kernel package: lathiat@ubuntu:~$ dpkg -S zfs.ko linux-image-4.4.0-35-generic: /lib/modules/4.4.0-35-generic/kernel/zfs/zfs/zfs.ko linux-image-4.4.0-36-generic: /lib/modules/4.4.0-36-generic/kernel/zfs/zfs/zfs.ko Announcement is here: https://insights.ubuntu.com/2016/02/16/zfs-is-the-fs-for-containers-in-ubuntu-16-04/ https://insights.ubuntu.com/2016/02/16/zfs-is-the-fs-for-con... There is a lot of debate about the licensing legality of it, and there are opposing positions from various parties. Here's the official post summarising it: http://blog.dustinkirkland.com/2016/02/zfs-licensing-and-linux.html http://blog.dustinkirkland.com/2016/02/zfs-licensing-and-lin...
    - e12e 10y ago
      Thank you for taking the time to reply with those two links. Very interesting (and directly helpful as one of my use-cases is led+zfs. Not needing the dkms module makes things simpler).
- brendangregg 10y ago
  That's Linux today. Upgrade. ZFS + DTrace capabilities.
vasili111 10y ago
FreeBSD already has DTrace :)
brendangregg 10y ago
Enhanced BPF is integrated in Linux 4.x series -- as in, it is part of the kernel, and everyone who deploys on Linux will be getting it once they upgrade -- and provides similar capabilities.
- lathiat 10y ago
  I have not yet looked at the new BPF functionality at all, so I desperately hope we are heading towards something more integrated. I really should set aside some time to catch up on some of the recent presentations about it... including yours Brendan :) The real shame of linux tracing utilities from my time using them, is that there is a large disjointed set of tools and frameworks that you need to use each in different circumstances. And there is no single language for using those tools and then processing the output. The great thing about dtrace, from using it personally on Solaris, is that it is a single integrated tool where 90% of the time you can get sufficiently summarised output from a single file dtrace script to debug most things. And this lends itself to sharing those scripts for re-use by others, which I have done extensively working on ZFS, iSCSI and NFS. This also makes it much easier for the less informed to take those scripts and run with them into their own modifications, rather than needing to figure a lot more out from the outset. This doesn't seem overly surprising based on the way the linux kernel is developed, by way of multiple parties mostly implementing there own interests and large projects not commonly being planned out in a co-ordinated fashion. You also have the luxury of being able to view, modify and run your own kernel with custom changes to debug these problems if you need to. By contrast, on Solaris generally you were stuck with what you got from SunOracle and probably didn't have source or the ability to hack your own modifications in... and there was great business case for them to create a single tooling from the outset.
- gbrown_ 10y ago
  Unfortunately it will be years before myself and many others get to use BPF on our production systems as we have to run an "enterprise" distribution like RHEL or god forbid SLES. But yes it certainly is good to see steps being made in this direction.
Philipp__ 10y ago
Just give FreeBSD a shot!

gbrown_ 10y ago

> Does anyone have any inside knowledge of when, if ever, DTrace might become a standard/official part of Linux? Never. Due to dtrace being licensed under the CDDL it will never be mainlined into the upstream kernel. I would be keen to point out this does not mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS kernel module in recent releases of Ubuntu. I shall say no more on the matter as a refuse to be drawn into a licensing debate... Getting back on topic, those seeking more on user space providers should check out this video from Bryan Cantrill. https://www.youtube.com/watch?v=6chLw2aodYQ https://www.youtube.com/watch?v=6chLw2aodYQ Full disclosure I never had to run Solaris in production so I can look back on Sun with rose tinted glasses. That said I still long for many of the features it pioneered. Seriously ZFS would eliminate half of my production problems and dtrace would make the remaining half solvable.

e12e 10y ago
> I would be keen to point out this does not mean there is a conflict of licenses from a legal point of view. Before anyone starts making claims like "you're not allowed to do this", see the use of the ZFS kernel module in recent releases of Ubuntu. AFAIK it's only packaged as a dkms-package: Package: zfs-dkms Version: 0.6.5.6-0ubuntu8 Priority: extra Section: universe/kernel Source: zfs-linux Origin: Ubuntu Maintainer: Ubuntu Developers... Original-Maintainer: Darik Horn ... Bugs: https://bugs.launchpad... Installed-Size: 8 325 kB Provides: lustre-backend-fs, lzfs, lzfs-dkms Pre-Depends: spl-dkms (>= 0.6.5.6) Depends: dkms (>= 2.2.0.2), zfs-doc (= 0.6.5.6-0ubuntu8) Conflicts: lzfs, lzfs-dkms Replaces: lzfs, lzfs-dkms (...) xenial/universe amd64 Packages Note that this is in "universe" not Ubuntu proper. From DKMS(8): dkms is a framework which allows kernel modules to be dynamically built for each kernel on your system in a simplified and organized fashion. > I shall say no more on the matter as a refuse to be drawn into a licensing debate... I really don't want to start that debate, but I also think the "Ubuntu distributes (binary) ZFS" isn't quite accurate. Now, how this all shakes out in terms of whether or not ZFS is bundled with Ubuntu ("on the ISO so to speak"), and if it then really makes a difference that it's distributed as source-code that can be automatically compiled -- I don't know. I wouldn't think shipping pythons source (to be later compiled to byte code, pyc) is enough to dodge GPL restrictions on bundling other code/binaries as part of a distribution. But, yeah, whichever it is, CDDL code isn't likely to ever be upstreamed to the kernel proper.
- lathiat 10y ago
  That's actually only half the story and mostly not relevant... the dkms package is still there, but ZFS is now shipped with the kernel package: lathiat@ubuntu:~$ dpkg -S zfs.ko linux-image-4.4.0-35-generic: /lib/modules/4.4.0-35-generic/kernel/zfs/zfs/zfs.ko linux-image-4.4.0-36-generic: /lib/modules/4.4.0-36-generic/kernel/zfs/zfs/zfs.ko Announcement is here: https://insights.ubuntu.com/2016/02/16/zfs-is-the-fs-for-containers-in-ubuntu-16-04/ https://insights.ubuntu.com/2016/02/16/zfs-is-the-fs-for-con... There is a lot of debate about the licensing legality of it, and there are opposing positions from various parties. Here's the official post summarising it: http://blog.dustinkirkland.com/2016/02/zfs-licensing-and-linux.html http://blog.dustinkirkland.com/2016/02/zfs-licensing-and-lin...
  - e12e 10y ago
    Thank you for taking the time to reply with those two links. Very interesting (and directly helpful as one of my use-cases is led+zfs. Not needing the dkms module makes things simpler).
brendangregg 10y ago
That's Linux today. Upgrade. ZFS + DTrace capabilities.

vasili111 10y ago

FreeBSD already has DTrace :)

brendangregg 10y ago

Enhanced BPF is integrated in Linux 4.x series -- as in, it is part of the kernel, and everyone who deploys on Linux will be getting it once they upgrade -- and provides similar capabilities.

lathiat 10y ago
I have not yet looked at the new BPF functionality at all, so I desperately hope we are heading towards something more integrated. I really should set aside some time to catch up on some of the recent presentations about it... including yours Brendan :) The real shame of linux tracing utilities from my time using them, is that there is a large disjointed set of tools and frameworks that you need to use each in different circumstances. And there is no single language for using those tools and then processing the output. The great thing about dtrace, from using it personally on Solaris, is that it is a single integrated tool where 90% of the time you can get sufficiently summarised output from a single file dtrace script to debug most things. And this lends itself to sharing those scripts for re-use by others, which I have done extensively working on ZFS, iSCSI and NFS. This also makes it much easier for the less informed to take those scripts and run with them into their own modifications, rather than needing to figure a lot more out from the outset. This doesn't seem overly surprising based on the way the linux kernel is developed, by way of multiple parties mostly implementing there own interests and large projects not commonly being planned out in a co-ordinated fashion. You also have the luxury of being able to view, modify and run your own kernel with custom changes to debug these problems if you need to. By contrast, on Solaris generally you were stuck with what you got from SunOracle and probably didn't have source or the ability to hack your own modifications in... and there was great business case for them to create a single tooling from the outset.
gbrown_ 10y ago
Unfortunately it will be years before myself and many others get to use BPF on our production systems as we have to run an "enterprise" distribution like RHEL or god forbid SLES. But yes it certainly is good to see steps being made in this direction.

Philipp__ 10y ago

Just give FreeBSD a shot!