7 ms·

Nobody would love a DTrace/BPF front-end more than I. And not just because I'd sell more copies of my DTrace book (I joke :). It is a really nice language, alth

Nobody would love a DTrace/BPF front-end more than I. And not just because I'd sell more copies of my DTrace book (I joke :). It is a really nice language, although missing a few things that BPF can do that DTrace can't (like saving and retrieving stacks), so it'd need to be enhanced. But with the warning that I'm not a lawyer: before beginning work on a DTrace/BPF front-end, I'd start by talking to a copyright lawyer to see if permission or a license is needed from Oracle. DTrace is Oracle copyrighted, and re-implementing a DTrace front-end on Linux sounds a lot like re-implementing an Oracle-copyrighted API.

Annatar 10y ago
What would stop one from enhancing the DTrace in illumos further? That is licensed under the CDDL, and Adam has made several enhancements to DTrace lately, and if my memory serves me correctly, Bryan fixed a couple of bugs in it recently as well.
- qwertyuiop924 10y ago
  Wouldn't really solve our problem, though: We'd need a new compiler for the language. As I understand it, that would require a almost entirely new codebase. But I don't work on DTrace internals, so I may be wrong.
  - Annatar 10y ago
    What? That doesn't make any sense. How is dtrace(1) built on illumos, then?
    - qwertyuiop924 10y ago
      I'm uncertain, but I thought dtrace on illumos was interpreted. Am I wrong? If I am, than all we'd have to rewrite was the compiler backend, which is much easier, so that would be nice.
      - Annatar 10y ago
        dtrace(1) itself is compiled from C into an ELF binary executable. % file `which dtrace` /usr/sbin/dtrace: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped The DTrace language, D, is interpreted. By DTrace. The problem lies in the fact that neither the GNU/Linux kernel, nor the GNU applications provide DTrace probe points. On Solaris and thus on illumos, and thus on SmartOS, there are tens of thousands of probe points and numerous probe providers. Some external applications, like PostgreSQL or PHP, added DTrace probes, and all is well on Solaris / illumos / SmartOS. Some, like node.js had providers and probes added by engineers at Joyent. http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-ustack-helpers/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-... http://dtrace.org/blogs/blog/category/node-js/ http://dtrace.org/blogs/blog/category/node-js/ GNU/Linux would have to do the same thing. It currently only has but a handful of DTrace probes and providers, which is understandably not very useful. http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/ http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/
        qwertyuiop924 10y ago
        None of that is at all what I meant: allow me to clarify. I was talking about the DTrace language, which I have been avoiding calling D up until this point, so as to avoid confusion with the other D. In this post, when I talk about D, I will be referring to the DTrace language. Linux now has a tracing system called eBPF, which provides many of the same advantages of DTrace. This is what Gregg's blog post was about. However, eBPF requires compiling tracer scripts, or at least parts of the tracer scripts, into bytecode (IIRC). Currently, the bytecode is usually compiled from C code. However, it is ungainly and impractical to write a bunch of C every time you want to run a trace. So I asked if they had plans to support compiling a higher level language. At this point, somebody suggested that somebody should work on compiling the DTrace language, D, to eBPF bytecode. I thought that this would be a good idea, and we were discussing how viable it would be. I thought that you had suggested using Illumos DTrace as a base for this compiler. Since by my impression, Illumos DTrace interprets D, I thought that this would require almost a complete rewrite, and thus it wouldn't very helpful. It seems you meant something else. So what did you mean?
qwertyuiop924 10y ago
Huh. Well than how does Illumos and FreeBSD DTrace exist? They're actually based on the code, yes, but it's the same legally AFAIK (but I'm not a lawyer). Besides, as I understand it, it's the implementation that's copyrighted, not the interface or the idea, which is why Oracle went after Google the way they did, and why things like Freeablo and OpenMW can exist. But again, I'm not a lawyer, so this may be incorrect.
- brendangregg 10y ago
  CDDL is a file based license, so they use the files and follow the license on those files, which grants them rights to use it. The GPL/CDDL license wars over DTrace went on for years. It was a real shame -- we should have been discussing or using a great technology instead. There must be a hundred different ways and languages we can try, some of them may be better than DTrace's D. One of them is Oracle copyrighted, and lacks some features we need (stack saving). I'd either A) talk to a lawyer, or B) pick one of the other 99 options.
  - qwertyuiop924 10y ago
    That's entirely reasonable. I had hoped the we'd be able to use scripts from the other unixes, and have at least a mostly common language. But it's not a deal breaker or anything.
caf 10y ago
I think it would be an absurd result if, for example, I could: buy your book (which presumably does not infringe Oracle's copyrights); write a new implementation of the probe language described therein; and thereby end up with something that infringes Oracle's copyrights.

Annatar 10y ago

What would stop one from enhancing the DTrace in illumos further? That is licensed under the CDDL, and Adam has made several enhancements to DTrace lately, and if my memory serves me correctly, Bryan fixed a couple of bugs in it recently as well.

qwertyuiop924 10y ago
Wouldn't really solve our problem, though: We'd need a new compiler for the language. As I understand it, that would require a almost entirely new codebase. But I don't work on DTrace internals, so I may be wrong.
- Annatar 10y ago
  What? That doesn't make any sense. How is dtrace(1) built on illumos, then?
  - qwertyuiop924 10y ago
    I'm uncertain, but I thought dtrace on illumos was interpreted. Am I wrong? If I am, than all we'd have to rewrite was the compiler backend, which is much easier, so that would be nice.
    - Annatar 10y ago
      dtrace(1) itself is compiled from C into an ELF binary executable. % file `which dtrace` /usr/sbin/dtrace: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped The DTrace language, D, is interpreted. By DTrace. The problem lies in the fact that neither the GNU/Linux kernel, nor the GNU applications provide DTrace probe points. On Solaris and thus on illumos, and thus on SmartOS, there are tens of thousands of probe points and numerous probe providers. Some external applications, like PostgreSQL or PHP, added DTrace probes, and all is well on Solaris / illumos / SmartOS. Some, like node.js had providers and probes added by engineers at Joyent. http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-ustack-helpers/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-... http://dtrace.org/blogs/blog/category/node-js/ http://dtrace.org/blogs/blog/category/node-js/ GNU/Linux would have to do the same thing. It currently only has but a handful of DTrace probes and providers, which is understandably not very useful. http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/ http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/
      - qwertyuiop924 10y ago
        None of that is at all what I meant: allow me to clarify. I was talking about the DTrace language, which I have been avoiding calling D up until this point, so as to avoid confusion with the other D. In this post, when I talk about D, I will be referring to the DTrace language. Linux now has a tracing system called eBPF, which provides many of the same advantages of DTrace. This is what Gregg's blog post was about. However, eBPF requires compiling tracer scripts, or at least parts of the tracer scripts, into bytecode (IIRC). Currently, the bytecode is usually compiled from C code. However, it is ungainly and impractical to write a bunch of C every time you want to run a trace. So I asked if they had plans to support compiling a higher level language. At this point, somebody suggested that somebody should work on compiling the DTrace language, D, to eBPF bytecode. I thought that this would be a good idea, and we were discussing how viable it would be. I thought that you had suggested using Illumos DTrace as a base for this compiler. Since by my impression, Illumos DTrace interprets D, I thought that this would require almost a complete rewrite, and thus it wouldn't very helpful. It seems you meant something else. So what did you mean?

qwertyuiop924 10y ago

Huh. Well than how does Illumos and FreeBSD DTrace exist? They're actually based on the code, yes, but it's the same legally AFAIK (but I'm not a lawyer). Besides, as I understand it, it's the implementation that's copyrighted, not the interface or the idea, which is why Oracle went after Google the way they did, and why things like Freeablo and OpenMW can exist. But again, I'm not a lawyer, so this may be incorrect.

brendangregg 10y ago
CDDL is a file based license, so they use the files and follow the license on those files, which grants them rights to use it. The GPL/CDDL license wars over DTrace went on for years. It was a real shame -- we should have been discussing or using a great technology instead. There must be a hundred different ways and languages we can try, some of them may be better than DTrace's D. One of them is Oracle copyrighted, and lacks some features we need (stack saving). I'd either A) talk to a lawyer, or B) pick one of the other 99 options.
- qwertyuiop924 10y ago
  That's entirely reasonable. I had hoped the we'd be able to use scripts from the other unixes, and have at least a mostly common language. But it's not a deal breaker or anything.

caf 10y ago

I think it would be an absurd result if, for example, I could: buy your book (which presumably does not infringe Oracle's copyrights); write a new implementation of the probe language described therein; and thereby end up with something that infringes Oracle's copyrights.