6 ms·

Wouldn't really solve our problem, though: We'd need a new compiler for the language. As I understand it, that would require a almost entirely new codebase. But

Wouldn't really solve our problem, though: We'd need a new compiler for the language. As I understand it, that would require a almost entirely new codebase. But I don't work on DTrace internals, so I may be wrong.

Annatar 10y ago
What? That doesn't make any sense. How is dtrace(1) built on illumos, then?
- qwertyuiop924 10y ago
  I'm uncertain, but I thought dtrace on illumos was interpreted. Am I wrong? If I am, than all we'd have to rewrite was the compiler backend, which is much easier, so that would be nice.
  - Annatar 10y ago
    dtrace(1) itself is compiled from C into an ELF binary executable. % file `which dtrace` /usr/sbin/dtrace: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped The DTrace language, D, is interpreted. By DTrace. The problem lies in the fact that neither the GNU/Linux kernel, nor the GNU applications provide DTrace probe points. On Solaris and thus on illumos, and thus on SmartOS, there are tens of thousands of probe points and numerous probe providers. Some external applications, like PostgreSQL or PHP, added DTrace probes, and all is well on Solaris / illumos / SmartOS. Some, like node.js had providers and probes added by engineers at Joyent. http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-ustack-helpers/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-... http://dtrace.org/blogs/blog/category/node-js/ http://dtrace.org/blogs/blog/category/node-js/ GNU/Linux would have to do the same thing. It currently only has but a handful of DTrace probes and providers, which is understandably not very useful. http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/ http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/
    - qwertyuiop924 10y ago
      None of that is at all what I meant: allow me to clarify. I was talking about the DTrace language, which I have been avoiding calling D up until this point, so as to avoid confusion with the other D. In this post, when I talk about D, I will be referring to the DTrace language. Linux now has a tracing system called eBPF, which provides many of the same advantages of DTrace. This is what Gregg's blog post was about. However, eBPF requires compiling tracer scripts, or at least parts of the tracer scripts, into bytecode (IIRC). Currently, the bytecode is usually compiled from C code. However, it is ungainly and impractical to write a bunch of C every time you want to run a trace. So I asked if they had plans to support compiling a higher level language. At this point, somebody suggested that somebody should work on compiling the DTrace language, D, to eBPF bytecode. I thought that this would be a good idea, and we were discussing how viable it would be. I thought that you had suggested using Illumos DTrace as a base for this compiler. Since by my impression, Illumos DTrace interprets D, I thought that this would require almost a complete rewrite, and thus it wouldn't very helpful. It seems you meant something else. So what did you mean?
      - Annatar 10y ago
        I think we both got lost, didn't we? (:-)) So let's rewind: It is a really nice language, although missing a few things that BPF can do that DTrace can't (like saving and retrieving stacks), so it'd need to be enhanced. But with the warning that I'm not a lawyer: before beginning work on a DTrace/BPF front-end, I'd start by talking to a copyright lawyer to see if permission or a license is needed from Oracle. I fail to see how Oracle would be relevant, given that the entire DTrace is released under the CDDL. My question to Brendan was why he couldn't just take or enhance the existing DTrace codebase to do the front-end for BPF? (Personally, I think the other guy working on integrating BPF deeper into the GNU/Linux kernel, rather than biting the bullet, taking DTrace and doing the same thing Solaris engineers did to the SunOS kernel, is terribly mis-guided, especially since some probes and providers already exist in Linux. In the end, it will be a "Linux zoo", like everything else in Linux: "56" competing solutions for doing one thing, none of them comprehensive, and no consistency. Linux history is being repeated again. You have one comprehensive tool which works across several operating systems, DTrace, and Linux is yet again different from everyone else. Reminds me a lot of Microsoft Windows.)

Annatar 10y ago

What? That doesn't make any sense. How is dtrace(1) built on illumos, then?

qwertyuiop924 10y ago
I'm uncertain, but I thought dtrace on illumos was interpreted. Am I wrong? If I am, than all we'd have to rewrite was the compiler backend, which is much easier, so that would be nice.
- Annatar 10y ago
  dtrace(1) itself is compiled from C into an ELF binary executable. % file `which dtrace` /usr/sbin/dtrace: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped The DTrace language, D, is interpreted. By DTrace. The problem lies in the fact that neither the GNU/Linux kernel, nor the GNU applications provide DTrace probe points. On Solaris and thus on illumos, and thus on SmartOS, there are tens of thousands of probe points and numerous probe providers. Some external applications, like PostgreSQL or PHP, added DTrace probes, and all is well on Solaris / illumos / SmartOS. Some, like node.js had providers and probes added by engineers at Joyent. http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-ustack-helpers/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-... http://dtrace.org/blogs/blog/category/node-js/ http://dtrace.org/blogs/blog/category/node-js/ GNU/Linux would have to do the same thing. It currently only has but a handful of DTrace probes and providers, which is understandably not very useful. http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/ http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/
  - qwertyuiop924 10y ago
    None of that is at all what I meant: allow me to clarify. I was talking about the DTrace language, which I have been avoiding calling D up until this point, so as to avoid confusion with the other D. In this post, when I talk about D, I will be referring to the DTrace language. Linux now has a tracing system called eBPF, which provides many of the same advantages of DTrace. This is what Gregg's blog post was about. However, eBPF requires compiling tracer scripts, or at least parts of the tracer scripts, into bytecode (IIRC). Currently, the bytecode is usually compiled from C code. However, it is ungainly and impractical to write a bunch of C every time you want to run a trace. So I asked if they had plans to support compiling a higher level language. At this point, somebody suggested that somebody should work on compiling the DTrace language, D, to eBPF bytecode. I thought that this would be a good idea, and we were discussing how viable it would be. I thought that you had suggested using Illumos DTrace as a base for this compiler. Since by my impression, Illumos DTrace interprets D, I thought that this would require almost a complete rewrite, and thus it wouldn't very helpful. It seems you meant something else. So what did you mean?
    - Annatar 10y ago
      I think we both got lost, didn't we? (:-)) So let's rewind: It is a really nice language, although missing a few things that BPF can do that DTrace can't (like saving and retrieving stacks), so it'd need to be enhanced. But with the warning that I'm not a lawyer: before beginning work on a DTrace/BPF front-end, I'd start by talking to a copyright lawyer to see if permission or a license is needed from Oracle. I fail to see how Oracle would be relevant, given that the entire DTrace is released under the CDDL. My question to Brendan was why he couldn't just take or enhance the existing DTrace codebase to do the front-end for BPF? (Personally, I think the other guy working on integrating BPF deeper into the GNU/Linux kernel, rather than biting the bullet, taking DTrace and doing the same thing Solaris engineers did to the SunOS kernel, is terribly mis-guided, especially since some probes and providers already exist in Linux. In the end, it will be a "Linux zoo", like everything else in Linux: "56" competing solutions for doing one thing, none of them comprehensive, and no consistency. Linux history is being repeated again. You have one comprehensive tool which works across several operating systems, DTrace, and Linux is yet again different from everyone else. Reminds me a lot of Microsoft Windows.)