7 ms·

How feasible would it be to compile the dtrace language itself to an eBPF backend?

by caf 10y ago

qwertyuiop924 10y ago
Huh. Is there a formal definition anywhere? If we have some kind of spec (even if it's not formal), it might be possible, since they are roughly equivalent, AFAIK. However, since I haven't worked in-depth in either, I'm unsure what work would be involved: it's possible that only a subset would compile. Anyways, this is probably a good goal to shoot for. Dtrace is the system tracer on pretty much all of the other big unixes, so it's a good idea to support as much of its language as possible. Plus, there are a lot of scripts already written in DTrace's language: having access to them would be invaluable.
- brendangregg 10y ago
  Nobody would love a DTrace/BPF front-end more than I. And not just because I'd sell more copies of my DTrace book (I joke :). It is a really nice language, although missing a few things that BPF can do that DTrace can't (like saving and retrieving stacks), so it'd need to be enhanced. But with the warning that I'm not a lawyer: before beginning work on a DTrace/BPF front-end, I'd start by talking to a copyright lawyer to see if permission or a license is needed from Oracle. DTrace is Oracle copyrighted, and re-implementing a DTrace front-end on Linux sounds a lot like re-implementing an Oracle-copyrighted API.
  - Annatar 10y ago
    What would stop one from enhancing the DTrace in illumos further? That is licensed under the CDDL, and Adam has made several enhancements to DTrace lately, and if my memory serves me correctly, Bryan fixed a couple of bugs in it recently as well.
    - qwertyuiop924 10y ago
      Wouldn't really solve our problem, though: We'd need a new compiler for the language. As I understand it, that would require a almost entirely new codebase. But I don't work on DTrace internals, so I may be wrong.
      - Annatar 10y ago
        What? That doesn't make any sense. How is dtrace(1) built on illumos, then?
        qwertyuiop924 10y ago
        I'm uncertain, but I thought dtrace on illumos was interpreted. Am I wrong? If I am, than all we'd have to rewrite was the compiler backend, which is much easier, so that would be nice.
        Annatar 10y ago
        dtrace(1) itself is compiled from C into an ELF binary executable. % file `which dtrace` /usr/sbin/dtrace: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped The DTrace language, D, is interpreted. By DTrace. The problem lies in the fact that neither the GNU/Linux kernel, nor the GNU applications provide DTrace probe points. On Solaris and thus on illumos, and thus on SmartOS, there are tens of thousands of probe points and numerous probe providers. Some external applications, like PostgreSQL or PHP, added DTrace probes, and all is well on Solaris / illumos / SmartOS. Some, like node.js had providers and probes added by engineers at Joyent. http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-ustack-helpers/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-... http://dtrace.org/blogs/blog/category/node-js/ http://dtrace.org/blogs/blog/category/node-js/ GNU/Linux would have to do the same thing. It currently only has but a handful of DTrace probes and providers, which is understandably not very useful. http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/ http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/
  - qwertyuiop924 10y ago
    Huh. Well than how does Illumos and FreeBSD DTrace exist? They're actually based on the code, yes, but it's the same legally AFAIK (but I'm not a lawyer). Besides, as I understand it, it's the implementation that's copyrighted, not the interface or the idea, which is why Oracle went after Google the way they did, and why things like Freeablo and OpenMW can exist. But again, I'm not a lawyer, so this may be incorrect.
    - brendangregg 10y ago
      CDDL is a file based license, so they use the files and follow the license on those files, which grants them rights to use it. The GPL/CDDL license wars over DTrace went on for years. It was a real shame -- we should have been discussing or using a great technology instead. There must be a hundred different ways and languages we can try, some of them may be better than DTrace's D. One of them is Oracle copyrighted, and lacks some features we need (stack saving). I'd either A) talk to a lawyer, or B) pick one of the other 99 options.
      - qwertyuiop924 10y ago
        That's entirely reasonable. I had hoped the we'd be able to use scripts from the other unixes, and have at least a mostly common language. But it's not a deal breaker or anything.
  - caf 10y ago
    I think it would be an absurd result if, for example, I could: buy your book (which presumably does not infringe Oracle's copyrights); write a new implementation of the probe language described therein; and thereby end up with something that infringes Oracle's copyrights.

How feasible would it be to compile the dtrace language itself to an eBPF backend?

by caf 10y ago

How feasible would it be to compile the dtrace language itself to an eBPF backend?

Huh. Is there a formal definition anywhere? If we have some kind of spec (even if it's not formal), it might be possible, since they are roughly equivalent, AFAIK. However, since I haven't worked in-depth in either, I'm unsure what work would be involved: it's possible that only a subset would compile. Anyways, this is probably a good goal to shoot for. Dtrace is the system tracer on pretty much all of the other big unixes, so it's a good idea to support as much of its language as possible. Plus, there are a lot of scripts already written in DTrace's language: having access to them would be invaluable.

brendangregg 10y ago
Nobody would love a DTrace/BPF front-end more than I. And not just because I'd sell more copies of my DTrace book (I joke :). It is a really nice language, although missing a few things that BPF can do that DTrace can't (like saving and retrieving stacks), so it'd need to be enhanced. But with the warning that I'm not a lawyer: before beginning work on a DTrace/BPF front-end, I'd start by talking to a copyright lawyer to see if permission or a license is needed from Oracle. DTrace is Oracle copyrighted, and re-implementing a DTrace front-end on Linux sounds a lot like re-implementing an Oracle-copyrighted API.
- Annatar 10y ago
  What would stop one from enhancing the DTrace in illumos further? That is licensed under the CDDL, and Adam has made several enhancements to DTrace lately, and if my memory serves me correctly, Bryan fixed a couple of bugs in it recently as well.
  - qwertyuiop924 10y ago
    Wouldn't really solve our problem, though: We'd need a new compiler for the language. As I understand it, that would require a almost entirely new codebase. But I don't work on DTrace internals, so I may be wrong.
    - Annatar 10y ago
      What? That doesn't make any sense. How is dtrace(1) built on illumos, then?
      - qwertyuiop924 10y ago
        I'm uncertain, but I thought dtrace on illumos was interpreted. Am I wrong? If I am, than all we'd have to rewrite was the compiler backend, which is much easier, so that would be nice.
        Annatar 10y ago
        dtrace(1) itself is compiled from C into an ELF binary executable. % file `which dtrace` /usr/sbin/dtrace: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped The DTrace language, D, is interpreted. By DTrace. The problem lies in the fact that neither the GNU/Linux kernel, nor the GNU applications provide DTrace probe points. On Solaris and thus on illumos, and thus on SmartOS, there are tens of thousands of probe points and numerous probe providers. Some external applications, like PostgreSQL or PHP, added DTrace probes, and all is well on Solaris / illumos / SmartOS. Some, like node.js had providers and probes added by engineers at Joyent. http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2012/04/25/profiling-node-js/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-ustack-helpers/ http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-... http://dtrace.org/blogs/blog/category/node-js/ http://dtrace.org/blogs/blog/category/node-js/ GNU/Linux would have to do the same thing. It currently only has but a handful of DTrace probes and providers, which is understandably not very useful. http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/ http://dtrace.org/blogs/ahl/2011/10/05/dtrace-for-linux-2/
- qwertyuiop924 10y ago
  Huh. Well than how does Illumos and FreeBSD DTrace exist? They're actually based on the code, yes, but it's the same legally AFAIK (but I'm not a lawyer). Besides, as I understand it, it's the implementation that's copyrighted, not the interface or the idea, which is why Oracle went after Google the way they did, and why things like Freeablo and OpenMW can exist. But again, I'm not a lawyer, so this may be incorrect.
  - brendangregg 10y ago
    CDDL is a file based license, so they use the files and follow the license on those files, which grants them rights to use it. The GPL/CDDL license wars over DTrace went on for years. It was a real shame -- we should have been discussing or using a great technology instead. There must be a hundred different ways and languages we can try, some of them may be better than DTrace's D. One of them is Oracle copyrighted, and lacks some features we need (stack saving). I'd either A) talk to a lawyer, or B) pick one of the other 99 options.
    - qwertyuiop924 10y ago
      That's entirely reasonable. I had hoped the we'd be able to use scripts from the other unixes, and have at least a mostly common language. But it's not a deal breaker or anything.
- caf 10y ago
  I think it would be an absurd result if, for example, I could: buy your book (which presumably does not infringe Oracle's copyrights); write a new implementation of the probe language described therein; and thereby end up with something that infringes Oracle's copyrights.