4 ms·

I think we both got lost, didn't we? (:-)) So let's rewind: It is a really nice language, although missing a few things that BPF can do that DTrace can't (like

by Annatar 10y ago

I think we both got lost, didn't we? (:-)) So let's rewind: It is a really nice language, although missing a few things that BPF can do that DTrace can't (like saving and retrieving stacks), so it'd need to be enhanced. But with the warning that I'm not a lawyer: before beginning work on a DTrace/BPF front-end, I'd start by talking to a copyright lawyer to see if permission or a license is needed from Oracle. I fail to see how Oracle would be relevant, given that the entire DTrace is released under the CDDL. My question to Brendan was why he couldn't just take or enhance the existing DTrace codebase to do the front-end for BPF? (Personally, I think the other guy working on integrating BPF deeper into the GNU/Linux kernel, rather than biting the bullet, taking DTrace and doing the same thing Solaris engineers did to the SunOS kernel, is terribly mis-guided, especially since some probes and providers already exist in Linux. In the end, it will be a "Linux zoo", like everything else in Linux: "56" competing solutions for doing one thing, none of them comprehensive, and no consistency. Linux history is being repeated again. You have one comprehensive tool which works across several operating systems, DTrace, and Linux is yet again different from everyone else. Reminds me a lot of Microsoft Windows.)

qwertyuiop924 10y ago
It's not ideal, but eBPF works now, and it works on all the new kernels, which is more than can be said for any of the DTrace on Linux projects. It looks like eBPF is becoming that unified solution (mostly: ftrace and a few others still exist, but eBPF is the most capable, and is picking up steam). As for being different from what the other Unixes did, that's why I was in favor of developing a frontend that supported D, so that we could at least have a shared language with the rest of unixland. >My question to Brendan was why he couldn't just take or enhance the existing DTrace codebase to do the front-end for BPF? Brendan didn't answer, so I don't know, but my guess was that turning the interpreter into a compiler would require a near-complete rewrite.
- brendangregg 10y ago
  > that's why I was in favor of developing a frontend that supported D, so that we could at least have a shared language with the rest of unixland. Yes, D everyone would be nice, but what exactly does it mean? We can share DTrace scripts? Docs? Blog posts? Books? I've been porting them all over to bcc/BPF. Am I missing something? People have already began work developing new languages, eg, ply https://wkz.github.io/ply/ https://wkz.github.io/ply/. What if we develop a language that's much better than D? We need to make enhancements, anyway. I should reiterate something I covered in the post: most people won't care about this. Most people didn't write DTrace scripts when they had the option to (did either of you write DTrace scripts? have some on github?). Most people used the tools. And today, people can "apt install bcc-tools" and use DTraceToolkit like tools. If someone wants to engage lawyers & Oracle and see if or what needs to be done to use DTrace, then great, it'd make my job easier when developing these tools (and I'd sell more DTrace books :). But I'd also like to see someone take a swing at developing a better language as another possibility.
  - qwertyuiop924 10y ago
    Well, something good about the eBPF design is that we can go in both directions. It's possible to write both new language projects, and to port old ones. I'd work on it myself, if I had the requisite knowledge. Heck, maybe I'll learn some requisite knowledge and get on it.
    - brendangregg 10y ago
      Out of curiosity, did you ever write DTrace scripts? SystemTap? BPF? Do you have some on github I can see?
      - qwertyuiop924 10y ago
        No. See what I mean about requisite knowledge? I've been interested in learning DTrace for some time now, as it seemed like a tremdously helpful tool, but my systems are all linux, I don't run Fedora or RHEL, so systemtap is out. In recent months, eBPF has become a viable option, so I've been looking into that, but haven't had time to dive in deep yet. This makes me a poor candidate for doing this sort of work, which is why I'm not saying for sure that I'd do it. But I might be willing to try. Don't get me wrong: I have no particular attachment to D as the tracing language. However, I do think it's worth supporting the tracing language that pretty much all of the other big unixes support if we can. Am I less than qualified to have that opinion? Yes. Think what you will.
        brendangregg 10y ago
        Thanks, was wondering if this came from a specific language pain or not. Since all of this requires being on a newer kernel anyway, I'd do that first, then try the existing bcc tools and see where you are at. You might discover that you can do everything you want to with the existing tools, which include some powerful ad-hoc analysis tools (trace and argdist). Or, if you're missing one or two tools and let us know, and we may develop them for you and add them to bcc. Many work on the current Ubuntu Xenial 4.4. A few more on 4.6 (stack tracing), more on 4.7 (tracepoints), and all work on 4.9.
brendangregg 10y ago
> I fail to see how Oracle would be relevant This sounds like a legal conclusion. Care to back that statement with your real name?
- Annatar 10y ago
  I have never been affiliated with either Sun Microsystems or the Oracle corporation, if that is what you're getting at. And I am not yet prepared to disclose the name that the system gave me on the Internet - because Internet never, ever forgets. Roaches check in - but they don't check out. I still don't see how Oracle is relevant in any of that, since DTrace source code is under CDDL. As far as I'm concerned, that's where the story begins and that's where it ends.
  - brendangregg 10y ago
    As far as an anonymous person is concerned? You keep stating that your opinion matters, but won't back it up with your actual name. Why is that?
    - Annatar 10y ago
      Because a) I value my privacy and b) name shouldn't matter - only whether the argument, assertion, or the idea is correct or not. Sadly, you and I have never met. I've got questions for you, and then some.
      - brendangregg 10y ago
        "and then some"?
        Annatar 10y ago
        Yes, lots of questions, irrelevant to this topic but as a huge fan of illumos and SmartOS very much relevant to me, personally.
    - 10y ago

by Annatar 10y ago

qwertyuiop924 10y ago

It's not ideal, but eBPF works now, and it works on all the new kernels, which is more than can be said for any of the DTrace on Linux projects. It looks like eBPF is becoming that unified solution (mostly: ftrace and a few others still exist, but eBPF is the most capable, and is picking up steam). As for being different from what the other Unixes did, that's why I was in favor of developing a frontend that supported D, so that we could at least have a shared language with the rest of unixland. >My question to Brendan was why he couldn't just take or enhance the existing DTrace codebase to do the front-end for BPF? Brendan didn't answer, so I don't know, but my guess was that turning the interpreter into a compiler would require a near-complete rewrite.

brendangregg 10y ago
> that's why I was in favor of developing a frontend that supported D, so that we could at least have a shared language with the rest of unixland. Yes, D everyone would be nice, but what exactly does it mean? We can share DTrace scripts? Docs? Blog posts? Books? I've been porting them all over to bcc/BPF. Am I missing something? People have already began work developing new languages, eg, ply https://wkz.github.io/ply/ https://wkz.github.io/ply/. What if we develop a language that's much better than D? We need to make enhancements, anyway. I should reiterate something I covered in the post: most people won't care about this. Most people didn't write DTrace scripts when they had the option to (did either of you write DTrace scripts? have some on github?). Most people used the tools. And today, people can "apt install bcc-tools" and use DTraceToolkit like tools. If someone wants to engage lawyers & Oracle and see if or what needs to be done to use DTrace, then great, it'd make my job easier when developing these tools (and I'd sell more DTrace books :). But I'd also like to see someone take a swing at developing a better language as another possibility.
- qwertyuiop924 10y ago
  Well, something good about the eBPF design is that we can go in both directions. It's possible to write both new language projects, and to port old ones. I'd work on it myself, if I had the requisite knowledge. Heck, maybe I'll learn some requisite knowledge and get on it.
  - brendangregg 10y ago
    Out of curiosity, did you ever write DTrace scripts? SystemTap? BPF? Do you have some on github I can see?
    - qwertyuiop924 10y ago
      No. See what I mean about requisite knowledge? I've been interested in learning DTrace for some time now, as it seemed like a tremdously helpful tool, but my systems are all linux, I don't run Fedora or RHEL, so systemtap is out. In recent months, eBPF has become a viable option, so I've been looking into that, but haven't had time to dive in deep yet. This makes me a poor candidate for doing this sort of work, which is why I'm not saying for sure that I'd do it. But I might be willing to try. Don't get me wrong: I have no particular attachment to D as the tracing language. However, I do think it's worth supporting the tracing language that pretty much all of the other big unixes support if we can. Am I less than qualified to have that opinion? Yes. Think what you will.
      - brendangregg 10y ago
        Thanks, was wondering if this came from a specific language pain or not. Since all of this requires being on a newer kernel anyway, I'd do that first, then try the existing bcc tools and see where you are at. You might discover that you can do everything you want to with the existing tools, which include some powerful ad-hoc analysis tools (trace and argdist). Or, if you're missing one or two tools and let us know, and we may develop them for you and add them to bcc. Many work on the current Ubuntu Xenial 4.4. A few more on 4.6 (stack tracing), more on 4.7 (tracepoints), and all work on 4.9.

brendangregg 10y ago

> I fail to see how Oracle would be relevant This sounds like a legal conclusion. Care to back that statement with your real name?

Annatar 10y ago
I have never been affiliated with either Sun Microsystems or the Oracle corporation, if that is what you're getting at. And I am not yet prepared to disclose the name that the system gave me on the Internet - because Internet never, ever forgets. Roaches check in - but they don't check out. I still don't see how Oracle is relevant in any of that, since DTrace source code is under CDDL. As far as I'm concerned, that's where the story begins and that's where it ends.
- brendangregg 10y ago
  As far as an anonymous person is concerned? You keep stating that your opinion matters, but won't back it up with your actual name. Why is that?
  - Annatar 10y ago
    Because a) I value my privacy and b) name shouldn't matter - only whether the argument, assertion, or the idea is correct or not. Sadly, you and I have never met. I've got questions for you, and then some.
    - brendangregg 10y ago
      "and then some"?
      - Annatar 10y ago
        Yes, lots of questions, irrelevant to this topic but as a huge fan of illumos and SmartOS very much relevant to me, personally.
  - 10y ago