4 ms·

Except for perhaps Lisp languages, almost no language makes compile-time computing and code-generation so easy. This allows for some really powerful language fe

Except for perhaps Lisp languages, almost no language makes compile-time computing and code-generation so easy. This allows for some really powerful language features that can be designed as libraries, and puts this power in the hands of "regular developers" rather than only in the hands of template wizards.

jordigh 10y ago
Yeah, I really love that at compile time you can meta-program in almost the same language that you use for normal run-time things. I think the comparison to lisp is apt. It almost feels like lisp macros. Think C++ template metaprogramming but much, much easier and thus, seemingly more powerful. It's not that you couldn't do the same in C++, but D's metaprogramming is so much more accessible that it makes you want to use it.
- dbcurtis 10y ago
  For someone who knows zero about D, but is a total Pythonista, how would you compare the meta-programming facilities?
  - jordigh 10y ago
    I guess the best comparison to Python would be the dunder methods. You know how in Python a class really is just a dict, right? Like, foo.bar is pretty much syntactic sugar for foo.__get_attribute__(foo.__dict__['bar']) in most cases. Or how a + b is sugar for a.__plus__(b). Well, imagine that all of the things you can do in Python by messing with dunder methods, function decorators, or metaclasses could be precomputed by a compiler and would not even execute at all during runtime. It makes everything really fast at runtime and easy to precompute at compile time.
    - dbcurtis 10y ago
      Well, overriding dunder methods isn't really metaprogramming. It's just method inheritance/override. Is there anything in D similar to decorators? How does the registration pattern work in D, for instance?
      - gmfawcett 10y ago
        Do you mean like this kind of registration decorator? https://github.com/rejectedsoftware/vibe.d/blob/master/examples/rest/source/app.d#L232 https://github.com/rejectedsoftware/vibe.d/blob/master/examp... Decorators (in D, 'user defined attributes', or UDA) work differently. It's not a function-composition feature, but more of a tagging feature. You write a class, function, etc., tag it with custom attributes; and then have a separate compile-time function walk over your code, find the decorators, and augment the code based the meaning of the tags. (I used to wish that D had adopted Python-style decorators, since they are easy to reason about and implement, but I can see the logic of the more general UDA system that they adopted.) In practice, though, you would often use templates to achieve the same effect. Given a memoize template (really, just a memoize function), and an expensive-computation function, auto fastComputer = memoize!expensiveComputer; produces roughly the equivalent of @memoize def expensiveComputer(): ... but with opportunities for compile-time optimization.
    - gmfawcett 10y ago
      Yes, and plus you get type safety with all of that! When I'm writing in D, my editor marks all the type errors (and typos) in my code, almost as fast as I make them. The DMD compiler is so quick that there's almost no lag. This leads not only to a rapid development cycle, but a much higher level of reassurance (than I've had when writing, e.g., metaclass hacks in Python).
- jacques_chirac 10y ago
  There's plenty you can do with D but not C++, see regex, bitfields, swap member-by-member with checks for mutual pointers, the pegged library for grammars, etc etc.
  - jordigh 10y ago
    I don't know, Boost has me convinced that even things like regexes in C++ templates could be possible, if your compiler has a big enough stack to handle very deep template recursions. It's theoretically possible to build a regex parser in C++ templates, right? It would be horrible, but possible.
    - humanrebar 10y ago
      > It's theoretically possible to build a regex parser in C++ templates, right? You'd probably do large portions of it in "constexpr" types and functions rather than relying on recursive metaprogramming techniques. They are also planning on adding overloading based on whether a function is constexpr, which is important if you want the same library to support both compile-time and run-time matchers. So it's all theoretically possible already. But D has the advantage here in that it's not just possible but usable.
    - qznc 10y ago
      With D it is possible and not horrible and already done. ;) Talk: http://dconf.org/2014/talks/olshansky.html http://dconf.org/2014/talks/olshansky.html
dom96 10y ago
Anybody out there that has experience with both Nim and D? I am curious how they both compare in terms of metaprogramming.
- gmfawcett 10y ago
  I've dabbled with Nim. It's also a great language with great metaprogramming features. Plus a fast compiler that generates very lean code. (The real reason I stick with D over Nim, above all else, is RAII -- once you have it, it's really hard to live without. Okay, that, and also ranges, and array/string slices... they are fantastic to work with. Nim's got a better GC story right now, IMO, but there's a lot of activity in the D community to become more competitive in that space.) I found that some of the Nim's MP features, in particular AST macros, are a little harder to work with than D's templates and compile-time function evaluation. You get a lot of flexibility, but the cost is high. I'm not a big fan of how "mixin" is used in D to splice source-text into a generated function, it's certainly less principled than an AST transformation, but in practice the resulting code tends to be concise and easily read, whereas the AST-macro approach introduces a lot of accidental "noise" and complexity. The complexity raises the bar for reaching for a compile-time solution, where in D it seems equally as natural to write compile-time code as it does to write regular code. (Not to pick on a strawman, though -- AST macros are not Nim's only MP tool.) The Nim MP feature I never really played with was the rewrite rules. They seem interesting in theory, but maybe a little too magical for my liking! I have a lot of respect for Nim. I am glad we live in a world with so many options. Like Walter said in an earlier comment, it's an embarrassment of riches. :)
  - theprotocol 10y ago
    Thanks for the writeup. I've just been evaluating Rust, D, and Nim, and reading about your experience has been helpful. Can you elaborate on how D is planning to sort out the garbage collection? Nim's GC is extremely fast and thread local, and can be disabled without breaking libraries (according to the author, it does something with memory regions that I haven't 100% grasped yet). I've googled about D's garbage collector and it's apparently been discussed as the language's biggest flaw since 2013, but I can't find any information whatsoever on what's being done in that regard.

jordigh 10y ago

Yeah, I really love that at compile time you can meta-program in almost the same language that you use for normal run-time things. I think the comparison to lisp is apt. It almost feels like lisp macros. Think C++ template metaprogramming but much, much easier and thus, seemingly more powerful. It's not that you couldn't do the same in C++, but D's metaprogramming is so much more accessible that it makes you want to use it.

dbcurtis 10y ago
For someone who knows zero about D, but is a total Pythonista, how would you compare the meta-programming facilities?
- jordigh 10y ago
  I guess the best comparison to Python would be the dunder methods. You know how in Python a class really is just a dict, right? Like, foo.bar is pretty much syntactic sugar for foo.__get_attribute__(foo.__dict__['bar']) in most cases. Or how a + b is sugar for a.__plus__(b). Well, imagine that all of the things you can do in Python by messing with dunder methods, function decorators, or metaclasses could be precomputed by a compiler and would not even execute at all during runtime. It makes everything really fast at runtime and easy to precompute at compile time.
  - dbcurtis 10y ago
    Well, overriding dunder methods isn't really metaprogramming. It's just method inheritance/override. Is there anything in D similar to decorators? How does the registration pattern work in D, for instance?
    - gmfawcett 10y ago
      Do you mean like this kind of registration decorator? https://github.com/rejectedsoftware/vibe.d/blob/master/examples/rest/source/app.d#L232 https://github.com/rejectedsoftware/vibe.d/blob/master/examp... Decorators (in D, 'user defined attributes', or UDA) work differently. It's not a function-composition feature, but more of a tagging feature. You write a class, function, etc., tag it with custom attributes; and then have a separate compile-time function walk over your code, find the decorators, and augment the code based the meaning of the tags. (I used to wish that D had adopted Python-style decorators, since they are easy to reason about and implement, but I can see the logic of the more general UDA system that they adopted.) In practice, though, you would often use templates to achieve the same effect. Given a memoize template (really, just a memoize function), and an expensive-computation function, auto fastComputer = memoize!expensiveComputer; produces roughly the equivalent of @memoize def expensiveComputer(): ... but with opportunities for compile-time optimization.
  - gmfawcett 10y ago
    Yes, and plus you get type safety with all of that! When I'm writing in D, my editor marks all the type errors (and typos) in my code, almost as fast as I make them. The DMD compiler is so quick that there's almost no lag. This leads not only to a rapid development cycle, but a much higher level of reassurance (than I've had when writing, e.g., metaclass hacks in Python).
jacques_chirac 10y ago
There's plenty you can do with D but not C++, see regex, bitfields, swap member-by-member with checks for mutual pointers, the pegged library for grammars, etc etc.
- jordigh 10y ago
  I don't know, Boost has me convinced that even things like regexes in C++ templates could be possible, if your compiler has a big enough stack to handle very deep template recursions. It's theoretically possible to build a regex parser in C++ templates, right? It would be horrible, but possible.
  - humanrebar 10y ago
    > It's theoretically possible to build a regex parser in C++ templates, right? You'd probably do large portions of it in "constexpr" types and functions rather than relying on recursive metaprogramming techniques. They are also planning on adding overloading based on whether a function is constexpr, which is important if you want the same library to support both compile-time and run-time matchers. So it's all theoretically possible already. But D has the advantage here in that it's not just possible but usable.
  - qznc 10y ago
    With D it is possible and not horrible and already done. ;) Talk: http://dconf.org/2014/talks/olshansky.html http://dconf.org/2014/talks/olshansky.html

dom96 10y ago

Anybody out there that has experience with both Nim and D? I am curious how they both compare in terms of metaprogramming.

gmfawcett 10y ago
I've dabbled with Nim. It's also a great language with great metaprogramming features. Plus a fast compiler that generates very lean code. (The real reason I stick with D over Nim, above all else, is RAII -- once you have it, it's really hard to live without. Okay, that, and also ranges, and array/string slices... they are fantastic to work with. Nim's got a better GC story right now, IMO, but there's a lot of activity in the D community to become more competitive in that space.) I found that some of the Nim's MP features, in particular AST macros, are a little harder to work with than D's templates and compile-time function evaluation. You get a lot of flexibility, but the cost is high. I'm not a big fan of how "mixin" is used in D to splice source-text into a generated function, it's certainly less principled than an AST transformation, but in practice the resulting code tends to be concise and easily read, whereas the AST-macro approach introduces a lot of accidental "noise" and complexity. The complexity raises the bar for reaching for a compile-time solution, where in D it seems equally as natural to write compile-time code as it does to write regular code. (Not to pick on a strawman, though -- AST macros are not Nim's only MP tool.) The Nim MP feature I never really played with was the rewrite rules. They seem interesting in theory, but maybe a little too magical for my liking! I have a lot of respect for Nim. I am glad we live in a world with so many options. Like Walter said in an earlier comment, it's an embarrassment of riches. :)
- theprotocol 10y ago
  Thanks for the writeup. I've just been evaluating Rust, D, and Nim, and reading about your experience has been helpful. Can you elaborate on how D is planning to sort out the garbage collection? Nim's GC is extremely fast and thread local, and can be disabled without breaking libraries (according to the author, it does something with memory regions that I haven't 100% grasped yet). I've googled about D's garbage collector and it's apparently been discussed as the language's biggest flaw since 2013, but I can't find any information whatsoever on what's being done in that regard.