5 ms·

The point here is not that metaprogramming is universally bad, it's that it should be used as a last resort (or second-to-last resort to codegen, depending on y

The point here is not that metaprogramming is universally bad, it's that it should be used as a last resort (or second-to-last resort to codegen, depending on your platform/application) and you should feel bad when you need to use it. Simply using it to save keystrokes is pretty lame since you'll spend much more time maintaining code than typing out the original and explicitness is valuable when returning to a piece of code. Additionally, if you find yourself that you need a lot of metaprogramming for a lot of things it's often an indication that you could just refactor your code using static idioms and be better off. Not to mention that the more you use metaprogramming, the more likely it is that you or someone else will kill some runtime optimizations of the JIT. IMO, syntax matters way less than people tend to think it does, and the additional implementation complexity and astonishment cute syntax introduces tends to make the trade-off not worth it.

epscylonb 13y ago
In my experience overuse of meta programming is usually a sign that the architecture of the system is bad and the software should be split up into separate libraries or web services.
- chongli 13y ago
  That all depends on the language. Lisp (pretty much all of its dialects) heavily emphasizes the use of macros to extend the language and build all kinds of control structures and embedded domain-specific languages (EDSLs).
  - peterhunt 13y ago
    Absolutely. One could make the argument, though, that many (including me) believe that expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance.
    - sillysaurus 13y ago
      expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance. Well, Arc is actually very maintainable, and quite fast. After all, HN is powered by Arc, and it serves >100k daily uniques on a single core.
      - peterhunt 13y ago
        Yep! Not saying that it's impossible, just that some of the concepts that Lisp is based on make these things harder to pull off. The reason for this is that Lisp emphasizes flexibility, and the more flexible your runtime environment is, the more ways to do something, which makes it easier to have hard-to-maintain code (and harder to reason about performance, too). But it's not impossible to do with Lisp for sure.
        chongli 13y ago
        Lisp macros also provide ways to take ugly, hand-optimized code and clean it up to provide a cleaner, friendlier interface. Like all macros, this comes at no runtime cost (since all macros have been fully expanded by this point).
        sillysaurus 13y ago
        Actually, it's not any harder to write maintainable code in Arc. In many ways, it's much easier, because you write your program as a set of parts which fit together like an arch.
      - gohrt 13y ago
        HN is fast because it is feature-barren. That's a design choice pg is free to make (and cutting features is a great way to improve performance), but not a proof of Arc/Lisp's ability to support complex applications with high-performance.
        sillysaurus 13y ago
        Which features would convince you that Arc is fast?
  - cgag 13y ago
    Lisp macros are a lot simpler and easier to understand than what goes on with the types of metaprogramming done in ruby/rails. Expanding a macro is a lot easier for me to wrap my head around than stuff like dynamically defining methods when something is subclassed or when magic happens in method_missing.
  - Homunculiheaded 13y ago
    At the same time the Lisp community is one of the fiercest in supporting the idea that nothing that can be a function should be a macro. Metaprogramming just because it is clever is more abhorred in the Lisp community than any other I've seen.
    - chongli 13y ago
      Of course! Macros are distinctly less powerful than functions for one reason: they are not first class values at runtime and thus cannot be passed to higher-order functions.
      - mst 13y ago
        This is solved by using an operative lisp - fexprs (or 'vau'/'$vau' in kernel and wat) are non-applicative combiners that execute at runtime, so you can have a first class combiner that chooses when/how/how many times to evaluate its arguments. I've been having a lot of fun recently writing code in wat-pl (my perl port of Manuel Simonyi's wat-js interpreter)
  - marshray 13y ago
    Yep, and it surely depends upon the problem being solved too.
  - dustingetz 13y ago
    The heavy emphasis of macros in Lisp may also be its greatest weakness. I see all sorts of Clojure code that uses macros where they needn't have, and it tremendously impedes my ability to use the code in ways the library author may not have expected. I love the way petehunt put it - metaprogramming is a last resort, you should feel bad when you use it, and language designers should include it but deemphasize it. (Scala and Haskell nail it I think; Clojure perhaps emphasizes macros a bit too much which harms the output of intermediate programmers. But, Clojure is not designed for intermediate programmers.)
area51org 13y ago
The point here is not that metaprogramming is universally bad, it's that it should be used as a last resort ... and you should feel bad you could just refactor your code using static idioms you or someone else will kill some runtime optimizations of the JIT. syntax matters way less than people tend to think ... astonishment [sic] cute syntax Many intelligent, design-savvy people would strongly disagree with you. Besides the obvious gains in the ability to create DSLs, there is a power and fluidity in metaprogramming that is not possible any other way. Speed (as, say, provided by runtime optimizations from a JIT compiler) is not always critical. If it was, one probably should be using a language other than Ruby or Javascript; they're meant to be flexible, powerful languages, and their emphasis is not on pure speed. Syntax is not "cute." Humans are going to read and use this code, and the more it reads like English, the more likely it is to be understood.
- adamnemecek 13y ago
  You seem to be suggesting that DSLs are universally accepted as a pinnacle of good design.
  - mindcrime 13y ago
    Indeed. I, for one, think that over-proliferation of DSL's can be a Bad Thing. To use one example from my own experience: I spend a lot of time coding in a Groovy / Grails environment. Now, I like Grails and Groovy, but Grails introduces it's own weird DSLs for things like: configuring Spring Beans configuring Log4J defining URL mappings configuring Hibernate mappings & queries etc. Now some of those are cool, but at least some of them replace other techniques that I already know, and find much more readable and understandable. Configuring Log4J, for example. I'd much rather simply jam a log4j.xml in there and forget about the Grails DSL. The thing is, none of these DSL's is, in and of itself, necessarily bad in any way... but there's a bit of "cognitive overload" in having to deal with 3 or 4 or 5 new DSLs, on top of the base Groovy stuff. The flip side is, these DSLs mean that your configuration is done mostly in .groovy files and you can use normal groovy syntax for looping and accessing variables, etc. So it's easier to code up more dynamic configurations than if you were using XML files or .ini files. Anyway, the point of this rant is not to say "DSLs are bad" but just to lend weight to the suggestion that they aren't universally Good either.
gtani 13y ago
There's some strong (i hope a decent number of people here would say "winning") counterarguments, scala macros and implicits are incredibly useful. (i see lots of complaints about C++ template error messages, but i haven't been doing C++ so can't comment informedly) I think a better rule is if the tooling can't look at the source, intermediate compiler stages/ AST and bytecode/binary and tell you unambiguously where something came from, then you can characterize it as a "last resort" And there's the counterargument that for statically typed languages with good type systems (scala, haskell, ocaml, F#), there's REPL prototyping/testing and code generated from template haskell or camlp4 has to type check, so you have multiple safety mechanisms
dom96 13y ago
While I mostly agree with what you are saying I think that there is a place for metaprogramming and some programming languages implement metaprogramming very well. Using metaprogramming should not make you "feel bad", however it should not be used excessively. Metaprogramming gives you a lot of power, it allows you to extend the language in any way you please. With this great power, of course, comes great responsibility but I would rather have the option to use this power than to not have it at all. I would also argue that metaprogramming can enable your code to be more efficient, if the macros/templates you write are evaluated at compile-time. This is the approach that the Nimrod programming language (http://nimrod-code.org http://nimrod-code.org) takes and it works well. This also means that in most cases compilation will fail if the macro is incorrect instead of causing some silent errors at runtime.

epscylonb 13y ago

In my experience overuse of meta programming is usually a sign that the architecture of the system is bad and the software should be split up into separate libraries or web services.

chongli 13y ago
That all depends on the language. Lisp (pretty much all of its dialects) heavily emphasizes the use of macros to extend the language and build all kinds of control structures and embedded domain-specific languages (EDSLs).
- peterhunt 13y ago
  Absolutely. One could make the argument, though, that many (including me) believe that expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance.
  - sillysaurus 13y ago
    expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance. Well, Arc is actually very maintainable, and quite fast. After all, HN is powered by Arc, and it serves >100k daily uniques on a single core.
    - peterhunt 13y ago
      Yep! Not saying that it's impossible, just that some of the concepts that Lisp is based on make these things harder to pull off. The reason for this is that Lisp emphasizes flexibility, and the more flexible your runtime environment is, the more ways to do something, which makes it easier to have hard-to-maintain code (and harder to reason about performance, too). But it's not impossible to do with Lisp for sure.
      - chongli 13y ago
        Lisp macros also provide ways to take ugly, hand-optimized code and clean it up to provide a cleaner, friendlier interface. Like all macros, this comes at no runtime cost (since all macros have been fully expanded by this point).
      - sillysaurus 13y ago
        Actually, it's not any harder to write maintainable code in Arc. In many ways, it's much easier, because you write your program as a set of parts which fit together like an arch.
    - gohrt 13y ago
      HN is fast because it is feature-barren. That's a design choice pg is free to make (and cutting features is a great way to improve performance), but not a proof of Arc/Lisp's ability to support complex applications with high-performance.
      - sillysaurus 13y ago
        Which features would convince you that Arc is fast?
- cgag 13y ago
  Lisp macros are a lot simpler and easier to understand than what goes on with the types of metaprogramming done in ruby/rails. Expanding a macro is a lot easier for me to wrap my head around than stuff like dynamically defining methods when something is subclassed or when magic happens in method_missing.
- Homunculiheaded 13y ago
  At the same time the Lisp community is one of the fiercest in supporting the idea that nothing that can be a function should be a macro. Metaprogramming just because it is clever is more abhorred in the Lisp community than any other I've seen.
  - chongli 13y ago
    Of course! Macros are distinctly less powerful than functions for one reason: they are not first class values at runtime and thus cannot be passed to higher-order functions.
    - mst 13y ago
      This is solved by using an operative lisp - fexprs (or 'vau'/'$vau' in kernel and wat) are non-applicative combiners that execute at runtime, so you can have a first class combiner that chooses when/how/how many times to evaluate its arguments. I've been having a lot of fun recently writing code in wat-pl (my perl port of Manuel Simonyi's wat-js interpreter)
- marshray 13y ago
  Yep, and it surely depends upon the problem being solved too.
- dustingetz 13y ago
  The heavy emphasis of macros in Lisp may also be its greatest weakness. I see all sorts of Clojure code that uses macros where they needn't have, and it tremendously impedes my ability to use the code in ways the library author may not have expected. I love the way petehunt put it - metaprogramming is a last resort, you should feel bad when you use it, and language designers should include it but deemphasize it. (Scala and Haskell nail it I think; Clojure perhaps emphasizes macros a bit too much which harms the output of intermediate programmers. But, Clojure is not designed for intermediate programmers.)

area51org 13y ago

The point here is not that metaprogramming is universally bad, it's that it should be used as a last resort ... and you should feel bad you could just refactor your code using static idioms you or someone else will kill some runtime optimizations of the JIT. syntax matters way less than people tend to think ... astonishment [sic] cute syntax Many intelligent, design-savvy people would strongly disagree with you. Besides the obvious gains in the ability to create DSLs, there is a power and fluidity in metaprogramming that is not possible any other way. Speed (as, say, provided by runtime optimizations from a JIT compiler) is not always critical. If it was, one probably should be using a language other than Ruby or Javascript; they're meant to be flexible, powerful languages, and their emphasis is not on pure speed. Syntax is not "cute." Humans are going to read and use this code, and the more it reads like English, the more likely it is to be understood.

adamnemecek 13y ago
You seem to be suggesting that DSLs are universally accepted as a pinnacle of good design.
- mindcrime 13y ago
  Indeed. I, for one, think that over-proliferation of DSL's can be a Bad Thing. To use one example from my own experience: I spend a lot of time coding in a Groovy / Grails environment. Now, I like Grails and Groovy, but Grails introduces it's own weird DSLs for things like: configuring Spring Beans configuring Log4J defining URL mappings configuring Hibernate mappings & queries etc. Now some of those are cool, but at least some of them replace other techniques that I already know, and find much more readable and understandable. Configuring Log4J, for example. I'd much rather simply jam a log4j.xml in there and forget about the Grails DSL. The thing is, none of these DSL's is, in and of itself, necessarily bad in any way... but there's a bit of "cognitive overload" in having to deal with 3 or 4 or 5 new DSLs, on top of the base Groovy stuff. The flip side is, these DSLs mean that your configuration is done mostly in .groovy files and you can use normal groovy syntax for looping and accessing variables, etc. So it's easier to code up more dynamic configurations than if you were using XML files or .ini files. Anyway, the point of this rant is not to say "DSLs are bad" but just to lend weight to the suggestion that they aren't universally Good either.

gtani 13y ago

There's some strong (i hope a decent number of people here would say "winning") counterarguments, scala macros and implicits are incredibly useful. (i see lots of complaints about C++ template error messages, but i haven't been doing C++ so can't comment informedly) I think a better rule is if the tooling can't look at the source, intermediate compiler stages/ AST and bytecode/binary and tell you unambiguously where something came from, then you can characterize it as a "last resort" And there's the counterargument that for statically typed languages with good type systems (scala, haskell, ocaml, F#), there's REPL prototyping/testing and code generated from template haskell or camlp4 has to type check, so you have multiple safety mechanisms

dom96 13y ago

While I mostly agree with what you are saying I think that there is a place for metaprogramming and some programming languages implement metaprogramming very well. Using metaprogramming should not make you "feel bad", however it should not be used excessively. Metaprogramming gives you a lot of power, it allows you to extend the language in any way you please. With this great power, of course, comes great responsibility but I would rather have the option to use this power than to not have it at all. I would also argue that metaprogramming can enable your code to be more efficient, if the macros/templates you write are evaluated at compile-time. This is the approach that the Nimrod programming language (http://nimrod-code.org http://nimrod-code.org) takes and it works well. This also means that in most cases compilation will fail if the macro is incorrect instead of causing some silent errors at runtime.