5 ms·

In my experience overuse of meta programming is usually a sign that the architecture of the system is bad and the software should be split up into separate libr

In my experience overuse of meta programming is usually a sign that the architecture of the system is bad and the software should be split up into separate libraries or web services.

chongli 13y ago
That all depends on the language. Lisp (pretty much all of its dialects) heavily emphasizes the use of macros to extend the language and build all kinds of control structures and embedded domain-specific languages (EDSLs).
- peterhunt 13y ago
  Absolutely. One could make the argument, though, that many (including me) believe that expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance.
  - sillysaurus 13y ago
    expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance. Well, Arc is actually very maintainable, and quite fast. After all, HN is powered by Arc, and it serves >100k daily uniques on a single core.
    - peterhunt 13y ago
      Yep! Not saying that it's impossible, just that some of the concepts that Lisp is based on make these things harder to pull off. The reason for this is that Lisp emphasizes flexibility, and the more flexible your runtime environment is, the more ways to do something, which makes it easier to have hard-to-maintain code (and harder to reason about performance, too). But it's not impossible to do with Lisp for sure.
      - chongli 13y ago
        Lisp macros also provide ways to take ugly, hand-optimized code and clean it up to provide a cleaner, friendlier interface. Like all macros, this comes at no runtime cost (since all macros have been fully expanded by this point).
      - sillysaurus 13y ago
        Actually, it's not any harder to write maintainable code in Arc. In many ways, it's much easier, because you write your program as a set of parts which fit together like an arch.
    - gohrt 13y ago
      HN is fast because it is feature-barren. That's a design choice pg is free to make (and cutting features is a great way to improve performance), but not a proof of Arc/Lisp's ability to support complex applications with high-performance.
      - sillysaurus 13y ago
        Which features would convince you that Arc is fast?
        waterhouse 13y ago
        Just for the record, let's count to 100 million. And add the numbers up while we're at it. ~ $ time (echo '(do (= n 0) (for i 1 100000000 (++ n i)) prn.n (quit))' | arc) Welcome to Racket v5.3.5.1. Use (quit) to quit, (tl) to return here after an interrupt. arc> 5000000050000000 real 0m28.561s user 0m28.308s sys 0m0.249s ---- ~ $ time (echo -e 'n=0 \ni=0 \nwhile (i <= 100000000): \n n += i \n i += 1 \n\nprint(n)\n' | python3.3) 5000000050000000 real 0m30.244s user 0m30.230s sys 0m0.013s It would appear to be competitive with Python on my machine on this particular task. (Also you can make it faster by dropping into Racket.)
        dzderic 13y ago
        A while loop and temporary mutable variables are definitely not the Pythonic way of doing this. More idiomatic: $ time python -c 'print sum(xrange(100000000 + 1))' 5000000050000000 real 0m1.398s user 0m1.383s sys 0m0.012s Comparison to baseline: $ time (echo -e 'n=0 \ni=0 \nwhile (i <= 100000000): \n n += i \n i += 1 \n\nprint(n)\n' | python) 5000000050000000 real 0m33.140s user 0m32.939s sys 0m0.023s
        waterhouse 13y ago
        I see. That is good to know; I merely chose mutating global variables in a loop because I knew how to do that in both languages. (I am not very familiar with Python.) That is not the idiomatic way to do it in Arc, either. I would normally use a recursive function, like this: arc> (time:xloop (i 0 n 0) (if (> i 100000000) n (next (+ i 1) (+ n i)))) time: 9121 cpu: 9130 gc: 0 mem: 480 ; the times are in msec 5000000050000000 Or perhaps a "higher-order function": arc> (time:sum idfn 1 100000000) time: 19889 cpu: 19908 gc: 0 mem: 1224 5000000050000000 Or use a deforestation macro that I wrote, which is closest to your Python example: arc> (time:thunkify:reduce + (range 1 100000000)) time: 17971 cpu: 17985 gc: 0 mem: 3592 5000000050000000 Also, here's what you can get by dropping into Racket: arc> (time:$:let loop ((i 0) (n 0)) (if (> i 100000000) n (loop (+ i 1) (+ n i)))) time: 402 cpu: 403 gc: 0 mem: 920 5000000050000000 I suppose Python has an analogue of that--dropping into C, or at least loading C libraries. Which Racket can do too. Mmm.
  - lispm 13y ago
    Common Lisp was designed for development of large/complex applications ('applications' really means here software that is used in production) with good performance. For example this CAD system from PTC is based on 6+ million lines of Common Lisp code and under development for two decades: http://www.ptc.com/product/creo/direct http://www.ptc.com/product/creo/direct
- cgag 13y ago
  Lisp macros are a lot simpler and easier to understand than what goes on with the types of metaprogramming done in ruby/rails. Expanding a macro is a lot easier for me to wrap my head around than stuff like dynamically defining methods when something is subclassed or when magic happens in method_missing.
- Homunculiheaded 13y ago
  At the same time the Lisp community is one of the fiercest in supporting the idea that nothing that can be a function should be a macro. Metaprogramming just because it is clever is more abhorred in the Lisp community than any other I've seen.
  - chongli 13y ago
    Of course! Macros are distinctly less powerful than functions for one reason: they are not first class values at runtime and thus cannot be passed to higher-order functions.
    - mst 13y ago
      This is solved by using an operative lisp - fexprs (or 'vau'/'$vau' in kernel and wat) are non-applicative combiners that execute at runtime, so you can have a first class combiner that chooses when/how/how many times to evaluate its arguments. I've been having a lot of fun recently writing code in wat-pl (my perl port of Manuel Simonyi's wat-js interpreter)
- marshray 13y ago
  Yep, and it surely depends upon the problem being solved too.
- dustingetz 13y ago
  The heavy emphasis of macros in Lisp may also be its greatest weakness. I see all sorts of Clojure code that uses macros where they needn't have, and it tremendously impedes my ability to use the code in ways the library author may not have expected. I love the way petehunt put it - metaprogramming is a last resort, you should feel bad when you use it, and language designers should include it but deemphasize it. (Scala and Haskell nail it I think; Clojure perhaps emphasizes macros a bit too much which harms the output of intermediate programmers. But, Clojure is not designed for intermediate programmers.)

In my experience overuse of meta programming is usually a sign that the architecture of the system is bad and the software should be split up into separate libraries or web services.

chongli 13y ago

That all depends on the language. Lisp (pretty much all of its dialects) heavily emphasizes the use of macros to extend the language and build all kinds of control structures and embedded domain-specific languages (EDSLs).

peterhunt 13y ago
Absolutely. One could make the argument, though, that many (including me) believe that expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance.
- sillysaurus 13y ago
  expressive power and conceptual beauty is Lisp's core strength, not code maintainability or runtime performance. Well, Arc is actually very maintainable, and quite fast. After all, HN is powered by Arc, and it serves >100k daily uniques on a single core.
  - peterhunt 13y ago
    Yep! Not saying that it's impossible, just that some of the concepts that Lisp is based on make these things harder to pull off. The reason for this is that Lisp emphasizes flexibility, and the more flexible your runtime environment is, the more ways to do something, which makes it easier to have hard-to-maintain code (and harder to reason about performance, too). But it's not impossible to do with Lisp for sure.
    - chongli 13y ago
      Lisp macros also provide ways to take ugly, hand-optimized code and clean it up to provide a cleaner, friendlier interface. Like all macros, this comes at no runtime cost (since all macros have been fully expanded by this point).
    - sillysaurus 13y ago
      Actually, it's not any harder to write maintainable code in Arc. In many ways, it's much easier, because you write your program as a set of parts which fit together like an arch.
  - gohrt 13y ago
    HN is fast because it is feature-barren. That's a design choice pg is free to make (and cutting features is a great way to improve performance), but not a proof of Arc/Lisp's ability to support complex applications with high-performance.
    - sillysaurus 13y ago
      Which features would convince you that Arc is fast?
      - waterhouse 13y ago
        Just for the record, let's count to 100 million. And add the numbers up while we're at it. ~ $ time (echo '(do (= n 0) (for i 1 100000000 (++ n i)) prn.n (quit))' | arc) Welcome to Racket v5.3.5.1. Use (quit) to quit, (tl) to return here after an interrupt. arc> 5000000050000000 real 0m28.561s user 0m28.308s sys 0m0.249s ---- ~ $ time (echo -e 'n=0 \ni=0 \nwhile (i <= 100000000): \n n += i \n i += 1 \n\nprint(n)\n' | python3.3) 5000000050000000 real 0m30.244s user 0m30.230s sys 0m0.013s It would appear to be competitive with Python on my machine on this particular task. (Also you can make it faster by dropping into Racket.)
        dzderic 13y ago
        A while loop and temporary mutable variables are definitely not the Pythonic way of doing this. More idiomatic: $ time python -c 'print sum(xrange(100000000 + 1))' 5000000050000000 real 0m1.398s user 0m1.383s sys 0m0.012s Comparison to baseline: $ time (echo -e 'n=0 \ni=0 \nwhile (i <= 100000000): \n n += i \n i += 1 \n\nprint(n)\n' | python) 5000000050000000 real 0m33.140s user 0m32.939s sys 0m0.023s
        waterhouse 13y ago
        I see. That is good to know; I merely chose mutating global variables in a loop because I knew how to do that in both languages. (I am not very familiar with Python.) That is not the idiomatic way to do it in Arc, either. I would normally use a recursive function, like this: arc> (time:xloop (i 0 n 0) (if (> i 100000000) n (next (+ i 1) (+ n i)))) time: 9121 cpu: 9130 gc: 0 mem: 480 ; the times are in msec 5000000050000000 Or perhaps a "higher-order function": arc> (time:sum idfn 1 100000000) time: 19889 cpu: 19908 gc: 0 mem: 1224 5000000050000000 Or use a deforestation macro that I wrote, which is closest to your Python example: arc> (time:thunkify:reduce + (range 1 100000000)) time: 17971 cpu: 17985 gc: 0 mem: 3592 5000000050000000 Also, here's what you can get by dropping into Racket: arc> (time:$:let loop ((i 0) (n 0)) (if (> i 100000000) n (loop (+ i 1) (+ n i)))) time: 402 cpu: 403 gc: 0 mem: 920 5000000050000000 I suppose Python has an analogue of that--dropping into C, or at least loading C libraries. Which Racket can do too. Mmm.
- lispm 13y ago
  Common Lisp was designed for development of large/complex applications ('applications' really means here software that is used in production) with good performance. For example this CAD system from PTC is based on 6+ million lines of Common Lisp code and under development for two decades: http://www.ptc.com/product/creo/direct http://www.ptc.com/product/creo/direct
cgag 13y ago
Lisp macros are a lot simpler and easier to understand than what goes on with the types of metaprogramming done in ruby/rails. Expanding a macro is a lot easier for me to wrap my head around than stuff like dynamically defining methods when something is subclassed or when magic happens in method_missing.
Homunculiheaded 13y ago
At the same time the Lisp community is one of the fiercest in supporting the idea that nothing that can be a function should be a macro. Metaprogramming just because it is clever is more abhorred in the Lisp community than any other I've seen.
- chongli 13y ago
  Of course! Macros are distinctly less powerful than functions for one reason: they are not first class values at runtime and thus cannot be passed to higher-order functions.
  - mst 13y ago
    This is solved by using an operative lisp - fexprs (or 'vau'/'$vau' in kernel and wat) are non-applicative combiners that execute at runtime, so you can have a first class combiner that chooses when/how/how many times to evaluate its arguments. I've been having a lot of fun recently writing code in wat-pl (my perl port of Manuel Simonyi's wat-js interpreter)
marshray 13y ago
Yep, and it surely depends upon the problem being solved too.
dustingetz 13y ago
The heavy emphasis of macros in Lisp may also be its greatest weakness. I see all sorts of Clojure code that uses macros where they needn't have, and it tremendously impedes my ability to use the code in ways the library author may not have expected. I love the way petehunt put it - metaprogramming is a last resort, you should feel bad when you use it, and language designers should include it but deemphasize it. (Scala and Haskell nail it I think; Clojure perhaps emphasizes macros a bit too much which harms the output of intermediate programmers. But, Clojure is not designed for intermediate programmers.)