10 ms·

Could someone please explain the difference between Lisp macros and, say, languages that have first-class functions? I get that a Lisp macro will be expanded in

Could someone please explain the difference between Lisp macros and, say, languages that have first-class functions? I get that a Lisp macro will be expanded into the respective code, while a function's execution is different. However, at the practical (i.e., developer's) level, are there any additional benefits? Can, say, a Lisp macro be 'partially formed', in the sense that it can expand into some boilerplate that represents an incomplete syntax tree? (Whereas a higher-order-function is necessarily complete.) I can see that being useful, but not inasmuch a it's made out.

coryrc 11y ago
Macros are run before your program is running with actual data (during "compile", if you will). The advantage of CL macros is you can use functions written for your program to use at run-time during compilation-time/macro evaluation too.
dfan 11y ago
One qualitative difference in expressive power comes from the fact that functions first evaluate all their arguments and macros don't.
- kazinator 11y ago
  Ah, but that is not the case in some languages in which arguments are evaluated lazily. Usually the iron-man version of this question is: "if I have a non-strictly evaluated language with higher order functions, do I still need macros?" A part of the answer is: you probably don't need the kinds of macros which cover up machine-generated lambdas, which simulate non-strict evaluation in strictly evaluated Lisp programs! The compiler for your language already has these "macros" in its compiler. The argument why "you always need macros no matter what else you have" is that your language has "hard-coded macros": the grammar rules in a compiler, which match patterns and transform bits and pieces to produce AST fragments. If you don't have macros, then that set of "hard-coded macros" is all you have. So, even in a functional language with nonstrict evaluation, you're using macros. It's hard to make a convincing argument that they provide all the expressivity you would conceivably ever need. (And their existence and use defeats any argument that you don't need macros at all).
  - the_af 11y ago
    I'm not entirely convinced. Which definition of "macro" are you using, if you equate lazy evaluation with a kind of "hardcoded macro"? More importantly, "it's hard to make a convincing argument that [these hardcoded macros] provide all the expressivity you would conceivably ever need" doesn't convince me. A better argument would be to produce a compelling example where these "macros" are not enough. And by compelling, I mean something that cannot be elegantly produced in a non-Lisp language.
    - kazinator 11y ago
      > Which definition of "macro" are you using, if you equate lazy evaluation with a kind of "hardcoded macro"? That definition of "macro" equivalent with "phrase structure rule in your functional language's compiler" which takes the input structure and generates whatever code brings about the lazy semantics (which is not inherent in the x86 instruction set or what have you). > example where these "macros" are not enough An example is any instance of language extension where, say, the maintainers of the compiler for a functional language have to ship a new compiler to the users to get them to use the new feature. Functional languages with lazy evaluation are not finished, right? They are developed actively.
      - the_af 11y ago
        > An example is any instance of language extension where, say, the maintainers of the compiler for a functional language have to ship a new compiler to the users to get them to use the new feature. > Functional languages with lazy evaluation are not finished, right? They are developed actively. Agreed, they are actively developed. Correct me if I'm wrong, but you seem to be saying "many interesting features can be implemented in a Lisp language with a macro, therefore Lisp programmers don't need to wait for a new release of their programming language when they want these features". I'm not convinced this is the case, or rather, that this is such a relevant case. Aren't Lisps actively developed too? Why is there such a multitude of Lisp implementations? Is any relevant real-world feature truly implementable with Lisp macros? Why is that more convenient than implementing them with functions in languages with lazy evaluation? Maybe I'm falling prey to the Blub paradox. I'm only passingly familiar with Racket, thanks to a course in Coursera, where they introduce macros and why they are so powerful in Lisp. But I still don't see the compelling "killer example"...
typedweb 11y ago
A lisp macro is a code transformer run by the compiler using the full features of the language to generate code. Macros allow for the arbitrary evaulation of it's arguments (rather than the standard left to right order before a function call), allowing you to do syntactic extensions without the added boilerplate that functional languages can require. Essentially, each macro allows you to define a mini-language that is parsed by the compiler that returns code that is then compiled. It takes a while to groc, but once you do you can never really go back.
ThrustVectoring 11y ago
This is only ~90% likely to be correct, since it's second-hand information and I don't use LISP actively. A LISP macro is a syntax transformation. It lets you write code in the way you want to, instead of whatever level of abstraction you used to have. I'm not sure about 'partially formed' code output, but 'partially formed' input is definitely possible. The way to invoke a LISP macro need not be valid LISP. In short, LISP macros excel at creating domain-specific languages.
- frou_dh 11y ago
  In my Lisp-esque language I use a temporary macro to automate the creation of some similar standard library procedures. This happens at runtime in the stdlib source file that is loaded: # Define procedures named int? float? etc that test the type of a value. (def def-type-predicate (mac (type-name) `(def ,(string-to-symbol (join "" $type-name "?")) (proc (x) (eq? (type x) ',type-name))))) (def-type-predicate int) (def-type-predicate float) (def-type-predicate bool) (def-type-predicate string) (def-type-predicate symbol) (def-type-predicate file) (def-type-predicate nil) (def-type-predicate pair) (def-type-predicate procedure) (def-type-predicate macro) (zap def-type-predicate) The crucial thing about what happens there being that the (def foo? ...) value produced by each macro invocation then gets evaluated in the root/top-level environment and so results in a "global" procedure definition. Using them: (string? (add 2 2)) => FALSE (float? 4.7) => TRUE (procedure? string?) => TRUE I thought it was a nice contained example of "code writing code" in the data realm.
  - GFK_of_xmaspast 11y ago
    Python you can do: def istype(i, t): return (type(i)==t) and istype(5, int) doesn't seem that much different than (int? 5) I bet there's a version in c++ with templates and typeid.
    - frou_dh 11y ago
      Yeah, I could do similar open testing: (def is-type (proc (x t) (eq? (type x) t))) (is-type 5 'int) But it's nice to have single-parameter ones for FP list stuff. I suppose that 2-parameter version could have their order swapped and do partial application on top of it. Anyway, was just sharing something I had fun making. No language wars intended.
beeworker 11y ago
One difference is flow control. When you call a function, all the arguments are evaluated before being passed into the function. If you want to delay evaluation, you have to wrap the argument values in a function. When you call a macro, the text forms get passed with no evaluation. Say Clojure forgot to ship with the boolean "or". "or" should evaluate its arguments one at a time (to allow for short circuiting) and return the first non-false value. You could do this with functions, but you have the source-level overhead of manually wrapping everything, and performance overhead of defining and passing an anonymous function for each argument. With a macro, at compile time you just translate the "or" macro into a simpler form that uses "if". This is how Clojure actually implements "or": (defmacro or "Evaluates exprs one at a time, from left to right. If a form returns a logical true value, or returns that value and doesn't evaluate any of the other expressions, otherwise it returns the value of the last expression. (or) returns nil." {:added "1.0"} ([] nil) ([x] x) ([x & next] `(let [or# ~x] (if or# or# (or ~@next)))))
- platz 11y ago
  I wonder how much macros are necessary one you add call-by-name parameters e.g. scala.
  - vog 11y ago
    Or, once you have lazy evaluation (like in Haskell), for that matter.
    - jlarocco 11y ago
      Laziness can achieve a similar thing in that case (less elegantly, IMO), but there are other uses for macros that can't be solved with laziness.
      - mbrock 11y ago
        Look at the uses for Template Haskell, for example. That's basically Haskell's macro system. And it's used quite a lot, though it's kind of arcane. If you want to create an abstraction that defines one or several data types, you'll think hard about whether you can use some kind of type-level programming instead—but if that's not possible, or not convenient, you can use TH macros. For example, the `lens` package defines TH macros for creating special kinds of accessors that are tedious to write by hand.
chaoky 11y ago
A higher order function doesn't serve the same purpose as a macro. A higher order function is meant to be applied, called, composed etc. A lisp macro is a different type of abstraction. For example, many people think that macros are just hiding lambda's of higher order functions. This is wrong. A macro abstracts over implementation details of a construct to make it read naturally. For example, you can write a function that opens and then closes a file like so... with-open-file(filename, lambda file: do stuff with file) but with a lisp macro, you only have to write with-open-file (filename): do stuff with file The point of an abstraction is so you don't have to think about the implementation. Written like the latter, with-open-file is simply more natural to write this way. You only have to think, "oh, its a construct that opens a file then closes it after the body is done", rather than "oh, its a higher order function that i have to pass another function into that takes the file as an argument..." etc. When you write with-open-file as a macro, it could be implemented as a higher order function, or it could be implemented as a low level set of GOTO statements. It doesn't matter. The macro abstracts away the low level detail, just providing the most natural way for you to use the construct. It might not seem like the macro is doing much in that particular example, but a construct that defines a class (like defclass) is something you can write as a macro, which can expand into functions that do the actual defining. You could write a class defining construct as a higher order function, but then you'd have to constantly worry about how your construct was implemented as a function. instead of just writing something natural like defclass tiger (animal): age init-value: 0 name type: string which you could write if you implemented defclass as a macro. otherwise, you'd have to do something crazy stupid like defclass(name='tiger', inherits-from = find-class('animal') slots=['age, name'] ...) how could a higher order function possibly implement that without making people using it tear their hair out? They have to bend to the implementation, not make the abstraction bend to what's natural.
- eatonphil 11y ago
  In what language are you writing?
  - mbrock 11y ago
    That might be pseudo-code, but Dylan has a similar syntax, like Lisp without the explicit S-expressions.
chaoky 11y ago
A higher order function doesn't serve the same purpose as a macro. A higher order function is meant to be applied, called, composed etc. A lisp macro is a different type of abstraction. For example, many people think that macros are just hiding lambda's of higher order functions. This is wrong. A macro abstracts over the detail. For example, you can write a function that opens and then closes a file like so... with-open-file(filename, lambda file: do stuff with file) but with a lisp macro, you only have to write with-open-file filename:
kazinator 11y ago
TXR Lisp is completely strictly evaluated, like many other Lisp dialects. Function argument expressions are reduced to their values, in left to right order. Then the application of the resulting values to the function takes place. Yet, with macros I have this: ./txr -p '(mlet ((x (lcons 1 x))) x)' (1 1 1 1 1 1 1 1 1 ... nonterminating sequence of 1's ... This created a circular list! The special mlet ("magic let" or "mutual let") construct has allowed the expression which initializes s, (lcons 1 x), to refer to x. This works because both mlet and lcons are macros. The lcons macro (rather, the code generated by the macro!) returns a lazy cons cell, without immediately evaluating its arguments 1 and x. In the case of x, this is a damn good thing because x is not yet initialized! When the lazy cons is accessed (when the list object is printed), the evaluation of x takes place. By that time, x holds the lazy cons cell, and since the variable x is in scope of the argument x in (lcons 1 x), the lazy cons is able to force, setting its CDR field back to itself, creating not a lazy list, but a circular list. With lcons, I we can make a fibonacci function quite similarly to how you might do it in Haskell: (defun fib2 (a b) (lcons a (fib2 b (+ a b)))) We call this as (fib 1 1) and it gives us a lazy list. Here, we have a marriage between a lazy data structure and a macro. Without lazy conses, the lcons macro would have no target language to expand into. Without the lcons macro, lazy conses can't be used in the above convenient way; we would have to write fib2 in terms of the macro expansion: $ ./txr -p "(sys:expand '(defun fib2 (a b) (lcons a (fib2 b (+ a b)))))" (defun fib2 (a b) (make-lazy-cons (lambda (#:lcons-0001) (rplaca #:lcons-0001 a) (rplacd #:lcons-0001 (fib2 b (+ a b)))))) The circular list mlet, when fully expanded looks like this, by the way: $ ./txr -p "(sys:expand '(mlet ((x (lcons 1 x))) x))" (let (#:g0001) (sys:setq #:g0001 (cons 'sys:promise (cons (lambda () (make-lazy-cons (lambda (#:lcons-0046) (rplaca #:lcons-0046 1) (rplacd #:lcons-0046 (force #:g0001))))) '(delay (lcons 1 x))))) (force #:g0001)) It's a gritty oat-meal of delays, lambdas, forces, and cons manipulation. One thing that is conspicuously absent amid the toenail clippings: what happened to the x variable? Haha!
snikeris 11y ago
My favorite example for this is the lame idiom you see in Java code: if (log.isDebugEnabled()) { log.debug("expensive" + debug + message); } This is "better" than just log.debug(...) because with the latter, your expensive log message argument needs to be evaluated even if debug is disabled. However, in a language w/ macros, you just say: (debug (str "expensive" debug message)) and these considerations are already taken care of for you: https://github.com/clojure/tools.logging/blob/master/src/main/clojure/clojure/tools/logging.clj#L91 https://github.com/clojure/tools.logging/blob/master/src/mai...
- TenDecigrams 11y ago
  Edit: Ohhh! I see! The message isn't evaluated until after! Got it. I don't understand why this is special? Why can't you rewrite log.debug to include the check? func debug(m string) { if log.isDebugEnabled() { log.debug(m); } } What makes macros special in this case?
  - dragonwriter 11y ago
    The point was that "debug" and "message" parts in the example were arbitrarily complex expressions which were computationally expensive to evaluate. Your wrapping of log debug would still require evaluating them to get the message string unconditionally, even when debug logging is not enabled. Of course, with support for first-class functions, you could do something like: func debug(produceMessage ()->string) { if log.isDebugEnabled() { log.debug(produceMessage()); } }
    - snikeris 11y ago
      And then what would I need to type to write a log message?
      - ufo 11y ago
        You pass a lambda that returns the debug message. With javascripty syntax: debug( function(){ return "my expensive log message" }) BTW, if your language is lazily evaluated (like Haskell) then you don't need to do this because arguments will only ve evaluated when they are needed.
crististm 11y ago
The sheer number of answers you've got should tell you that indeed there is something about those macros. But you need to 'get' them yourself to appreciate them. Each answer exposes one or more facets of macros. Indeed, there is not just 'one' thing that make them worthwhile.
aidenn0 11y ago
The majority of macros I write could be represented with HOF and lexically-closed lambdas. That adds significant extra syntax when you use them though. Consider a classic pattern of a with macro: (with-mutex-held-macro (some-mutex) (do-stuff)) (call-with-mutex-held-hof some-mutex (lambda () (do-stuff)) A minor advantage is that a macro will be expanded in-line; a Sufficiently Smart Compiler could transform the HOF version into something equivalent, so it's not strictly an advantage (except to compiler implementers I suppose). I think that e.g. generalized references (i.e. the common lisp setf macro) are not possible with HOF, but HOF is more common in pure languages where assignment is eschewed anyway. Mark Jason Dominus (author of Higher Order Perl) had some good comments on lisp macros as well: http://lists.warhead.org.uk/pipermail/iwe/2005-July/000130.html http://lists.warhead.org.uk/pipermail/iwe/2005-July/000130.h...
- dllthomas 11y ago
  One thing to note is that you're using the macro or lambda to delay evaluation. In a lazy-by-default language, that's unnecessary (which is a part of why macros are less useful in Haskell).
  - eru 11y ago
    True. They are still useful enough for Template Haskell to exist.
    - dllthomas 11y ago
      For sure.
  - wtetzner 11y ago
    There's a class of things that don't require macros in Haskell, but I don't think that means macros are less useful in Haskell. There are plenty of things you might want them for, like generating new definitions.
    - dllthomas 11y ago
ScottBurson 11y ago
It's true, as others have explained, that for many of the most common uses of macros, you can get the same effect with a higher-order function. But since macros operate at the syntax level, they can do things functions can't do. For example, they can generate and manipulate declarations. Say you're working with abstract syntax trees (assembly trees in a CAD app might be another example). These trees are built from nodes, where each node is of some class corresponding to a syntactic construct: if-statement, addition-expression, etc. etc. There's some functionality you want to have on every node class; a common example is a "children" method that gathers up all the node's child slots into a set and returns it. It is very convenient to have a 'define-node-class' macro that automatically generates the 'children' method, so that when you add a child slot, the method is updated automatically; there's no need for manual effort to keep them in sync. In this case, the macro is expanding to multiple top-level declarations: the class declaration along with the method declaration (probably, in practice, several methods). Higher-order functions don't begin to let you do stuff like this.
- GFK_of_xmaspast 11y ago
  How's that different than what other languages call 'inheritance.'
  - ScottBurson 11y ago
    Because most node classes have their own slots. Consider this example: class Node {...} class Expression extends Node {...} class Addition extends Expression { Expression left, Expression right; Set<Node> children() { return [a set containing 'left' and 'right']; } } There's no way to write a single method 'children' on Node that will work for all its subclasses, because the method on Node can't access the subclasses' slots -- unless you use reflection, which is ugly and slow.
vorg 11y ago
Macros can provide a lot of syntactic convenience over those first-class functions, especially with heavily nested structures. For example, I can replace this monadic parser definition... function SpecDeclP()Parser{ return Bind(SeqRight(MyKeywordP, IdentifierP), function(nm interface{})Parser{ return Bind(BetweenParensP(IdentifierP), function(parm interface{})Parser{ return Bind(SetSwitchUserStateP, function(_ interface{})Parser{ return Bind(BlockP, function(bod interface{})Parser{ return Result("func " + nm.(string) + "(" + parm.(string) + ")" + bod.(string)) }) }) }) }) } ...with this more readable version if I have a recursively defined "macro" called doparse... doparse{ nm <- SeqRight(MyKeywordP, IdentifierP) parm <- BetweenParensP(IdentifierP) SetSwitchUserStateP bod <- BlockP Result("func " + nm.(string) + "(" + parm.(string) + ")" + bod.(string)) } This requirement arose in Haskell for its Parsec before it became a part of the syntax, then again later with the Arrow library which later also added it to the syntax. Whenever new abstractions are discovered/created, a macroing facility, whether for lisp-like syntax or some other, helps makes all the nested functions more readable.
- DanWaterworth 11y ago
  I'm fairly sure that Haskell had do notation before Parsec.
calibraxis 11y ago
You can support new paradigms, just as a library. For example, core.async was written as a macro. So Clojure basically got Go channels in a library, not a language upgrade. Macros can fundamentally change code. Codewalkers. Back when OOP was a big thing (probably still is), Lisp programmers would probably amuse themselves by making object-oriented extensions to the language via macros, and sending them to each other. This means that you are empowered, not just a language implementer. (First-class functions are important, and typically a better idea than "Guess I'll write my own macro!" but they only go so far.)
jlarocco 11y ago
Your question doesn't make sense. High order functions and macros are orthogonal concepts. Lisp supports first-class, high order functions. In fact, it was probably the first language to do so. Macros are different. And they're not just "expanded." Think of macros as full blown Lisp functions that run at compile time and generate new Lisp code, that is itself compiled at compile time. Since Lisp code is represented as Lisp's list data structure, it's super easy. The macro system does allow expansion/replacement (like C's preprocessor), but that's just scratching the surface.
TuringTest 11y ago
The first order function is evaluated at runtime, every time the program is executed. Macros are expanded at compile time, so they will be calculated just once. Thus the syntactic sugar added by the macro doesn't incur a time penalty. This can provide an important speed improvement for complex macros or code used often, in tight loops or frequently called functions (it's like using inline methods in .h files in C++).
dschiptsov 11y ago
The difference is subtle. With high order function you coule define (abstract out) a new idiom, while with proper macros you could define a new special form. In case of using high-order procedure all its arguments will be evaluated in order before application of the procedure while in case if a macro you could explicitly define all the transformations (evaluation rules) for each argument. This is why it is called a special form - it has its own evalustion rules. Shortcircuiting if or and are canonical examples. So, with macros you are extending Lisp with new special forms.

coryrc 11y ago

Macros are run before your program is running with actual data (during "compile", if you will). The advantage of CL macros is you can use functions written for your program to use at run-time during compilation-time/macro evaluation too.

dfan 11y ago

One qualitative difference in expressive power comes from the fact that functions first evaluate all their arguments and macros don't.

kazinator 11y ago
Ah, but that is not the case in some languages in which arguments are evaluated lazily. Usually the iron-man version of this question is: "if I have a non-strictly evaluated language with higher order functions, do I still need macros?" A part of the answer is: you probably don't need the kinds of macros which cover up machine-generated lambdas, which simulate non-strict evaluation in strictly evaluated Lisp programs! The compiler for your language already has these "macros" in its compiler. The argument why "you always need macros no matter what else you have" is that your language has "hard-coded macros": the grammar rules in a compiler, which match patterns and transform bits and pieces to produce AST fragments. If you don't have macros, then that set of "hard-coded macros" is all you have. So, even in a functional language with nonstrict evaluation, you're using macros. It's hard to make a convincing argument that they provide all the expressivity you would conceivably ever need. (And their existence and use defeats any argument that you don't need macros at all).
- the_af 11y ago
  I'm not entirely convinced. Which definition of "macro" are you using, if you equate lazy evaluation with a kind of "hardcoded macro"? More importantly, "it's hard to make a convincing argument that [these hardcoded macros] provide all the expressivity you would conceivably ever need" doesn't convince me. A better argument would be to produce a compelling example where these "macros" are not enough. And by compelling, I mean something that cannot be elegantly produced in a non-Lisp language.
  - kazinator 11y ago
    > Which definition of "macro" are you using, if you equate lazy evaluation with a kind of "hardcoded macro"? That definition of "macro" equivalent with "phrase structure rule in your functional language's compiler" which takes the input structure and generates whatever code brings about the lazy semantics (which is not inherent in the x86 instruction set or what have you). > example where these "macros" are not enough An example is any instance of language extension where, say, the maintainers of the compiler for a functional language have to ship a new compiler to the users to get them to use the new feature. Functional languages with lazy evaluation are not finished, right? They are developed actively.
    - the_af 11y ago
      > An example is any instance of language extension where, say, the maintainers of the compiler for a functional language have to ship a new compiler to the users to get them to use the new feature. > Functional languages with lazy evaluation are not finished, right? They are developed actively. Agreed, they are actively developed. Correct me if I'm wrong, but you seem to be saying "many interesting features can be implemented in a Lisp language with a macro, therefore Lisp programmers don't need to wait for a new release of their programming language when they want these features". I'm not convinced this is the case, or rather, that this is such a relevant case. Aren't Lisps actively developed too? Why is there such a multitude of Lisp implementations? Is any relevant real-world feature truly implementable with Lisp macros? Why is that more convenient than implementing them with functions in languages with lazy evaluation? Maybe I'm falling prey to the Blub paradox. I'm only passingly familiar with Racket, thanks to a course in Coursera, where they introduce macros and why they are so powerful in Lisp. But I still don't see the compelling "killer example"...

typedweb 11y ago

A lisp macro is a code transformer run by the compiler using the full features of the language to generate code. Macros allow for the arbitrary evaulation of it's arguments (rather than the standard left to right order before a function call), allowing you to do syntactic extensions without the added boilerplate that functional languages can require. Essentially, each macro allows you to define a mini-language that is parsed by the compiler that returns code that is then compiled. It takes a while to groc, but once you do you can never really go back.

ThrustVectoring 11y ago

This is only ~90% likely to be correct, since it's second-hand information and I don't use LISP actively. A LISP macro is a syntax transformation. It lets you write code in the way you want to, instead of whatever level of abstraction you used to have. I'm not sure about 'partially formed' code output, but 'partially formed' input is definitely possible. The way to invoke a LISP macro need not be valid LISP. In short, LISP macros excel at creating domain-specific languages.

frou_dh 11y ago
In my Lisp-esque language I use a temporary macro to automate the creation of some similar standard library procedures. This happens at runtime in the stdlib source file that is loaded: # Define procedures named int? float? etc that test the type of a value. (def def-type-predicate (mac (type-name) `(def ,(string-to-symbol (join "" $type-name "?")) (proc (x) (eq? (type x) ',type-name))))) (def-type-predicate int) (def-type-predicate float) (def-type-predicate bool) (def-type-predicate string) (def-type-predicate symbol) (def-type-predicate file) (def-type-predicate nil) (def-type-predicate pair) (def-type-predicate procedure) (def-type-predicate macro) (zap def-type-predicate) The crucial thing about what happens there being that the (def foo? ...) value produced by each macro invocation then gets evaluated in the root/top-level environment and so results in a "global" procedure definition. Using them: (string? (add 2 2)) => FALSE (float? 4.7) => TRUE (procedure? string?) => TRUE I thought it was a nice contained example of "code writing code" in the data realm.
- GFK_of_xmaspast 11y ago
  Python you can do: def istype(i, t): return (type(i)==t) and istype(5, int) doesn't seem that much different than (int? 5) I bet there's a version in c++ with templates and typeid.
  - frou_dh 11y ago
    Yeah, I could do similar open testing: (def is-type (proc (x t) (eq? (type x) t))) (is-type 5 'int) But it's nice to have single-parameter ones for FP list stuff. I suppose that 2-parameter version could have their order swapped and do partial application on top of it. Anyway, was just sharing something I had fun making. No language wars intended.

beeworker 11y ago

One difference is flow control. When you call a function, all the arguments are evaluated before being passed into the function. If you want to delay evaluation, you have to wrap the argument values in a function. When you call a macro, the text forms get passed with no evaluation. Say Clojure forgot to ship with the boolean "or". "or" should evaluate its arguments one at a time (to allow for short circuiting) and return the first non-false value. You could do this with functions, but you have the source-level overhead of manually wrapping everything, and performance overhead of defining and passing an anonymous function for each argument. With a macro, at compile time you just translate the "or" macro into a simpler form that uses "if". This is how Clojure actually implements "or": (defmacro or "Evaluates exprs one at a time, from left to right. If a form returns a logical true value, or returns that value and doesn't evaluate any of the other expressions, otherwise it returns the value of the last expression. (or) returns nil." {:added "1.0"} ([] nil) ([x] x) ([x & next] `(let [or# ~x] (if or# or# (or ~@next)))))

platz 11y ago
I wonder how much macros are necessary one you add call-by-name parameters e.g. scala.
- vog 11y ago
  Or, once you have lazy evaluation (like in Haskell), for that matter.
  - jlarocco 11y ago
    Laziness can achieve a similar thing in that case (less elegantly, IMO), but there are other uses for macros that can't be solved with laziness.
    - mbrock 11y ago
      Look at the uses for Template Haskell, for example. That's basically Haskell's macro system. And it's used quite a lot, though it's kind of arcane. If you want to create an abstraction that defines one or several data types, you'll think hard about whether you can use some kind of type-level programming instead—but if that's not possible, or not convenient, you can use TH macros. For example, the `lens` package defines TH macros for creating special kinds of accessors that are tedious to write by hand.

chaoky 11y ago

A higher order function doesn't serve the same purpose as a macro. A higher order function is meant to be applied, called, composed etc. A lisp macro is a different type of abstraction. For example, many people think that macros are just hiding lambda's of higher order functions. This is wrong. A macro abstracts over implementation details of a construct to make it read naturally. For example, you can write a function that opens and then closes a file like so... with-open-file(filename, lambda file: do stuff with file) but with a lisp macro, you only have to write with-open-file (filename): do stuff with file The point of an abstraction is so you don't have to think about the implementation. Written like the latter, with-open-file is simply more natural to write this way. You only have to think, "oh, its a construct that opens a file then closes it after the body is done", rather than "oh, its a higher order function that i have to pass another function into that takes the file as an argument..." etc. When you write with-open-file as a macro, it could be implemented as a higher order function, or it could be implemented as a low level set of GOTO statements. It doesn't matter. The macro abstracts away the low level detail, just providing the most natural way for you to use the construct. It might not seem like the macro is doing much in that particular example, but a construct that defines a class (like defclass) is something you can write as a macro, which can expand into functions that do the actual defining. You could write a class defining construct as a higher order function, but then you'd have to constantly worry about how your construct was implemented as a function. instead of just writing something natural like defclass tiger (animal): age init-value: 0 name type: string which you could write if you implemented defclass as a macro. otherwise, you'd have to do something crazy stupid like defclass(name='tiger', inherits-from = find-class('animal') slots=['age, name'] ...) how could a higher order function possibly implement that without making people using it tear their hair out? They have to bend to the implementation, not make the abstraction bend to what's natural.

eatonphil 11y ago
In what language are you writing?
- mbrock 11y ago
  That might be pseudo-code, but Dylan has a similar syntax, like Lisp without the explicit S-expressions.

chaoky 11y ago

kazinator 11y ago

TXR Lisp is completely strictly evaluated, like many other Lisp dialects. Function argument expressions are reduced to their values, in left to right order. Then the application of the resulting values to the function takes place. Yet, with macros I have this: ./txr -p '(mlet ((x (lcons 1 x))) x)' (1 1 1 1 1 1 1 1 1 ... nonterminating sequence of 1's ... This created a circular list! The special mlet ("magic let" or "mutual let") construct has allowed the expression which initializes s, (lcons 1 x), to refer to x. This works because both mlet and lcons are macros. The lcons macro (rather, the code generated by the macro!) returns a lazy cons cell, without immediately evaluating its arguments 1 and x. In the case of x, this is a damn good thing because x is not yet initialized! When the lazy cons is accessed (when the list object is printed), the evaluation of x takes place. By that time, x holds the lazy cons cell, and since the variable x is in scope of the argument x in (lcons 1 x), the lazy cons is able to force, setting its CDR field back to itself, creating not a lazy list, but a circular list. With lcons, I we can make a fibonacci function quite similarly to how you might do it in Haskell: (defun fib2 (a b) (lcons a (fib2 b (+ a b)))) We call this as (fib 1 1) and it gives us a lazy list. Here, we have a marriage between a lazy data structure and a macro. Without lazy conses, the lcons macro would have no target language to expand into. Without the lcons macro, lazy conses can't be used in the above convenient way; we would have to write fib2 in terms of the macro expansion: $ ./txr -p "(sys:expand '(defun fib2 (a b) (lcons a (fib2 b (+ a b)))))" (defun fib2 (a b) (make-lazy-cons (lambda (#:lcons-0001) (rplaca #:lcons-0001 a) (rplacd #:lcons-0001 (fib2 b (+ a b)))))) The circular list mlet, when fully expanded looks like this, by the way: $ ./txr -p "(sys:expand '(mlet ((x (lcons 1 x))) x))" (let (#:g0001) (sys:setq #:g0001 (cons 'sys:promise (cons (lambda () (make-lazy-cons (lambda (#:lcons-0046) (rplaca #:lcons-0046 1) (rplacd #:lcons-0046 (force #:g0001))))) '(delay (lcons 1 x))))) (force #:g0001)) It's a gritty oat-meal of delays, lambdas, forces, and cons manipulation. One thing that is conspicuously absent amid the toenail clippings: what happened to the x variable? Haha!

snikeris 11y ago

My favorite example for this is the lame idiom you see in Java code: if (log.isDebugEnabled()) { log.debug("expensive" + debug + message); } This is "better" than just log.debug(...) because with the latter, your expensive log message argument needs to be evaluated even if debug is disabled. However, in a language w/ macros, you just say: (debug (str "expensive" debug message)) and these considerations are already taken care of for you: https://github.com/clojure/tools.logging/blob/master/src/main/clojure/clojure/tools/logging.clj#L91 https://github.com/clojure/tools.logging/blob/master/src/mai...

TenDecigrams 11y ago
Edit: Ohhh! I see! The message isn't evaluated until after! Got it. I don't understand why this is special? Why can't you rewrite log.debug to include the check? func debug(m string) { if log.isDebugEnabled() { log.debug(m); } } What makes macros special in this case?
- dragonwriter 11y ago
  The point was that "debug" and "message" parts in the example were arbitrarily complex expressions which were computationally expensive to evaluate. Your wrapping of log debug would still require evaluating them to get the message string unconditionally, even when debug logging is not enabled. Of course, with support for first-class functions, you could do something like: func debug(produceMessage ()->string) { if log.isDebugEnabled() { log.debug(produceMessage()); } }
  - snikeris 11y ago
    And then what would I need to type to write a log message?
    - ufo 11y ago
      You pass a lambda that returns the debug message. With javascripty syntax: debug( function(){ return "my expensive log message" }) BTW, if your language is lazily evaluated (like Haskell) then you don't need to do this because arguments will only ve evaluated when they are needed.

crististm 11y ago

The sheer number of answers you've got should tell you that indeed there is something about those macros. But you need to 'get' them yourself to appreciate them. Each answer exposes one or more facets of macros. Indeed, there is not just 'one' thing that make them worthwhile.

aidenn0 11y ago

The majority of macros I write could be represented with HOF and lexically-closed lambdas. That adds significant extra syntax when you use them though. Consider a classic pattern of a with macro: (with-mutex-held-macro (some-mutex) (do-stuff)) (call-with-mutex-held-hof some-mutex (lambda () (do-stuff)) A minor advantage is that a macro will be expanded in-line; a Sufficiently Smart Compiler could transform the HOF version into something equivalent, so it's not strictly an advantage (except to compiler implementers I suppose). I think that e.g. generalized references (i.e. the common lisp setf macro) are not possible with HOF, but HOF is more common in pure languages where assignment is eschewed anyway. Mark Jason Dominus (author of Higher Order Perl) had some good comments on lisp macros as well: http://lists.warhead.org.uk/pipermail/iwe/2005-July/000130.html http://lists.warhead.org.uk/pipermail/iwe/2005-July/000130.h...

dllthomas 11y ago
One thing to note is that you're using the macro or lambda to delay evaluation. In a lazy-by-default language, that's unnecessary (which is a part of why macros are less useful in Haskell).
- eru 11y ago
  True. They are still useful enough for Template Haskell to exist.
  - dllthomas 11y ago
    For sure.
- wtetzner 11y ago
  There's a class of things that don't require macros in Haskell, but I don't think that means macros are less useful in Haskell. There are plenty of things you might want them for, like generating new definitions.
  - dllthomas 11y ago

ScottBurson 11y ago

It's true, as others have explained, that for many of the most common uses of macros, you can get the same effect with a higher-order function. But since macros operate at the syntax level, they can do things functions can't do. For example, they can generate and manipulate declarations. Say you're working with abstract syntax trees (assembly trees in a CAD app might be another example). These trees are built from nodes, where each node is of some class corresponding to a syntactic construct: if-statement, addition-expression, etc. etc. There's some functionality you want to have on every node class; a common example is a "children" method that gathers up all the node's child slots into a set and returns it. It is very convenient to have a 'define-node-class' macro that automatically generates the 'children' method, so that when you add a child slot, the method is updated automatically; there's no need for manual effort to keep them in sync. In this case, the macro is expanding to multiple top-level declarations: the class declaration along with the method declaration (probably, in practice, several methods). Higher-order functions don't begin to let you do stuff like this.

GFK_of_xmaspast 11y ago
How's that different than what other languages call 'inheritance.'
- ScottBurson 11y ago
  Because most node classes have their own slots. Consider this example: class Node {...} class Expression extends Node {...} class Addition extends Expression { Expression left, Expression right; Set<Node> children() { return [a set containing 'left' and 'right']; } } There's no way to write a single method 'children' on Node that will work for all its subclasses, because the method on Node can't access the subclasses' slots -- unless you use reflection, which is ugly and slow.

vorg 11y ago

Macros can provide a lot of syntactic convenience over those first-class functions, especially with heavily nested structures. For example, I can replace this monadic parser definition... function SpecDeclP()Parser{ return Bind(SeqRight(MyKeywordP, IdentifierP), function(nm interface{})Parser{ return Bind(BetweenParensP(IdentifierP), function(parm interface{})Parser{ return Bind(SetSwitchUserStateP, function(_ interface{})Parser{ return Bind(BlockP, function(bod interface{})Parser{ return Result("func " + nm.(string) + "(" + parm.(string) + ")" + bod.(string)) }) }) }) }) } ...with this more readable version if I have a recursively defined "macro" called doparse... doparse{ nm <- SeqRight(MyKeywordP, IdentifierP) parm <- BetweenParensP(IdentifierP) SetSwitchUserStateP bod <- BlockP Result("func " + nm.(string) + "(" + parm.(string) + ")" + bod.(string)) } This requirement arose in Haskell for its Parsec before it became a part of the syntax, then again later with the Arrow library which later also added it to the syntax. Whenever new abstractions are discovered/created, a macroing facility, whether for lisp-like syntax or some other, helps makes all the nested functions more readable.

DanWaterworth 11y ago
I'm fairly sure that Haskell had do notation before Parsec.

calibraxis 11y ago

You can support new paradigms, just as a library. For example, core.async was written as a macro. So Clojure basically got Go channels in a library, not a language upgrade. Macros can fundamentally change code. Codewalkers. Back when OOP was a big thing (probably still is), Lisp programmers would probably amuse themselves by making object-oriented extensions to the language via macros, and sending them to each other. This means that you are empowered, not just a language implementer. (First-class functions are important, and typically a better idea than "Guess I'll write my own macro!" but they only go so far.)

jlarocco 11y ago

Your question doesn't make sense. High order functions and macros are orthogonal concepts. Lisp supports first-class, high order functions. In fact, it was probably the first language to do so. Macros are different. And they're not just "expanded." Think of macros as full blown Lisp functions that run at compile time and generate new Lisp code, that is itself compiled at compile time. Since Lisp code is represented as Lisp's list data structure, it's super easy. The macro system does allow expansion/replacement (like C's preprocessor), but that's just scratching the surface.

TuringTest 11y ago

The first order function is evaluated at runtime, every time the program is executed. Macros are expanded at compile time, so they will be calculated just once. Thus the syntactic sugar added by the macro doesn't incur a time penalty. This can provide an important speed improvement for complex macros or code used often, in tight loops or frequently called functions (it's like using inline methods in .h files in C++).

dschiptsov 11y ago

The difference is subtle. With high order function you coule define (abstract out) a new idiom, while with proper macros you could define a new special form. In case of using high-order procedure all its arguments will be evaluated in order before application of the procedure while in case if a macro you could explicitly define all the transformations (evaluation rules) for each argument. This is why it is called a special form - it has its own evalustion rules. Shortcircuiting if or and are canonical examples. So, with macros you are extending Lisp with new special forms.