10 ms·
How Lisp macros differ from static code-generation and metaprogramming
- tom_b 16y agoLisp macros remind me of dynamically generating SQL statements. If you've ever built a string in T-SQL or PL/SQL and then exec'd it, you're probably much closer to being productive with macros than you think. Of course, this is probably because dynamically generating SQL is something that I've done. Other programmers probably have their own mental models that warp into lisp.
- vegashacker 16y agoYeah, I think that's a fair way to start thinking about them. But Lisp macros, to extend your example, are more like if you could generate the SQL using SQL. Which sounds like a totally horrible thing actually. :) But with Lisp it makes much more sense because lisp programs are lists. If SQL programs were tables, then it'd be a better fit...
- jfm3 16y agoNice try blanco niño, but the following program runs fine. This doesn't detract from your overall point, but if you're trying to win over people who think the Boost macros are super awesome, this argument won't do it. #include <stdio.h> #define LOOP(n) \ int i; \ for (i = 0; i < n; i++) #define LOOP10 \ LOOP(10) printf("%d\n", i) int main(int argc, char *argv[]) { LOOP10; }
- mahmud 16y agoYesterday I was writing code that kept messing my Lisp's signal handling, so I wrote a FOR-DURATION macro that arms a timeout and makes sure whatever that runs in its body gets killed after N seconds. Here it is: (defmacro for-duration ((seconds) &body body) `(handler-case (bt:with-timeout (,seconds) ,@body) (bt:timeout () nil))) Five lines to alter the evaluation model of your language. Not bad. Use as: (for-duration (10) (loop (print "Infinite loop! where is your Godel now?"))) "bt" is the bordeaux-threads package, a portable Lisp library for thread programming.
- GregBuchholz 16y ago(defun for-duration (seconds body) (handler-case (bt:with-timeout seconds (body)) (bt:timeout () nil))) (for-duration 10 (lambda () (loop (print "Infinite loop! where is your Godel now?"))))
- makmanalp 16y agoI think Greg is trying to say "why the hell do we even need macros for this"?
- mahmud 16y agoWith functions you have to make a conscious effort to quote arguments, otherwise the strict evaluation will evaluate its arguments before the function. Also, with macros you can introduce you new "implicit" control and structure semantics; it's very common for forms to have an "implicit progn", or a block named NIL or similar.
- funptr 16y agoC99 + GCC extensions + POSIX: #define for_duration(seconds, body) \ { \ pthread_t tid_task, tid_watcher; \ \ void* task(void* arg) { \ body ; \ pthread_cancel(tid_watcher); \ return NULL; \ } \ \ void* watcher(void* arg) { \ sleep((seconds)); \ pthread_cancel(tid_task); \ return NULL; \ } \ \ pthread_create(&tid_task, NULL, &task, NULL); \ pthread_create(&tid_watcher, NULL, &watcher, NULL); \ pthread_join(tid_task, NULL); \ } Use as: for_duration(10, { while (true) { printf("Infinite loop! where is your Godel now?\n"); } }); Or with single brace-less statements: for_duration(10, while (true) printf("something\n") ); Lexical scope is sane: int i = 0; for_duration(10, { while (true) { printf("%d\n", i++); } }); My comparison isn't quite fair, because your threading library provides 'with-timeout', while pthreads doesn't. If you factored this out, say with a signature like: void with_timeout(unsigned int seconds, void (*func)(void)); (in analogy to the common lisp function), then the macro part becomes just four lines: #define for_duration(seconds, body) { \ void func() { body ; } \ with_timeout((seconds), &func); \ } I acknowledge that the lisp solution is rather more elegant. Also, my C macro is potentially dangerous because it is unhygenic. (It shadows outer declarations of 'task()', 'watcher()', 'tid_task', 'tid_watcher', and 'arg', in the body of 'body'.)
- mahmud 16y agoBy the way, don't confuse macros with backquote; the backquote facility is probably closer to C's preprocessors. I haven't heard this claim before, and I might not be entirely right, but it's what provides for the "fill in the blanks" type symbolic computing. For example: (defun sum (x y) `(,x + ,y = ,(+ x y))) This is a SUM function which looks like this when run: (sum 4 5) ==> (4 + 5 = 9) The backquote (`) says the following form and its sub-forms (if any) are all quoted. But you can poke holes in the quote and fill in with the value of outside variables, using comma (,). So in the example, everything is quoted in the body of sum, except three things: x, y, and (+ x y). Each of those is then evaluated yielding 4, 5 and 9, then, finally, the template is filled in (4 + 5 = 9). If you want to confirm this, you can change the + and = to any other symbol and it would still display. E.g: (defun sum (x y) `(,x frob ,y != ,(+ x y))) (sum 4 5) ==> (4 FROB 5 != 9)
- arethuza 16y agoYou can also splice values into backquoted expressions using ,@foo rather than ,foo - which I seem to remember being rather useful.
- zck 16y agoIt's incredibly useful. The actual definition of until from Arc: (mac until (test . body) `(while (no ,test) ,@body)) It's really useful for bodies of code like this.
- sedachv 16y agoDon't forget that `, (backquote-comma) can also be adapted for pattern matching/destructuring (http://www.cliki.net/fare-matcher http://www.cliki.net/fare-matcher does this). I think combining backquote-comma's s-exp-making and s-exp-destructuring powers is a viable way to abstract s-exps from their representations as linked lists/arrays/whatever sequence you want to use.
- wingo 16y agoI like Conrad Barski's take on explaining quasiquote: http://www.lisperati.com/looking.html http://www.lisperati.com/looking.html "Flip, flop, flip, flop..."
- swannodette 16y agoNice post, though I find the summarization of SICP and the Little Schemer as academic texts inaccurate. The paradigms covered in them are of greater importance than macros - that is, your macros are only as powerful as the abstractions your language gives you.
- grayrest 16y agoA couple points on the metaprogramming space (these aren't about the article, but the article reminded me of them): 1. Macros aren't unique to lisp. They're most developed and used in the lisp family, but you don't need to have a homoiconic language to have macros. The Mirah programming language--statically typed Ruby on the JVM--has non-hygenic macros, I know there's been a number of efforts to add macros to coffeescript (so far without success), and there are a number of academic languages that also have them. None are in widespread use, but they exist. 2. Metaprogramming's power is primarily limited by the restrictions of the host language. The best example I know of is Io where pretty much anything other than commas and parentheses is up for grabs. Example: http://anttih.com/blog/2010/10/29/json-in-io.html http://anttih.com/blog/2010/10/29/json-in-io.html 3. User definable mixfix operators are in the sameish domain. Unfortunately, they're almost always mentioned in passing by people who know what they are so I don't have a good reference for introducing what they are and don't have direct experience working with them. I'm just mentioning them as something to be aware of if you're interested in general metaprogramming concepts. Here's the closest thing to an intro I could google up: http://maude.cs.uiuc.edu/maude1/manual/maude-manual-html/maude-manual_31.html http://maude.cs.uiuc.edu/maude1/manual/maude-manual-html/mau...
- eru 16y agoFor (1) look at OCaml's P4 (very integrated) preprocessor for some macros in a non-homoiconic statically typed language. For (3) look at how Haskell defines new operators. I find Haskell's use of completely new operator-squiggles plus type classes for something faintly similar to C++-style overloading, much more sensible than say, C++-style overloading.
- swannodette 16y ago1. Yet, still to this day, Lisp remains one of the few languages with powerful macros that normal people cannot only use, but enjoy. Also the Scheme/Racket community actively researches how to make them more robust and easier to write (yes equally powerful macro systems do exist for other languages - however, ease of use is often sacrificed) 2. Io is great but the depth of it's runtime meta-programming facilities make it horrifically inefficient.
- deleted 16y ago[deleted]
- igravious 16y agoAre natural languages homoiconic? (If you get my drift.) Does that question even make sense?
- jerf 16y agoHmm, I don't know about in general, but in English we have the double-quote when being used to refer to the statement itself. For example, I can talk about the use of "for example" by quoting it like that. So (written) English can embed English in a reasonably standard format. Very like the Lisp quote operator, no? The name is not coincidence. However, we have no useful English operators to take the quoted words and change them into something else meaningfully. Though a non-English set can be argued to exist: http://creativeservices.iu.edu/resources/guide/marks.shtml http://creativeservices.iu.edu/resources/guide/marks.shtml If those were English themselves that would arguably complete the loop, but I would consider that stretching the point; these were explicitly invented to avoid the use of English and also translate to any other character-based language.
- pangram 16y agoI can't think of a natural language equivalent for the comma operator, other than (maybe) incredibly awkward and confusing uses of parenthetical statements
- jfm3 16y agoEven early stabs at x-bar theory indicate that natural languages are highly homoiconic. http://en.wikipedia.org/wiki/X-bar_theory http://en.wikipedia.org/wiki/X-bar_theory
- sedachv 16y agoCan you please explain more?
- jfm3 16y agoHomoiconicity means program and data have identical representation. But after you Lisp for a while the boundary blurs entirely. X-bar theory says that in all natural languages, Noun Phrases, Verb Phrases, and depending on who you read other types of phrases all have the same form; a single Head (which may be composite) and a Tail with zero or more elements. Never mind that they've just discovered S-expressions. The point is that in the natural language program: multiply the radius squared and pi Both the data-like parts and the program-like parts share the same structure.
- joedavis512 16y agoGood post, but I still think that C's macro are pretty powerful. I've created a foreach loop in C (GNU C at least, though I could probably port it using C1x's _Generic) just to prove this: https://gist.github.com/632544 https://gist.github.com/632544.
- nelhage 16y ago> Except, of course, that doesn’t work. The preprocessor only makes one pass > through the file, meaning a macro can’t call another macro. Huh? That's just blatantly wrong, and I (and many others) have used the C preprocessor to create multi-level macros in quite powerful ways. See for example http://blog.nelhage.com/2010/07/implementing-an-edsl-in-cpp/ http://blog.nelhage.com/2010/07/implementing-an-edsl-in-cpp/ or the insanity Linux's tracing macros have implemented: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=include/trace/define_trace.h;h=1dfab54015113b83bce9f3302470c3a5ed95b5e7;hb=HEAD http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6... His article has a point, but he's not doing C macros justice -- they are quite powerful and can be used in really cool ways, even if people tend to look at you funny when you do :)
- adobriyan 16y agoThe reason Linux trace macros are insane is because C preprocessor is so primitive. Think about it. > they are quite powerful For small values of "quite". C macros are strictly less (and by large margin powerful than Lisp macros. One can't even use #ifdef inside #define!
- kenjackson 16y agoNote, the power of the macro system is not equal to its general good. The C macro system was already too powerful, and one of the things that hurts languages is a preprocessor of virtually any kind. The badness of the C preprocessor starts much earlier with things like include files, and the fact that you can't statically analyze the code in any reasonable way. The benefit from the preprocessor is absurdly small. With that said, I think the benefit of the Lisp preprocessor is pretty small too. C would be harmed more by its omission due its compilation model, but as we've seen with languanges in its family, like C#, its unnecessary. In a decade or less, the Lispers will see this light too.
- ohyes 16y ago"I think the benefit of the Lisp preprocessor is pretty small too" Lisp doesn't have a preprocessor, Lisp is a preprocessor. You can program with it, but thinking about it as a programming language (ASCII or Unicode Strings to be Interpreted) is at least partially wrong. "In a decade or less, the Lispers will see this light too." It hasn't happened in the last half century, so how much do you want to bet on it?
- JeanPierre 16y agoSeeing how this guy comments about loop, be a little careful when using it. There is no formal standard on how to implement loop, so what you write might result in different answers depending on what Common Lisp implementation you use. Evaluate these loop-expressions and see what happens (from ANSI Common Lisp by PG): (loop for y = 0 then z for x from 1 to 5 sum 1 into z finally (return (values y z))) and then evaluate this: (loop for x from 1 to 5 for y = 0 then z sum 1 into z finally (return (values y z)))
- ekiru 16y agoWhat aspects of the actual behavior of loop are unspecified?
- JeanPierre 16y agoHow to combine them is poorly defined and complex. The loop clauses by themselves is not the problem.
- tygorius 16y agoSo Lisp macros provide compile-time code generation, but in doing so, you have access not just to the code-generation instructions – which are just Lisp data – but also to the entire Lisp language and environment. The same can be said for classic Forth. Indeed, the enthusiastic way that Lispers describe macros I find eerily similar to how I used to hear Forthians describe their language. Except in Forth the relevant term isn't macro but "defining word". Indeed, I think programmers in both camps see the proper use of macros/defining words as a sign that one has passed the beginner stage of using the language. A classic compiled Forth word (ie, function) would consist of the word's name followed a string of addresses to execute; how those addresses were laid down, where they pointed, etc. were totally open to programmer control -- both during compilation and afterwards. It's been a couple of decades since I looked at Forth code, but I don't think there's anything that could be done in Lisp macros that couldn't also be done easily in Forth as well. I think the big difference between the two -- at least, so far as the macro topic goes -- is that Lisp starts with an assumption that code and data can and should be interchangeable while Forth was initially developed for the machine control (embedded systems before they were called embedded systems) and thus code was merely a vehicle for creating applications.
- eru 16y agoForth is almost the complete opposite of Lisp, and thus very similar.
- tygorius 16y agoYou're gotten a number of up-votes, yet this statement makes as much sense to me as saying bananas are the complete opposite of apples. You can make a case of similarity or dissimilarity depending on which language aspect you focus on. But, "opposite"? I have no idea what that means.
- Someone 16y agoLet me guess at some of what (s)he means: Forth is an interpreter that can be switched into compilation mode, Lisp is a compiler that can be switched into interpreter mode. [edit: thinking of that, this more reflects how I have used the two than what they are. Lisps start in an eval loop, too, yet they feel different to me] Forth is postfix, Lisp is prefix. Forth programmers know their stacks are the environment, but rarely think of it as an environment. Lisp programmers know their environment is in some stack-like data structures, but rarely think of those stacks.
- dauphin 16y agoNothing beats Lisp, except perhaps Paul Graham.
- gridspy 16y agoThanks for presenting Lisp macros in terms that I (as a C++, Python et al programmer) can understand. I can see why they are as powerful as claimed.
- evanrmurphy 16y agoIn dialects that support fexprs such as Kernel, PicoLisp and Eight, you get something almost like first-class macros. Macros (or fexprs, rather) are then even more powerful, because they can be passed around as arguments to functions or even to other fexprs. A key tradeoff is that they must be expanded at run-time, and in doing so they incur a performance penalty that doesn't exist with compile-time macros.
- njharman 16y agoI like run-time meta programing. Which i assume lisps are capable of. speed excites me not at all, and i consider having a compile-time to be a disadvantage. i wish authors didnt craft their arguments around C/Java.
- nkurz 16y ago> Except, of course, that doesn’t work. The preprocessor > only makes one pass through the file, meaning a macro > can’t call another macro. macro.c: #include <stdio.h> #define LOOP(n) for (int i = 0; i < n; i++) #define PRINT10 LOOP(10) { printf("%d", i); } int main(void) { PRINT10; printf("\n"); } gcc -Wall -std=gnu99 macro.c && a.out 0123456789 Of course macros can call other macros: http://gcc.gnu.org/onlinedocs/cppinternals/Macro-Expansion.html http://gcc.gnu.org/onlinedocs/cppinternals/Macro-Expansion.h... Ignoring his poorly protected macros, and taking nothing away from Lisp, am I missing some subtle point he's making, or is the author just flat out wrong here? I think he's just wrong, and this makes it hard for me to even read the rest of his argument.
- leppie 16y agoI noted this too. Seems the author has a broken C compiler.
- dexen 16y agoA C macro can't directly -- nor indirectly -- refer to itself; even through other macro. You can't do loops, you can't do recursion. (a nonsensical example, as I can't think of a proper one right now) #define PRINTFOO PRINTBAR #define PRINTBAR PRINTFOO PRINTFOO The ANSI C standard says a macro, while it is being expanded by the preprocessor, ``is painted blue'' (is no subject to further expansion). I'm not sure, but probably it follows the C macros are not Turing-complete. EDIT: on the other hand, a LISP macro system is a Turing-complete programming language that's geared towards manipulating lists of symbols and values (which, incidentally, is LISP program).
- nkurz 16y agoDo you have a link for the "painted blue" reference? I agree with your explanation, but had never heard that particular phrasing and was intrigued by it. But searching on Google for things like 'expand ansi macro "painted blue"' returns only your comment and a scraped spam site of someone answering a similar question.
- 16y ago
- cygx 16y agoThe factual errors concerning the C preprocessor aside, the author also ignores C++-style metaprogramming. Granted, C++ templates have their flaws, but they do provide a Turing-complete compile-time metaprogramming environment...
- jongraehl 16y agoMetaprogramming is a more general concept than the author claims. For example, lisp macros and eval/quote (at any level of compilation or runtime) are particular metaprogramming facilities. You can even metaprogram using only explicit code quotation if you prefer (with no implicit reflection exposure of regular code). I agree with the definition at http://en.wikipedia.org/wiki/Metaprogramming http://en.wikipedia.org/wiki/Metaprogramming