35 ms·

The article doesn't discuss macros, which is one of the answers to "Why Lisp?" I didn't "get" macros until I read a footnote in the (freely available) book Pra

The article doesn't discuss macros, which is one of the answers to "Why Lisp?" I didn't "get" macros until I read a footnote in the (freely available) book Practical Common Lisp. In chapter 7, it introduces the `dolist` macro. DOLIST loops across the items of a list, executing the loop body with a variable holding the successive items of the list. This is the basic skeleton (leaving out some of the more esoteric options): (dolist (var list-form) body-form*) ...For instance: CL-USER> (dolist (x '(1 2 3)) (print x)) 1 2 3 NIL Buried in a footnote is this: "DOLIST is similar to Perl's `foreach` or Python's `for`. Java added a similar kind of loop construct with the 'enhanced' for loop in Java 1.5, as part of JSR-201. Notice what a difference macros make. A Lisp programmer who notices a common pattern in their code can write a macro to give themselves a source-level abstraction of that pattern. A Java programmer who notices the same pattern has to convince Sun that this particular abstraction is worth adding to the language. Then Sun has to publish a JSR and convene an industry-wide "expert group" to hash everything out. That process--according to Sun--takes an average of 18 months. After that, the compiler writers all have to go upgrade their compilers to support the new feature. And even once the Java programmer's favorite compiler supports the new version of Java, they probably still can't use the new feature until they're allowed to break source compatibility with older versions of Java. So an annoyance that Common Lisp programmers can resolve for themselves within five minutes plagues Java programmers for years." http://www.gigamonkeys.com/book/macros-standard-control-constructs.html http://www.gigamonkeys.com/book/macros-standard-control-cons...

lisper 12y ago
And if you want Python-style iterators, that's easy too: http://blog.rongarret.info/2008/02/joy-of-iterators.html http://blog.rongarret.info/2008/02/joy-of-iterators.html
- ScottBurson 12y ago
  So, that thing I mentioned yesterday that I wrote 35 years ago? It might interest you: https://github.com/slburson/misc-extensions Look in src/gmap.lisp. I admit it wouldn't work that well for 'n-at-a-time', but here's a closely related example: (defun alist-to-plist (alist) (gmap :plist nil (:alist alist))) There are some idiosyncrasies here: 'nil' is shorthand for the identity function (in this case, the two-argument, two-value identity function), and the use of keyword symbols is weird and potentially problematic, though I've never run into a problem with it in practice. This is the map-reduce model of iteration -- obviously not with support for distributed computation, but nonetheless useful for programming in the small. And far more elegant, to my eyes, than that LOOP monstrosity :-)
IshKebab 12y ago
On the other hand, this really hurts readability. When reading other people's code you now effectively have to learn what "language" they use too. I'd say it's probably worth that cost, if used judiciously.
- kazinator 12y ago
  "If used judiciously" is just FUD. Soviet-era authority figure: "Western-style freedom has its good points---if applied judiciously". If you're a proper Lisper, you use macros like it's going out of style, and other proper Lispers love your code for it. All programs have their own dictionary of whatever it is they define, whether it be macros or variables. You can no more understand a function call just by looking at it than a macro call. (Should functional decomposition be introduced "judiciously" into large, monolithic blocks of code that have everything "at a glance" in one place?)
  - Dewie3 12y ago
    I guess the "problem" with understanding this perspective for us non-lisp people is that we may not be able to imagine all the places that we could have used macros (or something equivalent) if we haven't even learnt it. You don't know what you don't know, or in this case, you don't know how to use something you haven't used. But this seems to be the same for boatload of programming features that many people aren't taught with their first language, but rather later - higher order functions, lazy evaluation/generators/etc., type-level functions... we might at first think "this is a special tool only to be used in certain circumstances/only to be used by wizards". And then it might turn out that they are useful all the damn time. The net result, at least for me, is that I get spoiled and then it is hard for me to go back 'more primitive ways'. :)
    - kazinator 12y ago
      Macros are easy to understand if you are shown that whatever language you are using already has them. The difference is that they are locked in and wrapped behind at least two layers. Firstly, there rigid surface syntax which customizes the look of every macro at the character level. For instance, in C, the do ... while(); loop must have a trailing semicolon. (No Lisp macro has such requirements.) Secondly, that surface syntax translates to a limited set of abstract syntax tree forms which is not extensible. Lisp macros open that up. The outer layer of varnish is stripped away, so any conceivable abstract syntax tree form has a notation; you do not have to invent new character-level surface syntax in order to work with a new form. And then, custom recognizers for arbitrary forms can be written by the language users, which do tree to tree transformations. So then how you use these things once you have them is the same way that yu use the features of programming languages that you know. E.g. loop is a macro in Lisp. We use it like this: [1]> (loop for x from 1 to 10 for a = 2 then (* 2 a) collect (list x a))) ((1 2) (2 4) (3 8) (4 16) (5 32) (6 64) (7 128) (8 256) (9 512) (10 1024)) Someone wrote the loop macro, so in this situation I'm just a user. I don't care whether this is a macro, or built-in to the language like "foreach x in list do". When I use loop, I'm benefiting from macro-writing. The benefits are far-reaching. For instance, language experimentation takes place in the user base, not behind the closed doors of an ivory tower (ISO committee or whatever). Language ideas are packaged as code and shared around. Technical problems turn into social problems, as someone noted.
PuercoPop 12y ago
Just to expand on your comment. One can write the equivalent code for a for in python[0] but one can't make python rewrite itself. [0]: https://gist.github.com/PuercoPop/9d192f94f88074d06625 https://gist.github.com/PuercoPop/9d192f94f88074d06625
- Foxboron 12y ago
  cough I'd love to say it's possible, but with a few modifications. https://github.com/hylang/hy https://github.com/hylang/hy http://docs.hylang.org/en/latest/language/api.html?highlight=defmacro#defmacro http://docs.hylang.org/en/latest/language/api.html?highlight...
  - PuercoPop 12y ago
    It is possible with macropy as well. But both hylang and macropy depend on the load module hook to run the transformation iirc. So the transformation don't work the the __main__ module.
ra88it 12y ago
> The act of describing what you want the machine to do is interleaved with the machine actually doing what you have described, observing the results, and then changing the description of what you want the machine to do based on those observations. There is no bright line where a program is finished and becomes an artifact unto itself. Author doesn't mention macros explicitly, but I think the above quote is referring to them implicitly.
don_draper 12y ago
It would help if I saw a real world problem and how you can solve it Lisp and not in say Python.
- kazinator 12y ago
  Any real world problem solved in Lisp, relative to being solved in Python, eliminates the superfluous problem of Python being involved: a syntactically Fortran-like cumbersome scripting language with somewhat Lispy semantics. Note that you can program the CL system by expressing yourself in Python: https://common-lisp.net/project/clpython/ https://common-lisp.net/project/clpython/ Common Lisp is not only a language, but also a platform (analogous to Mono or JVMs). It has a model of computation: programs expand fully down to a set of special forms, which then compile. You can build languages on top of this.
  - GFK_of_xmaspast 12y ago
    So your answer is 'I have no such example' ?
    - nefreat 12y ago
      I think it would be pretty hard to write something like algo.monads in python when macros are not available. https://github.com/clojure/algo.monads https://github.com/clojure/algo.monads
- typedweb 12y ago
  How about syntactic extensions: (defmacro unless (test &body body) `(if (not ,test) ,@body))
  - cosarara97 12y ago
    That's a theoretical problem, not a real world problem.
    - 12y ago
swuecho 12y ago
there is two sides. probably, one evil thing is you have to deal with every other's abstraction and it will make you frustrated when there is a large codebase and a deadline.
- kazinator 12y ago
  Let's see, what would I rather deal with. * Another programmer's 15 function API, and a document on how to use it properly? * Or another programmer's 15 function API, along with three macros which use the API properly, and capture all the scenarios I need based on a couple of examples.
  - GFK_of_xmaspast 12y ago
    How's this any different than taking a big ugly api and writing three shims around it to call it in the way you want?
    - kazinator 12y ago
      The difference isn't whether or not code is written; the question is who/what writes it: - Does the machine write it, using a reliable syntax rule loaded into the compiler (implemented as a macro); or - Does the programmer, by hand (or with textual preprocessing, copy and paste, IDE help, ...). Macros can be as smart as you want them to be.
  - swuecho 12y ago
    generally, even bad api is easier to understand or debug than macro. and not all macros are well written.
    - kazinator 12y ago
      Macros being hard to debug may be true of some macro systems that you know. Not all macros are well written is a poor argument; not all anything is well written ... so don't use anything!
Xophmeister 12y ago
Could someone please explain the difference between Lisp macros and, say, languages that have first-class functions? I get that a Lisp macro will be expanded into the respective code, while a function's execution is different. However, at the practical (i.e., developer's) level, are there any additional benefits? Can, say, a Lisp macro be 'partially formed', in the sense that it can expand into some boilerplate that represents an incomplete syntax tree? (Whereas a higher-order-function is necessarily complete.) I can see that being useful, but not inasmuch a it's made out.
- coryrc 12y ago
  Macros are run before your program is running with actual data (during "compile", if you will). The advantage of CL macros is you can use functions written for your program to use at run-time during compilation-time/macro evaluation too.
- dfan 12y ago
  One qualitative difference in expressive power comes from the fact that functions first evaluate all their arguments and macros don't.
  - kazinator 12y ago
    Ah, but that is not the case in some languages in which arguments are evaluated lazily. Usually the iron-man version of this question is: "if I have a non-strictly evaluated language with higher order functions, do I still need macros?" A part of the answer is: you probably don't need the kinds of macros which cover up machine-generated lambdas, which simulate non-strict evaluation in strictly evaluated Lisp programs! The compiler for your language already has these "macros" in its compiler. The argument why "you always need macros no matter what else you have" is that your language has "hard-coded macros": the grammar rules in a compiler, which match patterns and transform bits and pieces to produce AST fragments. If you don't have macros, then that set of "hard-coded macros" is all you have. So, even in a functional language with nonstrict evaluation, you're using macros. It's hard to make a convincing argument that they provide all the expressivity you would conceivably ever need. (And their existence and use defeats any argument that you don't need macros at all).
dorfsmay 12y ago
Aren't macros breaking the homocionicity of the lisps? And making maintenance more difficult (aren't you re-inventing a new language with macros?)? Where does the "First rule of the macro club" coming from? When should you break it?
- deleted 12y ago
  [deleted]
- dragonwriter 12y ago
  > Aren't macros breaking the homocionicity of the lisps? No. How would that be the case? > And making maintenance more difficult If used poorly, sure; the same is true of most features of most languages. > (aren't you re-inventing a new language with macros?) You are extending the existing language with them, much as you are when using a language with user-definable functions rather than only builtins.
- sjolsen 12y ago
  >Aren't macros breaking the homocionicity of the lisps? No. How would macros break homoiconicity? They expand into atoms and lists (and other datatypes), the same stuff of which macro-free programs are made. >And making maintenance more difficult? No. Unless you intentionally write unmaintainable macros. It's the same as if you write unmaintainable functions, classes, etc. They're just abstracting a different thing— syntax. >aren't you re-inventing a new language with macros? No, you're expanding the language; you're just expanding the grammar instead of the vocabulary. >Where does the "First rule of the macro club" coming from? When should you break it? What is the "first rule of the macro club?"
  - roryokane 12y ago
    I have heard the “first rule of the macro club” to be “don’t write macros”. The idea is before writing a macro, you should try writing it as a function instead. If that is possible, that is usually better, because functions, unlike macros, can be passed around as first-class values, and I think they are easier to debug. You should break that rule only when the behavior can’t be written as a function, such as these cases: • The call needs to avoid evaluating its arguments. For example, the `if` built into the language is sometimes defined as a macro. (if (= (+ 2 2) 4) (print "math works") (print "math is broken")) If `if` were a function, it would first print both statements, and then return the return value of `print` in whichever branch. By making `if` a macro, it can avoid evaluating the branch that is inapplicable. • The call relies on information about the environment only available at compile-time. Perhaps the macro reads a configuration file on the developer’s computer to decide how to set something up. • You have profiled the program and determined that it is better to run the function at compile-time. For example, you might want to make `(regex "[a-z][a-z0-9]+")` compile the string to a regular expression at compile-time instead of run-time.
GFK_of_xmaspast 12y ago
It's kind of cool how in lisp you're supposed to do macros to change major aspects of how the language behaves, but in C if you try having a little bit of fun with #define and the pre-processor everybody starts getting just extremely rude at you.
- swuecho 12y ago
  That is part of the reason. everybody think lisp cool, but very few programmer use lisp.
- the_af 12y ago
  To be fair, C #define macros are extremely dangerous and error prone in ways that Lisp macros aren't. I haven't bought into this "Lisp macros are the best thing since sliced bread" idea, but without a doubt they are infinitely more awesome than C macros.
- Dewie3 12y ago
  Could the reason perhaps be that Lisp macro facilities tend to be much more sane than C's pre-processor? Just a wiiild guess.
stephengillie 12y ago
I'm sorry for asking a dumb question; I'm a sysadmin not a software developer: Is this similar to needing a dozen nearly-identical lines of code, writing one, then using Excel to manipulate the other 11 lines into what you want, then copying it back (and cleaning tabs in the process)? Essentially programmatically writing the program - is that what Lisp's macros are?
- bliti 12y ago
  More or less. It allows you to write generic code you can reuse. In this case, you are extending the language. Feel free to post these "dumb" questions, or send them my way.
- willismichael 12y ago
  Not a dumb question at all, and yes, that's more-or-less what Lisp macros are, albeit more elegant because you're using the same language to write your code as you use to write the macros that manipulate the code.
- WalterGR 12y ago
  Essentially programmatically writing the program - is that what Lisp's macros are? Macros are often described that way. "Lisp macros let you write code that writes code!" While that's a true statement, it's kind of useless for coming to an initial understanding of what macros let you really do. Think of it this way. Programming languages have built-in control flow operators like if-then, do, while, and for. Macros let you write your own operators - that work just like the ones built into the language itself. Here's an example. Let's say you want to print a list of names. In a language without an operator specifically for going over lists - and without macros - you may have to write: ListIterator iterator = namesList.GetListIterator(); while(iterator.HasMoreItems()) { string name = iterator.GetNextItem(); print(name); } If you have a language that supports macros, you can create a macro called forEachItemInList. And then it'll work just like it came with the language all along: forEachItemInList(string currentName in namesList) { print(currentName); } If you count how many words the non-macro solution took, it's 13. The macro solution takes 7. Having to solve a problem with more code means more time to understand, teach, write, test, debug, and document - so savings like this can really add up. And there's also the subjective part. Having to write code like in the first example just isn't satisfying. You have to repeat yourself constantly. You may think, "Look, I loop over lists all the time. And it's always the same pattern: I call GetListIterator(), I do a 'while' loop while it HasMoreItems(), then I call GetNextItem(), etc. Why can't I just inform the compiler of the general pattern - and then tell it how to fill in the blanks when I need to actually loop over a list?" And that's what macros let you do.
hzhou321 12y ago
I agree. Macros are what made lisp, not parentheses. However, if we broaden the concept a little, we can do this type of macros for any language with a meta-layer such as MyDef. e.g. if you observe certain foreach pattern in Java, you can make a macro for that pattern and have the meta-layer look out and translate that for you (like having an automatic translator between you and javac. All you need is a scope type macro. Conventional macro packages like M4 does not provide such, but MyDef does. Example: &call each_member, AList # java code that work on $(member) where the macro may be defined as subcode: each_member(list) Enumeration e = $(list).elements(); while (e.hasMoreElements()){ String name = (String) e.nextElement(); $(set:member=name) BLOCK } Where it is understood that the definition can be any literal block pattern.
- WalterGR 12y ago
  Macros are what made lisp, not parentheses. The parentheses are huge, though. It enables "Code is data" and "Data is code" in a powerful way. About Lisp, people always say "Macros are great" and "Code is data" and "Data is code", but it's hard to see what they mean without good examples. I mean, you can write code that writes code in any language that has a `print` statement. And obviously code is data, so what is that aside from some pseudo-philosophic BS? There's a lot of discussion in this thread about macros - so here's an example of Code and Data Being One, for those that are unconvinced. I hope it'll shed some light. I have a slang dictionary website. It's backed by a database now, but it used to be statically-generated HTML. I represented the data as XML. It looked something like this: <Term term="slick"> <PartOfSpeech pos="adj"> <Definition> impressive. </Definition> <Definition> smart. </Definition> </PartOfSpeech> </Term> So then I needed an XML parser to parse the data. Maybe it parsed the XML into a object model that the code would navigate and output the appropriate HTML. Or maybe the code got callbacks according to node type and would output the appropriate HTML then. XML is a pretty verbose format, so - this being a Lisp example - we could probably save some typing if we represented the data as an S-expression. The above XML would become something like this: (Term "slick" (PartOfSpeech "adj" (Definition "impressive." ) (Definition "smart." ) ) ) But that's an idiosyncratic style. So, let's lowercase the node types, use hyphens rather than camelCase, and remove the unnecessary line breaks. Now the data looks like this: (term "slick" (part-of-speech "adj" (definition "impressive.") (definition "smart."))) Great. We've got our data. Now we need to write the Lisp code to convert that S-expression data into the appropriate HTML output. We'll need some Lisp functions to handle the nodes and their attributes (such as the definition text and part of speech) and write the HTML for them. We could use an S-expression parser library to load the data, and then walk through it and call those functions. But that's not necessarily the best way. We can simplify it by creating exactly 3 functions that take some arguments and output HTML: term, part-of-speech, and definition. Since Lisp code is - like the data - also represented as S-expressions, once we've written those 3 functions, the data is literally executable Lisp code.
larme 12y ago
Macros are great. I'm writing a hy-like lisp language that compiles to lua. A lot of features (Simple OO, list/dict comprehensive etc.) are written in macros. https://github.com/larme/hua/tree/master/hua/core https://github.com/larme/hua/tree/master/hua/core On the other hand, they may be difficult to debug.
agumonkey 12y ago
Note that this is a double edged sword. Everyone is having is own little DSL world. A thing that Lispers often enjoy, but it's an art, a beautiful one, and a beautiful freedom too, and most people want a python with easy to access libraries more than that. Lisp aims itself into niche problems where there's nothing good enough done yet. For the rest people won't care about the ability to fit the language to their need, they want 'productivity' and if a pattern become useful, backpressure will force language designer / BDFL to change it.
hueving 12y ago
>So an annoyance that Common Lisp programmers can resolve for themselves within five minutes plagues Java programmers for years I get the sentiment, but this is seriously advocating for unmaintainable code that likely breaks in strange corner cases and requires 10 times as long for a new person to the code base to understand. First contributor, "I know my code should be able to handle out of memory errors in this corner case, but I know I won't use it that way so it shouldn't be an issue." Second contributor, "Hey this macro from First contributor looks nice, I'm going to use it." A pile of shit emerges.
- badpenny 12y ago
  A pile of shit emerging whenever programmers are lazy and try to cut corners or when they try to do things they don't fully understand is not unique to macro writing.
  - lazzlazzlazz 12y ago
    I think the point above is that this problem could be exacerbated by macro writing.
  - hueving 12y ago
    Right, the point i was making is that not everyone is a 'rockstar' and macros just give you quite a bit more rope to hang yourself with. Awful java/python/whatever is obvious because you can't redefine control flow so easily. With macros you can have something as fundamental as loops be broken.
current_call 12y ago
The book you linked has examples of macros being used for writing a music database, parsing binary files, a unit testing framework, and several other things I haven't reached yet. Those are much better examples of macros than basic control structures. The 'dolist' form is a toy example.
renox 12y ago
Similar, similar, only if you don't look too closely: what about break and continue? They are usually supported by native foreach construct but are difficult to build with macros..
- WalterGR 12y ago
  only if you don't look too closely Look as closely as you'd like. :) what about break and continue? They are usually supported by native foreach construct but are difficult to build with macros.. For break, DOLIST uses return. I haven't worked with Lisp in some time, so I don't remember if it has a dedicated continue. But a continue is just a break in a loop that doesn't loop. Or a GOTO by another name. Or a jump to a particular case in a switch. But if your looping macro uses a native looping construct or another macro that supports break and continue, then your looping macro will inherit that support, provided that you're careful when writing the macro to do nothing that will break it. If your looping macro doesn't use a construct that supports break and continue, then you're still in luck. Continue I described above. Like continue, break is also GOTO by another name. You've got TAGBODY, GO, CALL/CC, etc. Why do you think that supporting break and continue in macros is difficult?
  - renox 12y ago
    > Why do you think that supporting break and continue in macros is difficult? In many languages, emulation of control loop do not support break and continue so I thought that this was the case also here, but apparently I'm wrong, sorry for my 'too quick' post and thanks for the correction.

lisper 12y ago

And if you want Python-style iterators, that's easy too: http://blog.rongarret.info/2008/02/joy-of-iterators.html http://blog.rongarret.info/2008/02/joy-of-iterators.html

ScottBurson 12y ago
So, that thing I mentioned yesterday that I wrote 35 years ago? It might interest you: https://github.com/slburson/misc-extensions Look in src/gmap.lisp. I admit it wouldn't work that well for 'n-at-a-time', but here's a closely related example: (defun alist-to-plist (alist) (gmap :plist nil (:alist alist))) There are some idiosyncrasies here: 'nil' is shorthand for the identity function (in this case, the two-argument, two-value identity function), and the use of keyword symbols is weird and potentially problematic, though I've never run into a problem with it in practice. This is the map-reduce model of iteration -- obviously not with support for distributed computation, but nonetheless useful for programming in the small. And far more elegant, to my eyes, than that LOOP monstrosity :-)

IshKebab 12y ago

On the other hand, this really hurts readability. When reading other people's code you now effectively have to learn what "language" they use too. I'd say it's probably worth that cost, if used judiciously.

kazinator 12y ago
"If used judiciously" is just FUD. Soviet-era authority figure: "Western-style freedom has its good points---if applied judiciously". If you're a proper Lisper, you use macros like it's going out of style, and other proper Lispers love your code for it. All programs have their own dictionary of whatever it is they define, whether it be macros or variables. You can no more understand a function call just by looking at it than a macro call. (Should functional decomposition be introduced "judiciously" into large, monolithic blocks of code that have everything "at a glance" in one place?)
- Dewie3 12y ago
  I guess the "problem" with understanding this perspective for us non-lisp people is that we may not be able to imagine all the places that we could have used macros (or something equivalent) if we haven't even learnt it. You don't know what you don't know, or in this case, you don't know how to use something you haven't used. But this seems to be the same for boatload of programming features that many people aren't taught with their first language, but rather later - higher order functions, lazy evaluation/generators/etc., type-level functions... we might at first think "this is a special tool only to be used in certain circumstances/only to be used by wizards". And then it might turn out that they are useful all the damn time. The net result, at least for me, is that I get spoiled and then it is hard for me to go back 'more primitive ways'. :)
  - kazinator 12y ago
    Macros are easy to understand if you are shown that whatever language you are using already has them. The difference is that they are locked in and wrapped behind at least two layers. Firstly, there rigid surface syntax which customizes the look of every macro at the character level. For instance, in C, the do ... while(); loop must have a trailing semicolon. (No Lisp macro has such requirements.) Secondly, that surface syntax translates to a limited set of abstract syntax tree forms which is not extensible. Lisp macros open that up. The outer layer of varnish is stripped away, so any conceivable abstract syntax tree form has a notation; you do not have to invent new character-level surface syntax in order to work with a new form. And then, custom recognizers for arbitrary forms can be written by the language users, which do tree to tree transformations. So then how you use these things once you have them is the same way that yu use the features of programming languages that you know. E.g. loop is a macro in Lisp. We use it like this: [1]> (loop for x from 1 to 10 for a = 2 then (* 2 a) collect (list x a))) ((1 2) (2 4) (3 8) (4 16) (5 32) (6 64) (7 128) (8 256) (9 512) (10 1024)) Someone wrote the loop macro, so in this situation I'm just a user. I don't care whether this is a macro, or built-in to the language like "foreach x in list do". When I use loop, I'm benefiting from macro-writing. The benefits are far-reaching. For instance, language experimentation takes place in the user base, not behind the closed doors of an ivory tower (ISO committee or whatever). Language ideas are packaged as code and shared around. Technical problems turn into social problems, as someone noted.

PuercoPop 12y ago

Just to expand on your comment. One can write the equivalent code for a for in python[0] but one can't make python rewrite itself. [0]: https://gist.github.com/PuercoPop/9d192f94f88074d06625 https://gist.github.com/PuercoPop/9d192f94f88074d06625

Foxboron 12y ago
cough I'd love to say it's possible, but with a few modifications. https://github.com/hylang/hy https://github.com/hylang/hy http://docs.hylang.org/en/latest/language/api.html?highlight=defmacro#defmacro http://docs.hylang.org/en/latest/language/api.html?highlight...
- PuercoPop 12y ago
  It is possible with macropy as well. But both hylang and macropy depend on the load module hook to run the transformation iirc. So the transformation don't work the the __main__ module.

ra88it 12y ago

> The act of describing what you want the machine to do is interleaved with the machine actually doing what you have described, observing the results, and then changing the description of what you want the machine to do based on those observations. There is no bright line where a program is finished and becomes an artifact unto itself. Author doesn't mention macros explicitly, but I think the above quote is referring to them implicitly.

don_draper 12y ago

It would help if I saw a real world problem and how you can solve it Lisp and not in say Python.

kazinator 12y ago
Any real world problem solved in Lisp, relative to being solved in Python, eliminates the superfluous problem of Python being involved: a syntactically Fortran-like cumbersome scripting language with somewhat Lispy semantics. Note that you can program the CL system by expressing yourself in Python: https://common-lisp.net/project/clpython/ https://common-lisp.net/project/clpython/ Common Lisp is not only a language, but also a platform (analogous to Mono or JVMs). It has a model of computation: programs expand fully down to a set of special forms, which then compile. You can build languages on top of this.
- GFK_of_xmaspast 12y ago
  So your answer is 'I have no such example' ?
  - nefreat 12y ago
    I think it would be pretty hard to write something like algo.monads in python when macros are not available. https://github.com/clojure/algo.monads https://github.com/clojure/algo.monads
typedweb 12y ago
How about syntactic extensions: (defmacro unless (test &body body) `(if (not ,test) ,@body))
- cosarara97 12y ago
  That's a theoretical problem, not a real world problem.
  - 12y ago

swuecho 12y ago

there is two sides. probably, one evil thing is you have to deal with every other's abstraction and it will make you frustrated when there is a large codebase and a deadline.

kazinator 12y ago
Let's see, what would I rather deal with. * Another programmer's 15 function API, and a document on how to use it properly? * Or another programmer's 15 function API, along with three macros which use the API properly, and capture all the scenarios I need based on a couple of examples.
- GFK_of_xmaspast 12y ago
  How's this any different than taking a big ugly api and writing three shims around it to call it in the way you want?
  - kazinator 12y ago
    The difference isn't whether or not code is written; the question is who/what writes it: - Does the machine write it, using a reliable syntax rule loaded into the compiler (implemented as a macro); or - Does the programmer, by hand (or with textual preprocessing, copy and paste, IDE help, ...). Macros can be as smart as you want them to be.
- swuecho 12y ago
  generally, even bad api is easier to understand or debug than macro. and not all macros are well written.
  - kazinator 12y ago
    Macros being hard to debug may be true of some macro systems that you know. Not all macros are well written is a poor argument; not all anything is well written ... so don't use anything!

Xophmeister 12y ago

Could someone please explain the difference between Lisp macros and, say, languages that have first-class functions? I get that a Lisp macro will be expanded into the respective code, while a function's execution is different. However, at the practical (i.e., developer's) level, are there any additional benefits? Can, say, a Lisp macro be 'partially formed', in the sense that it can expand into some boilerplate that represents an incomplete syntax tree? (Whereas a higher-order-function is necessarily complete.) I can see that being useful, but not inasmuch a it's made out.

coryrc 12y ago
Macros are run before your program is running with actual data (during "compile", if you will). The advantage of CL macros is you can use functions written for your program to use at run-time during compilation-time/macro evaluation too.
dfan 12y ago
One qualitative difference in expressive power comes from the fact that functions first evaluate all their arguments and macros don't.
- kazinator 12y ago
  Ah, but that is not the case in some languages in which arguments are evaluated lazily. Usually the iron-man version of this question is: "if I have a non-strictly evaluated language with higher order functions, do I still need macros?" A part of the answer is: you probably don't need the kinds of macros which cover up machine-generated lambdas, which simulate non-strict evaluation in strictly evaluated Lisp programs! The compiler for your language already has these "macros" in its compiler. The argument why "you always need macros no matter what else you have" is that your language has "hard-coded macros": the grammar rules in a compiler, which match patterns and transform bits and pieces to produce AST fragments. If you don't have macros, then that set of "hard-coded macros" is all you have. So, even in a functional language with nonstrict evaluation, you're using macros. It's hard to make a convincing argument that they provide all the expressivity you would conceivably ever need. (And their existence and use defeats any argument that you don't need macros at all).

dorfsmay 12y ago

Aren't macros breaking the homocionicity of the lisps? And making maintenance more difficult (aren't you re-inventing a new language with macros?)? Where does the "First rule of the macro club" coming from? When should you break it?

deleted 12y ago
[deleted]
dragonwriter 12y ago
> Aren't macros breaking the homocionicity of the lisps? No. How would that be the case? > And making maintenance more difficult If used poorly, sure; the same is true of most features of most languages. > (aren't you re-inventing a new language with macros?) You are extending the existing language with them, much as you are when using a language with user-definable functions rather than only builtins.
sjolsen 12y ago
>Aren't macros breaking the homocionicity of the lisps? No. How would macros break homoiconicity? They expand into atoms and lists (and other datatypes), the same stuff of which macro-free programs are made. >And making maintenance more difficult? No. Unless you intentionally write unmaintainable macros. It's the same as if you write unmaintainable functions, classes, etc. They're just abstracting a different thing— syntax. >aren't you re-inventing a new language with macros? No, you're expanding the language; you're just expanding the grammar instead of the vocabulary. >Where does the "First rule of the macro club" coming from? When should you break it? What is the "first rule of the macro club?"
- roryokane 12y ago
  I have heard the “first rule of the macro club” to be “don’t write macros”. The idea is before writing a macro, you should try writing it as a function instead. If that is possible, that is usually better, because functions, unlike macros, can be passed around as first-class values, and I think they are easier to debug. You should break that rule only when the behavior can’t be written as a function, such as these cases: • The call needs to avoid evaluating its arguments. For example, the `if` built into the language is sometimes defined as a macro. (if (= (+ 2 2) 4) (print "math works") (print "math is broken")) If `if` were a function, it would first print both statements, and then return the return value of `print` in whichever branch. By making `if` a macro, it can avoid evaluating the branch that is inapplicable. • The call relies on information about the environment only available at compile-time. Perhaps the macro reads a configuration file on the developer’s computer to decide how to set something up. • You have profiled the program and determined that it is better to run the function at compile-time. For example, you might want to make `(regex "[a-z][a-z0-9]+")` compile the string to a regular expression at compile-time instead of run-time.

GFK_of_xmaspast 12y ago

It's kind of cool how in lisp you're supposed to do macros to change major aspects of how the language behaves, but in C if you try having a little bit of fun with #define and the pre-processor everybody starts getting just extremely rude at you.

swuecho 12y ago
That is part of the reason. everybody think lisp cool, but very few programmer use lisp.
the_af 12y ago
To be fair, C #define macros are extremely dangerous and error prone in ways that Lisp macros aren't. I haven't bought into this "Lisp macros are the best thing since sliced bread" idea, but without a doubt they are infinitely more awesome than C macros.
Dewie3 12y ago
Could the reason perhaps be that Lisp macro facilities tend to be much more sane than C's pre-processor? Just a wiiild guess.

stephengillie 12y ago

I'm sorry for asking a dumb question; I'm a sysadmin not a software developer: Is this similar to needing a dozen nearly-identical lines of code, writing one, then using Excel to manipulate the other 11 lines into what you want, then copying it back (and cleaning tabs in the process)? Essentially programmatically writing the program - is that what Lisp's macros are?

bliti 12y ago
More or less. It allows you to write generic code you can reuse. In this case, you are extending the language. Feel free to post these "dumb" questions, or send them my way.
willismichael 12y ago
Not a dumb question at all, and yes, that's more-or-less what Lisp macros are, albeit more elegant because you're using the same language to write your code as you use to write the macros that manipulate the code.
WalterGR 12y ago
Essentially programmatically writing the program - is that what Lisp's macros are? Macros are often described that way. "Lisp macros let you write code that writes code!" While that's a true statement, it's kind of useless for coming to an initial understanding of what macros let you really do. Think of it this way. Programming languages have built-in control flow operators like if-then, do, while, and for. Macros let you write your own operators - that work just like the ones built into the language itself. Here's an example. Let's say you want to print a list of names. In a language without an operator specifically for going over lists - and without macros - you may have to write: ListIterator iterator = namesList.GetListIterator(); while(iterator.HasMoreItems()) { string name = iterator.GetNextItem(); print(name); } If you have a language that supports macros, you can create a macro called forEachItemInList. And then it'll work just like it came with the language all along: forEachItemInList(string currentName in namesList) { print(currentName); } If you count how many words the non-macro solution took, it's 13. The macro solution takes 7. Having to solve a problem with more code means more time to understand, teach, write, test, debug, and document - so savings like this can really add up. And there's also the subjective part. Having to write code like in the first example just isn't satisfying. You have to repeat yourself constantly. You may think, "Look, I loop over lists all the time. And it's always the same pattern: I call GetListIterator(), I do a 'while' loop while it HasMoreItems(), then I call GetNextItem(), etc. Why can't I just inform the compiler of the general pattern - and then tell it how to fill in the blanks when I need to actually loop over a list?" And that's what macros let you do.

hzhou321 12y ago

I agree. Macros are what made lisp, not parentheses. However, if we broaden the concept a little, we can do this type of macros for any language with a meta-layer such as MyDef. e.g. if you observe certain foreach pattern in Java, you can make a macro for that pattern and have the meta-layer look out and translate that for you (like having an automatic translator between you and javac. All you need is a scope type macro. Conventional macro packages like M4 does not provide such, but MyDef does. Example: &call each_member, AList # java code that work on $(member) where the macro may be defined as subcode: each_member(list) Enumeration e = $(list).elements(); while (e.hasMoreElements()){ String name = (String) e.nextElement(); $(set:member=name) BLOCK } Where it is understood that the definition can be any literal block pattern.

WalterGR 12y ago
Macros are what made lisp, not parentheses. The parentheses are huge, though. It enables "Code is data" and "Data is code" in a powerful way. About Lisp, people always say "Macros are great" and "Code is data" and "Data is code", but it's hard to see what they mean without good examples. I mean, you can write code that writes code in any language that has a `print` statement. And obviously code is data, so what is that aside from some pseudo-philosophic BS? There's a lot of discussion in this thread about macros - so here's an example of Code and Data Being One, for those that are unconvinced. I hope it'll shed some light. I have a slang dictionary website. It's backed by a database now, but it used to be statically-generated HTML. I represented the data as XML. It looked something like this: <Term term="slick"> <PartOfSpeech pos="adj"> <Definition> impressive. </Definition> <Definition> smart. </Definition> </PartOfSpeech> </Term> So then I needed an XML parser to parse the data. Maybe it parsed the XML into a object model that the code would navigate and output the appropriate HTML. Or maybe the code got callbacks according to node type and would output the appropriate HTML then. XML is a pretty verbose format, so - this being a Lisp example - we could probably save some typing if we represented the data as an S-expression. The above XML would become something like this: (Term "slick" (PartOfSpeech "adj" (Definition "impressive." ) (Definition "smart." ) ) ) But that's an idiosyncratic style. So, let's lowercase the node types, use hyphens rather than camelCase, and remove the unnecessary line breaks. Now the data looks like this: (term "slick" (part-of-speech "adj" (definition "impressive.") (definition "smart."))) Great. We've got our data. Now we need to write the Lisp code to convert that S-expression data into the appropriate HTML output. We'll need some Lisp functions to handle the nodes and their attributes (such as the definition text and part of speech) and write the HTML for them. We could use an S-expression parser library to load the data, and then walk through it and call those functions. But that's not necessarily the best way. We can simplify it by creating exactly 3 functions that take some arguments and output HTML: term, part-of-speech, and definition. Since Lisp code is - like the data - also represented as S-expressions, once we've written those 3 functions, the data is literally executable Lisp code.

larme 12y ago

Macros are great. I'm writing a hy-like lisp language that compiles to lua. A lot of features (Simple OO, list/dict comprehensive etc.) are written in macros. https://github.com/larme/hua/tree/master/hua/core https://github.com/larme/hua/tree/master/hua/core On the other hand, they may be difficult to debug.

agumonkey 12y ago

Note that this is a double edged sword. Everyone is having is own little DSL world. A thing that Lispers often enjoy, but it's an art, a beautiful one, and a beautiful freedom too, and most people want a python with easy to access libraries more than that. Lisp aims itself into niche problems where there's nothing good enough done yet. For the rest people won't care about the ability to fit the language to their need, they want 'productivity' and if a pattern become useful, backpressure will force language designer / BDFL to change it.

hueving 12y ago

>So an annoyance that Common Lisp programmers can resolve for themselves within five minutes plagues Java programmers for years I get the sentiment, but this is seriously advocating for unmaintainable code that likely breaks in strange corner cases and requires 10 times as long for a new person to the code base to understand. First contributor, "I know my code should be able to handle out of memory errors in this corner case, but I know I won't use it that way so it shouldn't be an issue." Second contributor, "Hey this macro from First contributor looks nice, I'm going to use it." A pile of shit emerges.

badpenny 12y ago
A pile of shit emerging whenever programmers are lazy and try to cut corners or when they try to do things they don't fully understand is not unique to macro writing.
- lazzlazzlazz 12y ago
  I think the point above is that this problem could be exacerbated by macro writing.
- hueving 12y ago
  Right, the point i was making is that not everyone is a 'rockstar' and macros just give you quite a bit more rope to hang yourself with. Awful java/python/whatever is obvious because you can't redefine control flow so easily. With macros you can have something as fundamental as loops be broken.

current_call 12y ago

The book you linked has examples of macros being used for writing a music database, parsing binary files, a unit testing framework, and several other things I haven't reached yet. Those are much better examples of macros than basic control structures. The 'dolist' form is a toy example.

renox 12y ago

Similar, similar, only if you don't look too closely: what about break and continue? They are usually supported by native foreach construct but are difficult to build with macros..

WalterGR 12y ago
only if you don't look too closely Look as closely as you'd like. :) what about break and continue? They are usually supported by native foreach construct but are difficult to build with macros.. For break, DOLIST uses return. I haven't worked with Lisp in some time, so I don't remember if it has a dedicated continue. But a continue is just a break in a loop that doesn't loop. Or a GOTO by another name. Or a jump to a particular case in a switch. But if your looping macro uses a native looping construct or another macro that supports break and continue, then your looping macro will inherit that support, provided that you're careful when writing the macro to do nothing that will break it. If your looping macro doesn't use a construct that supports break and continue, then you're still in luck. Continue I described above. Like continue, break is also GOTO by another name. You've got TAGBODY, GO, CALL/CC, etc. Why do you think that supporting break and continue in macros is difficult?
- renox 12y ago
  > Why do you think that supporting break and continue in macros is difficult? In many languages, emulation of control loop do not support break and continue so I thought that this was the case also here, but apparently I'm wrong, sorry for my 'too quick' post and thanks for the correction.