8 ms·

The problem with having powerful macros is always the same: as a project grows and grows you end up inventing your own little dialect of the language which is o

• flavio81 9y ago
  >as a project grows and grows you end up inventing your own little dialect of the language which is opaque to any 3rd party reading your code unless they take the time to unravel your macros. This is bad use of macros, or an ugly macro system. Macros, at least in Lisp, made code even clearer to understand; because they let you create the constructs that make the problem domain map more directly, straightforwardly, easily, to the programming language. So they reduce line count, they reduce need for kludges or workarounds. They allow more straight code. But this is within the Land of Lisp, where writing a macro isn't something "advanced" nor "complex" nor "esoteric". In the Lisp world, writing a macro is 95% similar to writing a run-of-the-mill function.
  • jdmichal 9y ago
    No true Scotsman would ever write macros in such a confusing manner! But really, this is a recognized problem of Lisp, and has been called the Lisp Curse. [0] One is never programming in "just Lisp", but rather in Lisp plus some half-baked DSL haphazardly created by whoever wrote the program in the first place. Also, don't confuse readability with understanding. Yes, DSLs are typically easier to read, but only after you come to understand the primitives of the language. When every program has its own DSL with its own primitives, even programs that do similar things... That becomes quite a burden. [0] http://www.winestockwebdesign.com/Essays/Lisp_Curse.html http://www.winestockwebdesign.com/Essays/Lisp_Curse.html
    • klibertp 9y ago
      You're replying to an, arguably, "no true Scotsman" argument with a straw man of your own: > but rather in Lisp plus some half-baked DSL Why does it need to be "half-baked"? Why do you assume that writing a good DSL is impossible for most Lisp users? Are you sure it's actually the case?
      • AnimalMuppet 9y ago
        If I understand "the Lisp curse" correctly, the claim is that Lisp often winds up with a "half-baked DSL" because making DSLs in Lisp is so easy. You can do it without putting very much thought into it, so it's easy for the original author to just slap something together. Note well: This is my understanding of the claim. I take no position on whether it is true.
      • btilly 9y ago
        Why does it need to be "half-baked"? Why do you assume that writing a good DSL is impossible for most Lisp users? Are you sure it's actually the case? Writing and maintaining a good DSL is like writing and maintaining bug free code. You always start with the best of intentions, but human fallibility and entropy are always pulling you in the wrong direction. This doesn't mean that the attempt is not worthwhile. But it does mean that you should expect eventual failure.
      • jdmichal 9y ago
        As other commenters have replied: It's half-baked the same way Java tends towards half-baked enterprise design lasagna and/or design pattern bingo. It's easy to do, and most won't question it. Also, yes I would propose that a good DSL would be difficult to write for most programmers of any type. Not because of any inherit deficiency in the programmer, but rather because we tend not to spend enough time in a single domain to understand it well enough to write a good DSL.
    • wtetzner 9y ago
      > Also, don't confuse readability with understanding. Yes, DSLs are typically easier to read, but only after you come to understand the primitives of the language. This is also true of functions. Without reading the body, you don't know if it's just going to return the sum of the two integers you passed to it, or if it's going to change some global variable, launch the missiles, and then return a random int. Yes, macros are more powerful, and therefore you need to be more careful with them. But they are still much better than what ends up being used instead. With languages that don't have macros, you end up with complex frameworks that use runtime reflection, or code generators that run as part of the build system (which end up being an ad-hoc, messy macro system). Or, some horrible solution where you embed a DSL by interpreting trees of objects, which effectively represent an AST. In this case, the embedded language doesn't follow the language rules, but it seems like it does, because you're looking at the implementation of the interpreter, instead of at the syntax of the embedded language.
  • aaron-lebo 9y ago
    Isn't loop essentially a macro (probably special op) that a lot of people hate, specifically because it is a sprawling dsl? Because lisp has an expressive macro system, it can lead people to that. The existence of macros requires an at least above average dev community, because it's one thing when a decently-designed but controversial dsl like loop is default to the language, it's another when every project can roll out their own poorly considered and implemented dsls using macros when simpler abstractions would have sufficed. Most lisp docs will tell you to use macros only when necessary, because as great as they are, they have inherent issues that aren't fixed just by having a good macro system. This is bad use of macros, or an ugly macro system. No true scotsman?
    • lispm 9y ago
      The LOOP macro has actually seen a lot of work in designing and implementating it, including a syntax spec. Some people don't hate it because it is a DSL, but because it is a bit different from the regular Lisp macro, in that it requires more parsing and its clauses are not grouped by parentheses/s-expressions. Plus: understanding the relationship of the clauses is not really that simple, since there is some implicit grouping and dependencies. This is what a LOOP macro looks like in code: (loop for i from 0 below n for v across vec when (evenp n) collect (* i v)) This is what a typical Lisp programmer would prefer: (loop ((for i :from 0 :below n) (for v :across vec)) (when (evenp n) (collect (* i v)))) The clauses would be grouped in a list and each clause would be a list. The body would then use the usual Lisp syntax and the WHEN and COLLECT features would look similar to normal Lisp macros. The LOOP macro is historically coming from Interlisp (70s), where it was a part of a certain language design trend called 'conversational programming'. The idea was to have some more natural language like programming constructs, combined with tools like spell checkers and automatic syntax repair (Do What I Mean, DWIM). From there this idea and the FOR macro was influencing the LOOP macro for Maclisp. The LOOP macro grew over time in capabilities and was then transferred to later Lisp dialects, like Common Lisp. There are actually Lisp macros which are even more complicated to implement and even more powerful, but which create less resistance, since they are a bit better integrated in the usual Lisp language syntax. An example is the ITERATE macro: https://common-lisp.net/project/iterate/ https://common-lisp.net/project/iterate/ Thus it is not the complexity or the functionality of the macro itself, but a particular style of macro and its implementation. I personally also prefer something like ITERATE, but LOOP works fine for me, too. The advantage of something like ITERATE or even LOOP is that they are mostly on the developer level, and not fully the implementor level. A developer or group of developers can develop such a complex macro and can integrate it seamlessly into the language, making the language more powerful and allows us to reuse much of the knowledge/infrastrucure about/of the language. Implementing and designing something like ITERATE and LOOP requires above the usual dev capabilities. Generally macros require some of that, since it makes it necessary that the dev can use or program on the syntactic meta-level. It's there where language constructs are reused, implemented and integrated. Lisp docs will tell you to use macros only when necessary? They tell you to WRITE macros only when necessary. Since many Lisp dialects have a lot macros, you have to use them anyway. Most of the top-level definition constructs are already macros. If we use DEFUN to define a function, we already use macros. In my experience actual Common Lisp uses a lot of macros. I also tend to write a fair number of macros. But generally good macro programming style is slightly underdocumented, especially when we think of various qualities: robustness, good syntax, usefulness, readability, avoiding the obvious macro bugs, ... On Lisp from Paul Graham is useful, as a few other sources, ... Read On Lisp here: http://www.paulgraham.com/onlisp.html http://www.paulgraham.com/onlisp.html Macros are very useful and I use them a lot, but at the same time one needs to put a bit more care/discipline into them and some help of the development environment is useful...
  • Shorel 9y ago
    While I do agree completely with you, let me point that some of the worst offenders in the world of macros also originated in Lisp. Give a mad man enough rope and he can find a way to hang himself.
  • simias 9y ago
    >So they reduce line count, they reduce need for kludges or workarounds. They allow more straight code. >But this is within the Land of Lisp, where writing a macro isn't something "advanced" nor "complex" nor "esoteric". In the Lisp world, writing a macro is 95% similar to writing a run-of-the-mill function. I agree with you but I think there are two aspects to code readability: A/ is it clear what the code does (the intent) and B/ is it clear how it does it (the implementation). I think macros can help massively with A (hiding redundant code and clunky constructs) at the cost of obfuscating B. The thing is that if you want to hack into the code at some point you'll need to understand B too. To take a very simplistic example imagine that you're reading some CL code and see something like: (let ((var 42)) (magicbox var (format t "side effect~%")) (format t "~a~%" var)) Now for some reason you need to figure out what this does. Maybe instead of the 2nd "format" call there's a function you're currently working on and you want to know how it's called. So, without running the code or looking at what "magicbox" does, if you assume that it's a function call, then you can expect that this code will print "side effect" when magicbox is called (since the parameter will be evaluated before the call) and then whenever it returns the 2nd format will display "42" since var is not modified between the let and there. You run your code and you see that it only displays "magic" instead. Uh, What happened? Well there you're lucky because there's only a single statement to consider, the magicbox invocation. You ask emacs to find the definition and you see: (defmacro magicbox (var unused) (list 'setq var "magic")) Now you might say that it's a contrieved example and a bad use of macros and you'd be right, but I'm sure you could find real world example of macros with similar side effects that are well justified. Maybe it makes the code a whole lot nicer and maintainable. Still doesn't change the fact that it makes it harder to figure exactly what's going on. If this magicbox call was in the middle of dozens of statements it might take you a while to figure out why some variably seems to magically change its value.
    • wtetzner 9y ago
      The thing that's nice about macros is that they happen at compile time. If you want to know what a macro is doing, just ask your editor or language implementation to expand the macro invocation. Then you will see exactly what code is being generated.
    • 9y ago
  • _asummers 9y ago
    The best macros are the macros you don't even know exist. Take a look around [0] where thing like if, def, pipeline (|>) and common "base" entities are defined simply as a construction of the Kernel special forms (the parts that won't be expanded by macros). Using the language you'd not know that they were macros because they're in the language of the problem, and do not leak their abstractions. A macro is just a tool and like other tools should be used when it's the right one to use, not because you contorted the problem to fit the tool. [0] https://hexdocs.pm/elixir/Kernel.html https://hexdocs.pm/elixir/Kernel.html
  • btilly 9y ago
    Macros, at least in Lisp, made code even clearer to understand; because they let you create the constructs that make the problem domain map more directly, straightforwardly, easily, to the programming language. Macros are a tool for creating abstractions. In the mind of the author, abstractions are always clearer. Others who have to work with those abstractions legitimately may or may not agree. In the case of authors who think that abstractions are an unmitigated good, I seldom agree with their abstractions. And I don't care whether they are implemented as deep object hierarchies, macros, or functions with a lot of black magic. If you are unaware of the cognitive load for others that are inherent in your abstractions, then you are unlikely to find a good tradeoff between conciseness and how much of your mental state the maintainer has to understand to follow along.
• pjmlp 9y ago
  You should watch the Rustconf Zurich talks regarding macros. Between debugging C++ meta-template programming and Rust macros, I am undecided what to choose as best.
  • Shorel 9y ago
    I choose D template mixins.
  • kibwen 9y ago
    The crucial difference between TMPL in C++ and Rust macros is that TMPL is unavoidable because it provides so many features of the language, whereas Rust macros are only good for DSLs. For example, parametric polymorphism in C++ must be done via templates, but Rust just has generics, full-stop, and debugging those is no problem. I'm in agreement with the grandparent comment that praises Rust macros for having gross syntax; whether or not it was intentional, it discourages people from using them except in extreme circumstances, which in my mind is a feature. :)
    • deleted 9y ago
      [deleted]
• hyperpallium 9y ago
  But when you define a function (or a class), you are also inventing your own little dialect/dsl. Operating overloading is another. They can all be used well or confusingly. Where do you draw the line? At least with functions, there's only one syntactic structure that is user-definable, making it easy to distinguish foreground from background.
  • simias 9y ago
    > But when you define a function (or a class), you are also inventing your own little dialect/dsl. I don't understand what you mean by that. When I call a function in Common Lisp I have some garantees, I know it won't change the value of the parameters that I pass by value, I know it won't add or remove variables from the current scope, I know that the parameters to the function will be evaluated before the invocation. For macros anything goes, for all you know it could expand to a RETURN-FROM and exit your function prematuraly, it could implicitly modify a local variable etc... > Operating overloading is another. They can all be used well or confusingly. I completely agree and I consider that operator overloading for anything that's not "semantically" equivalent to the builtin operator behaviour is inherently evil. It makes sense to overload "*" and "+" on a matrix or vector type, it doesn't make a lot of sense to overload "<<" or ">>" to deal with input/output. But obviously nobody would be insane enough to do that. When I read "a = b + c" in some C++ code I fully expect that "a" will be set to the value of a sum of "b" and "c", whatever makes sense for the types of "b" and "c". If adding "b" and "c" doesn't make any obvious sense semantically (adding database handles with strings) then it shouldn't be done. I think that it's easier to draw the line and design sane guidelines for when you're allowed to implement overloaded operators. Macros have much broader use cases.
    • hyperpallium 9y ago
      > inventing your own little dialect of the language which is opaque to any 3rd party reading your code unless they take the time to unravel your macros. I meant that a whole bunch of specialzed functions can also be a dialect, which are opaque unless one takes the trouble to look into them. You can write very confusing code, unless it matches up closely to a domain (and even then, you need to be on top of the domain). Of course, the "unraveling" is easier, (and less literal) than for macros. I was just trying to say it's a difference in degree not in kind - if you measure in terms of power and usability. I take your point that it's not just syntax, but differences in (basic) guarantees, so there's less ability to treat it as a black box (information hiding within a module). But any module (e.g. a fn) can behave differently from how you expect. tl;dr macros vs fn is a difference in degree, in terms of outcome (i.e. confusion).
• agumonkey 9y ago
  I believe all lispers know this, and don't abuse it. They'll craft a simply-functional metalayer and then drop a few macros for syntactic purposes on top. The whole ordeal being used only if it has more pros than cons. It's an experience based thing.
• nabla9 9y ago
  >..inventing your own little dialect of the language which is opaque to any 3rd party reading your code unless they take the time to unravel your... And that, ladies and gentlemen, is called a framework. Macros or no macros, if the problem domain is large and you stay on it long time, you end up with framework. Macros can make that framework easier, not harder. In most languages you end up with large configuration files and huge number of strings as parameter. The program semantics might be defined by interpreting JSON structures, template languages and ungodly amount of string parameters that are in effect reserved words. If you persist long enough, you have code generators and state machines and define execution semantics with petri nets. What you want to do us use the programming language that is best suited to solving the problems you have. Growing the language towards the problem you have is great opportunity to make the programming interface simpler to understand. With Lisp, especially with Common Lisp it's possible to grow the language so that it stays familiar as much as possible.
  • simias 9y ago
    I don't understand your post at all. Can you give some examples maybe? >In most languages you end up with large configuration files and huge number of strings as parameter. Not in the languages I'm familiar with at least. I don't understand what configuration files and macros have to do with each other. >Growing the language towards the problem you have is great opportunity to make the programming interface simpler to understand. Simpler to use, not to understand. You know what it does, you don't know how it does it or how it can be extended because it doesn't play by the language's usual rules. It's nice if you're copy/pasting stack overflow snippets or your use case is very standard, it sucks if you want to modify the code to do something a bit different and out of the box because now you have to learn the rules of this custom DSL. >With Lisp, especially with Common Lisp it's possible to grow the language so that it stays familiar as much as possible. So adding custom constructs to the language makes it stay familiar? That doesn't make a lot of sense to me
    • jeremiep 9y ago
      Basically you evolve the language into the domain of the application. Rather than lowering the application into the domain of the language. Emacs-lisp is one example of a Lisp that has evolved into the domain of text editing.
• js8 9y ago
  Macros have their problems, but automated code generation is almost always worse. In any case, I would like to point out that there are other answers to the problem than macros. For example, Haskell has lazy evaluation, currying and strong type system, and many use cases for macros can be implemented just as ordinary functions.
• bunderbunder 9y ago
  I don't think that macros are really all that different from the alternatives in this respect. It's also hard to pick apart a pile of generated code, or an overly cute DSL that was implemented without macros. If there's a sin, it's maybe that macros make it a bit too easy to (prematurely) optimize the conciseness of your code, and that becomes an enticement to play code golf.
• wtetzner 9y ago
  > your own little dialect of the language which is opaque to any 3rd party reading your code unless they take the time to unravel your macros. This exact same thing happens with functions, too. At some point, a third party will have to read the code to understand it. You don't know what a function does until you read its implementation either. Often, having a domain specific language (even if the domain is very specific to the application) will make the application easier to understand. If you write code in say, Go, then sure, a third party will be able to look at a loop and be able to figure out the mechanics of what it does. But that person will still not understand how the application works. They will still need to learn both the domain and the structure of the application. And with the right macros, the application will likely be smaller and better organized. Avoiding abstraction doesn't magically make it easier to understand the solution to a complex problem.
• jbeja 9y ago
  That's no a problem of macros, but bad design. You should use macros to build multiple well designed DSLs to reduce as much complexity as possible. Not petty syntax one-liners.