4 ms·

Don't think of it as modifying the language. Think of it as adding new ways of expressing ideas in the existing language. These sorts of languages/language ext

Don't think of it as modifying the language. Think of it as adding new ways of expressing ideas in the existing language. These sorts of languages/language extensions pop up all the time in "ordinary" programming. For example, common DSLs we see often. - regex is a language for string patterns - JSON is a language for simple nested data - SQL is a language for expressing DB queries - OpenMP contains (essentially) a language for parallelizing loops - (this might sound crazy because it's so common and basically built in standard in almost all programming languages:) infix notation for math The problem is that many of these are absolute kludges in existing languages. Anybody who has had the displeasure of trying to interpolate DB table names into SQL SELECT strings knows this. Most languages try to make it a bit better by adding preprocessors, using builder-patterns, abusing reflection, or otherwise. Looking under the hood of some languages' implementations of these ideas will often lead you to see the most cursed of programmer things in existence. Each of those languages (or close syntactic cousins of them) can be written directly in Lisp, as a library, without shoving them into strings, or using external tools, or making custom file formats. They call can exist as a part of the Lisp language as a syntax extension. Making DSLs requires exercising that muscle a bit; it's non-trivial to design a new and useful language that fits orthogonally into an existing system. It's like being introduced to OO programming where, at least for me, it made sense but it wasn't immediately obvious what a good object model for a problem might be. It took me lots and trial and error to understand how OO works in "the real world", where it's good, and where it falls short. Once you get they hang of making DSLs—or even just small but powerful syntax extensions—it's an amazing tool to have at your disposal.

michaelwww 4y ago
My second thought, which I excluded for brevity, was aren't functions and libraries language extensions? I mean technically you could write C without malloc, but it wouldn't be very useful. Regex is a great example of an embedded DSL, but I've never thought "I wish I could write a macro to change it's behavior on the fly in this special case". The last time I looked at this issue I saw that sure, if you don't have generics, it would be helpful to have a language that could write typed functions for you. That makes sense. Changing the parser and compiler behavior before any runtime code executes could be very interesting. There are certainly enough people waxing poetic about the expressiveness of it to make it worth learning.
- reikonomusha 4y ago
  Again, you don't want to change regex, you want to create it in the first place. Maybe a simpler example, what if I wanted this in (pseudo-)C? with(s = fopen("/foo/bar", ...)) { ...; ...; println(readln(s)); ...; ...; } When the "with" clause ends, the file will automatically be closed. This is more or less bog standard RAII expressed as a new language construct. In Lisp, this is a feature you could build and provide as a library feature. In fact, we can do it, straight here in an HN comment. (defgeneric destroy (x) (:documentation "A destructor for use with the WITH feature".)) (defmacro with ((var = val) &body body) "Python-style RAII macro. VAR will be bound to the value of VAL, and upon exiting scope, will be destroyed with DESTROY." (assert (eq = ':=)) (let ((orig (gensym))) `(let* ((,orig ,val) (,var ,orig)) (unwind-protect (progn ,@body) (destroy ,orig)))) We can register a new destructor easily for open file streams: (defmethod destroy ((s stream)) (close stream)) (We could add some more error handling, etc. to make it more robust.) Now, we effectively have RAII in Lisp: (with (s := (open "/foo/bar" ...)) ... ... (write-line (read-line s)) ... ...) This is now real Lisp code you can actually run, completely integrated into the language, and nicely orthogonal to the existing gamut of features. This is a concept we can use almost immediately after learning about it, instead of waiting for Python's PEP 6363 to pass with consensus from people who may have different programming goals than you do.
  - michaelwww 4y ago
    I can see a lot of uses for macros, but in this case C# has a "using" keyword and the designers have put a lot of thought into it and how it is called in a wide variety of situations. Lisp was a language developed in a world when people worked on their own cars, like me. I would have loved Lisp if I had found it earlier, like about the time I rebuilt my VW engine in the 80's.
    - deleted 4y ago
      [deleted]
    - 4y ago
eru 4y ago
Extending a language with macros isn't all that different from extending a language with your own functions. At least in principle. And a lot of what tends to be macros in Lisp turns into functions in eg Haskell. Eg you can define your own new branching and looping constructs (like 'if' or 'while') etc as functions in Haskell, but in Lisp these need to be macros.
- reikonomusha 4y ago
  Unlike call-by-need evaluation, macros provide the opportunity to compute things at compile time (think lookup tables) or analyze the arguments at compile time (think DB query syntax checking). In Lisp, you can also define branching and looping as functions, but you need to concede and have the arguments themselves be (lambda) functions. (defun while (c body) (when (funcall c) (funcall body) (while c body)))
  - eru 4y ago
    Yes, I know that laziness doesn't take care of all macros. That's why even Haskell has macros. Yes, if you wrap your arguments in lambdas, you can write loops like that. Though to be as performant as the built-in constructs, you need a rather clever compiler that 'removes' the lambdas again.

michaelwww 4y ago

My second thought, which I excluded for brevity, was aren't functions and libraries language extensions? I mean technically you could write C without malloc, but it wouldn't be very useful. Regex is a great example of an embedded DSL, but I've never thought "I wish I could write a macro to change it's behavior on the fly in this special case". The last time I looked at this issue I saw that sure, if you don't have generics, it would be helpful to have a language that could write typed functions for you. That makes sense. Changing the parser and compiler behavior before any runtime code executes could be very interesting. There are certainly enough people waxing poetic about the expressiveness of it to make it worth learning.

reikonomusha 4y ago
Again, you don't want to change regex, you want to create it in the first place. Maybe a simpler example, what if I wanted this in (pseudo-)C? with(s = fopen("/foo/bar", ...)) { ...; ...; println(readln(s)); ...; ...; } When the "with" clause ends, the file will automatically be closed. This is more or less bog standard RAII expressed as a new language construct. In Lisp, this is a feature you could build and provide as a library feature. In fact, we can do it, straight here in an HN comment. (defgeneric destroy (x) (:documentation "A destructor for use with the WITH feature".)) (defmacro with ((var = val) &body body) "Python-style RAII macro. VAR will be bound to the value of VAL, and upon exiting scope, will be destroyed with DESTROY." (assert (eq = ':=)) (let ((orig (gensym))) `(let* ((,orig ,val) (,var ,orig)) (unwind-protect (progn ,@body) (destroy ,orig)))) We can register a new destructor easily for open file streams: (defmethod destroy ((s stream)) (close stream)) (We could add some more error handling, etc. to make it more robust.) Now, we effectively have RAII in Lisp: (with (s := (open "/foo/bar" ...)) ... ... (write-line (read-line s)) ... ...) This is now real Lisp code you can actually run, completely integrated into the language, and nicely orthogonal to the existing gamut of features. This is a concept we can use almost immediately after learning about it, instead of waiting for Python's PEP 6363 to pass with consensus from people who may have different programming goals than you do.
- michaelwww 4y ago
  I can see a lot of uses for macros, but in this case C# has a "using" keyword and the designers have put a lot of thought into it and how it is called in a wide variety of situations. Lisp was a language developed in a world when people worked on their own cars, like me. I would have loved Lisp if I had found it earlier, like about the time I rebuilt my VW engine in the 80's.
  - deleted 4y ago
    [deleted]
  - 4y ago

eru 4y ago

Extending a language with macros isn't all that different from extending a language with your own functions. At least in principle. And a lot of what tends to be macros in Lisp turns into functions in eg Haskell. Eg you can define your own new branching and looping constructs (like 'if' or 'while') etc as functions in Haskell, but in Lisp these need to be macros.

reikonomusha 4y ago
Unlike call-by-need evaluation, macros provide the opportunity to compute things at compile time (think lookup tables) or analyze the arguments at compile time (think DB query syntax checking). In Lisp, you can also define branching and looping as functions, but you need to concede and have the arguments themselves be (lambda) functions. (defun while (c body) (when (funcall c) (funcall body) (while c body)))
- eru 4y ago
  Yes, I know that laziness doesn't take care of all macros. That's why even Haskell has macros. Yes, if you wrap your arguments in lambdas, you can write loops like that. Though to be as performant as the built-in constructs, you need a rather clever compiler that 'removes' the lambdas again.