4 ms·

One thing I still miss from this and most other Lisp tutorials is good practical examples of problems you would solve with macros. Its one of the biggest differ

by ufo 13y ago

One thing I still miss from this and most other Lisp tutorials is good practical examples of problems you would solve with macros. Its one of the biggest differences between Lisp and other languages but I have no idea where I should use macros and where I should simply use functions instead. For example, in this tutorial he shows a stopwatch macro but this is precisely the sort of stuff that is easy to do with lambdas: (define stopwatch (body) (start-timer) (body) (end-timer)) (stopwatch (lambda () (my_code 42)))

rayiner 13y ago
Macros unfortunately make themselves most useful in cases that don't lend themselves to short snippets. My favorite example involves looking at some real code. Compare how to add a new intrinsic to LLVM: http://llvm.org/docs/ExtendingLLVM.html http://llvm.org/docs/ExtendingLLVM.html versus how to do it for SBCL: http://pvk.ca/Blog/Lisp/hacking_SSE_intrinsics-part_1.html http://pvk.ca/Blog/Lisp/hacking_SSE_intrinsics-part_1.html. Look at how SBCL's x86-64 backend defines the x86-64 register set and specifies what kind of values can go in what registers: https://github.com/sbcl/sbcl/blob/master/src/compiler/x86-64/vm.lisp https://github.com/sbcl/sbcl/blob/master/src/compiler/x86-64.... Compare that to the equivalent for LLVM: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86RegisterInfo.td?revision=187030&view=markup http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X8.... The register descriptions are written in a very similar, declarative, way, except the SBCL code uses macros built into the language while the LLVM code uses a separate tool called TableGen that generates C++ code from the declarative descriptions. Using a separate tool means complicating the build process, having an ad-hoc parser/expander that can be a source of bugs, losing access to compiler facilities that allow you to step through the expansions and look at the expanded code, etc. If you can't get through the prefix syntax, look at how the OpenDylan compiler uses a macro to define the x86 instruction set: https://github.com/dylan-lang/opendylan/blob/master/sources/harp/x86/instructions.dylan https://github.com/dylan-lang/opendylan/blob/master/sources/... Also see the x86-64 assembler I wrote in Common Lisp: https://github.com/rayiner/amd64-asm/blob/master/encoders.lisp https://github.com/rayiner/amd64-asm/blob/master/encoders.li.... Specifically, lines 374-504, 621+. Each instruction description basically specifies a list of alternative patterns that are matched against the assembly source. When a pattern matches, it is handed off to a compiled function that generates the actual encoded bytes. That function is automatically generated at compile-time from the declarative list of patterns. Not exactly brilliant code like you'll find in SBCL, but in a couple of weekends I was getting bit-for-bit identical assembly to YASM over a substantial subset of the instruction set. Part of the testsuite uses the same instruction descriptions that are used to generate the encoder: https://github.com/rayiner/amd64-asm/blob/master/testsuite.lisp https://github.com/rayiner/amd64-asm/blob/master/testsuite.l... (Line 128).
zck 13y ago
Sure, you can do it that way -- but isn't it ugly? Isn't it harder to read? One thing I recently did that required a lot of macros was writing a unit test framework for Arc, the Lisp Hacker News is written in. (framework, with examples, here: https://bitbucket.org/zck/unit-test.arc https://bitbucket.org/zck/unit-test.arc) So to create a suite, you write: (suite math this-will-pass (assert-same 4 (+ 2 2)) this-will-fail (assert-same 3 (+ 2 2))) Like stopwatch, instead of passing a lambda that runs the body of the unit test, it just lets you put the body in. Why? In c-like languages, you don't write an if statement this way: if((val % 2 == 0), lambda (): { return true; }, lambda (): { return false; }) Isn't it easier to write: if(val %2 == 0) { return true; } else { return false; } So if using lambdas isn't good for if, why is it good for stopwatch? Also, none of math, this-will-pass, or this-will-fail are predefined in the language, or even by my framework. You don't have to use strings, as you might otherwise: (suite "math" "this-will-pass" (assert-same 4 (+ 2 2)) "this-will-fail" (assert-same 3 (+ 2 2))) Why might this be useful? Well, when you're writing a function: int abs(int val){ return val<0 ? -val : val; } Wouldn't you find it annoying to write: int "abs" (int "val"){ return val<0 ? -val : val ; } So Lisp lets you incorporate your code as part of the language, and doesn't force you (as much) to hammer your thoughts into the built-in structures of it.
- ufo 13y ago
  > Sure, you can do it that way -- but isn't it ugly? Isn't it harder to read? Its the old tradeoff between using long-winded existing constructs or paying a complexity cost and forcing the reader to learn a new abstraction or DSL. But I like your example. I imagine the extra lambda or string-quoting noise starts getting significant when you have lots of tiny test cases.
  - zck 13y ago
    >Its the old tradeoff between using long-winded existing constructs or paying a complexity cost and forcing the reader to learn a new abstraction or DSL. Yeah, that's definitely true. When macros are used to add to the language, you're acting, by definition, as a language designer. You can make good or bad choices. I normally think it's good to eliminate boilerplate (e.g., the lambda calls in the examples) and to simplify things (passing bare symbols instead of strings). To my eye, the new coder actually has to learn less than if they had to remember to wrap with lambdas, or use strings: you code just like other parts of the language. And that's a win. However, bad choices with macros can definitely be made. Some people consider the loop macro ugly, because it's a hairy ball of special cases with new syntax.
jamii 13y ago
Most macros fall under one of three categories. Improving performance by removing function calls or doing work at compile time eg https://github.com/ztellman/vertigo https://github.com/ztellman/vertigo emulates c-style structs and uses macros to calculate the correct array offsets at compile-time, so you can treat a byte-array like a rich clojure datastructure but it compiles down to inline array access. Controlling binding or control flow eg https://github.com/clojure/core.match https://github.com/clojure/core.match adds a pattern matching construct similar to those found in eg haskell or ocaml. Rewriting code eg the core.async/go macro mechanically converts synchronous code into js-style callback code (examples - http://swannodette.github.io/2013/08/17/comparative/ http://swannodette.github.io/2013/08/17/comparative/ , internals - http://hueypetersen.com/posts/2013/08/02/the-state-machines-of-core-async/ http://hueypetersen.com/posts/2013/08/02/the-state-machines-...) EDIT: In a lazy language like haskell, you can do most of that with lambdas, monad/arrow syntax and rewrite rules but the dsl syntax often ends up with just as much cognitive overhead as using a macro. Imagine how much less clear pattern matching would be in haskell if it was a library construct instead of having its own syntax.
MichaelSalib 13y ago
Object relational mappers. Or think of anything that in the java world you'd do with an XML file and an external build process, like SOAP APIs or parser generators.

by ufo 13y ago

rayiner 13y ago

Macros unfortunately make themselves most useful in cases that don't lend themselves to short snippets. My favorite example involves looking at some real code. Compare how to add a new intrinsic to LLVM: http://llvm.org/docs/ExtendingLLVM.html http://llvm.org/docs/ExtendingLLVM.html versus how to do it for SBCL: http://pvk.ca/Blog/Lisp/hacking_SSE_intrinsics-part_1.html http://pvk.ca/Blog/Lisp/hacking_SSE_intrinsics-part_1.html. Look at how SBCL's x86-64 backend defines the x86-64 register set and specifies what kind of values can go in what registers: https://github.com/sbcl/sbcl/blob/master/src/compiler/x86-64/vm.lisp https://github.com/sbcl/sbcl/blob/master/src/compiler/x86-64.... Compare that to the equivalent for LLVM: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X86/X86RegisterInfo.td?revision=187030&view=markup http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/X8.... The register descriptions are written in a very similar, declarative, way, except the SBCL code uses macros built into the language while the LLVM code uses a separate tool called TableGen that generates C++ code from the declarative descriptions. Using a separate tool means complicating the build process, having an ad-hoc parser/expander that can be a source of bugs, losing access to compiler facilities that allow you to step through the expansions and look at the expanded code, etc. If you can't get through the prefix syntax, look at how the OpenDylan compiler uses a macro to define the x86 instruction set: https://github.com/dylan-lang/opendylan/blob/master/sources/harp/x86/instructions.dylan https://github.com/dylan-lang/opendylan/blob/master/sources/... Also see the x86-64 assembler I wrote in Common Lisp: https://github.com/rayiner/amd64-asm/blob/master/encoders.lisp https://github.com/rayiner/amd64-asm/blob/master/encoders.li.... Specifically, lines 374-504, 621+. Each instruction description basically specifies a list of alternative patterns that are matched against the assembly source. When a pattern matches, it is handed off to a compiled function that generates the actual encoded bytes. That function is automatically generated at compile-time from the declarative list of patterns. Not exactly brilliant code like you'll find in SBCL, but in a couple of weekends I was getting bit-for-bit identical assembly to YASM over a substantial subset of the instruction set. Part of the testsuite uses the same instruction descriptions that are used to generate the encoder: https://github.com/rayiner/amd64-asm/blob/master/testsuite.lisp https://github.com/rayiner/amd64-asm/blob/master/testsuite.l... (Line 128).

zck 13y ago

Sure, you can do it that way -- but isn't it ugly? Isn't it harder to read? One thing I recently did that required a lot of macros was writing a unit test framework for Arc, the Lisp Hacker News is written in. (framework, with examples, here: https://bitbucket.org/zck/unit-test.arc https://bitbucket.org/zck/unit-test.arc) So to create a suite, you write: (suite math this-will-pass (assert-same 4 (+ 2 2)) this-will-fail (assert-same 3 (+ 2 2))) Like stopwatch, instead of passing a lambda that runs the body of the unit test, it just lets you put the body in. Why? In c-like languages, you don't write an if statement this way: if((val % 2 == 0), lambda (): { return true; }, lambda (): { return false; }) Isn't it easier to write: if(val %2 == 0) { return true; } else { return false; } So if using lambdas isn't good for if, why is it good for stopwatch? Also, none of math, this-will-pass, or this-will-fail are predefined in the language, or even by my framework. You don't have to use strings, as you might otherwise: (suite "math" "this-will-pass" (assert-same 4 (+ 2 2)) "this-will-fail" (assert-same 3 (+ 2 2))) Why might this be useful? Well, when you're writing a function: int abs(int val){ return val<0 ? -val : val; } Wouldn't you find it annoying to write: int "abs" (int "val"){ return val<0 ? -val : val ; } So Lisp lets you incorporate your code as part of the language, and doesn't force you (as much) to hammer your thoughts into the built-in structures of it.

ufo 13y ago
> Sure, you can do it that way -- but isn't it ugly? Isn't it harder to read? Its the old tradeoff between using long-winded existing constructs or paying a complexity cost and forcing the reader to learn a new abstraction or DSL. But I like your example. I imagine the extra lambda or string-quoting noise starts getting significant when you have lots of tiny test cases.
- zck 13y ago
  >Its the old tradeoff between using long-winded existing constructs or paying a complexity cost and forcing the reader to learn a new abstraction or DSL. Yeah, that's definitely true. When macros are used to add to the language, you're acting, by definition, as a language designer. You can make good or bad choices. I normally think it's good to eliminate boilerplate (e.g., the lambda calls in the examples) and to simplify things (passing bare symbols instead of strings). To my eye, the new coder actually has to learn less than if they had to remember to wrap with lambdas, or use strings: you code just like other parts of the language. And that's a win. However, bad choices with macros can definitely be made. Some people consider the loop macro ugly, because it's a hairy ball of special cases with new syntax.

jamii 13y ago

Most macros fall under one of three categories. Improving performance by removing function calls or doing work at compile time eg https://github.com/ztellman/vertigo https://github.com/ztellman/vertigo emulates c-style structs and uses macros to calculate the correct array offsets at compile-time, so you can treat a byte-array like a rich clojure datastructure but it compiles down to inline array access. Controlling binding or control flow eg https://github.com/clojure/core.match https://github.com/clojure/core.match adds a pattern matching construct similar to those found in eg haskell or ocaml. Rewriting code eg the core.async/go macro mechanically converts synchronous code into js-style callback code (examples - http://swannodette.github.io/2013/08/17/comparative/ http://swannodette.github.io/2013/08/17/comparative/ , internals - http://hueypetersen.com/posts/2013/08/02/the-state-machines-of-core-async/ http://hueypetersen.com/posts/2013/08/02/the-state-machines-...) EDIT: In a lazy language like haskell, you can do most of that with lambdas, monad/arrow syntax and rewrite rules but the dsl syntax often ends up with just as much cognitive overhead as using a macro. Imagine how much less clear pattern matching would be in haskell if it was a library construct instead of having its own syntax.

MichaelSalib 13y ago

Object relational mappers. Or think of anything that in the java world you'd do with an XML file and an external build process, like SOAP APIs or parser generators.