5 ms·

Project for bringing macros to Scala

39 points by mike_esspe 15y ago

nwjsmith 15y ago
I'm not that familiar with Scala, but the examples aren't very convincing. Why must `printf` be defined as a macro? Why not a plain old function? What are the benefits to the second example? The other thing that bothers me is the empty 'Use cases' page. Scala macros might be a solution to a problem nobody has. A problem that people _do_ have is the breadth of Scala language, and macros will add to the conceptual dog pile.
- mike_esspe 15y ago
  You can make printf type safe with macro, so it will complain at compile time if you are passing incorrect type.
- bickfordb 15y ago
  This lets you embed programs in other languages (DSLs) within your program that are parsed and compiled at compile time instead of run-time. This has the following benefits over run-time DSLs / languages embedded inside of strings: 1. The embedded language will be type checked at compile time. For example a regular expression macro could enforce that your regular expression has balanced parentheses in capture groups. This potentially lets you shift a class of errors from run-time to compile time. 2. The language will be parsed at compile time saving run-time cycles. 3. The host language can be extended and experimented with without adding new syntax. Features like this actually limit the growth of the "dog pile", since many language experiments features can be tested as macros first. Some common DSL examples are regular expressions, parser generators, SQL / ORM generators and LINQ
- bad_user 15y ago
  An impressive example of what you could do with macros is LINQ - a static and type-safe DSL for querying collections of any kind. People did impressive things with it, like offloading the work to the GPU. Another example would be about adding your own improvements to the time-system, improvements that are actually going to help you keep the code safe. Like the units of measure from F#. However, adding powerful macros to a language that was not designed for macros and that is NOT simple ... I do think it is a bad idea. I'd be happy to see in Scala something like the code quotations from F#. Anything less is useless, anything more will make Scala way too complicated.
- dxbydt 15y ago
  > Scala macros might be a solution to a problem nobody has Pls read slide 10 of this document - http://scalamacros.org/talks/2011-10-18-WhatsUp.pdf http://scalamacros.org/talks/2011-10-18-WhatsUp.pdf --Proposed Code snippet with macro --- class Queryable[T, Repr](query: Query) { macro def filter(p: T => Boolean): Repr = <[ val b = $this.newBuilder b.query = Filter($this.query, $reify(p)) b.result ]> } val products = db.products products.filter(p => p.startsWith("foo")).toList -- My takeaways : 1. adding LINQ support to Scala via macros 2. macros virtualize XXX, where XXX = vals, vars, defs, types, classes, traits. So macro XXX can be used in place of XXX. The author also says "We can tailor macros to fit your particular use case". He mentions OR-mapping, and a Scala-to-javascript compiler that might end up using these macro invocations, among other things.
- scalamacros 15y ago
  Thanks for pointing out the weak spots of the content! We've posted several compelling use cases enabled by macros (http://scalamacros.org/usecases/index.html http://scalamacros.org/usecases/index.html) and added some more information to the main page that uncovers the inner beauty of the examples. Btw, we realize the responsibility of adding a competely new feature to Scala, so every new macro-related idea is being weighed against the complexity it brings to the table. So far we've done away with several really interesting prospects in order to keep the overall proposal simple.
HnNoPassMailer 15y ago
I am an average programmer, I know C#, Java and dart. Can anyone explain macro's? Wikipedia/internet is not very helpfull because of ambiguity. Thanks!
- sreque 15y ago
  We're all average programmers here that know C#, Java, and/or Dart, but here goes! A macro, in its simplest form, is a function of type syntax -> syntax, or, a function that takes syntax as input and produces syntax as output. These functions are invoked at "compile time" as the compiler is processing your source code, and the resulting code is then compiled or interpreted along with the rest of the code. The way you write these functions, what these functions are capable of doing, and even the precise data type of syntax varies between the languages that implement macros. The most powerful macro systems allow you to "extend" your language with new syntactic constructs and features. In this sense you can think of macros as a way for you to team up with the compiler to improve the language. Macros also allow you to generate repetitive code that would be tedious or error-prone to hand-write and maintain on your own. The easiest way to understand good macro systems would be to learn about them from a Lisp-variant like Clojure, Scheme, or Common Lisp, as macros have typically been the domain of these languages. It is still an active area of language design research to try to effectively integrate a good macro system into non-Lisp languages. Scala's macro proposal is supposedly heavily inspired by a language called Nemerle, which runs on the .NET platform. I don't know much about Nemerle, but it appears to be innovating very well in this particular space.
  - nickik 15y ago
    The Dylan language (witch had Lisp syntax at first)probebly had one of the early macro-systems in a language with syntax. Its ok to work with but much harder then a normal lisp. One of the designers of Dylan tried to push this in his new language called PLOT (Programming Language for Old Timers). It has pythonic syntax and provids a full powerd macro-system. Unfortently there is no implmentation of PLOT just a discription: http://users.rcn.com/david-moon/PLOT/ http://users.rcn.com/david-moon/PLOT/
- Dn_Ab 15y ago
bborud 15y ago
Macros are bad, but in order to understand why they are bad it helps if you are not a language nut. Most language nuts have one shortcoming: they do not understand their consumer. If you want proof you only need to look at the error messages generated by most compilers. They probably make perfect sense to someone who is trying to debug the compiler, but they usually leave the consumer, ie the programmer using the compiler, somewhat confused. Macros are bad because they redefine the language. The point of having a language in the first place is to make it easy to communicate. This is only possible if the semantics are understood. If you do good language design the semantics are clear. If you do sloppy language design people have to look things up all the time. Now, if everyone gets to extend the language in arbitrary ways, you have suddenly unloaded a massive burden onto your programmers. It is no longer enough to say that you know language X. In order to understand the software you have to understand the specific dialects (plural) of language X -- because for a non-trivial project with lots of people extending the language in various ways, you will end up with multiple dialects. When you can no longer bring up a piece of code in your editor and see what it does without having to reference dozens of other files that may modify the semantics of the language (sometimes in non-obvious ways), you are going to be a lot less productive. I think Joshua Bloch was absolutely right when he said that one of the really good decisions of Java was to NOT have macros. He made some of the same points I made above. Now I understand that people think macros can lead to elegant code, but that code only has value if other people can understand it. If you need to re-learn the language, in big or small ways, to understand it, its value is very limited. And again: what excites language nuts and what has real value is not always the same thing.
- dman 15y ago
  You make some valid points, but they would be better made if you toned down your rhetoric about people who work on programming languages.
  - bborud 15y ago
    I'm an engineer at heart. One of the hardest things that I have to do on a daily basis is to ask myself where what I think is cool conflicts with what the user needs. Or indeed if my cool idea will harm the user. As engineers we tend to focus on the cool bit. I too have fallen into that trap and I expect to do it many times in the future. But I try to be aware that this can and will happen. I think asking language nuts (which isn't limited to those who write compilers, but also extends to practitioners with deep knowlede) to be aware of this is particularly important in that people look to them for enlightenment and advice -- and when they do harmful things it has a huge multiplier effect.
wbhart 15y ago
This is a significant breakthrough. Scala is once again showing the way forward. Instead of having ad hoc macros (which everyone knows are bad), Scala macros don't change the syntax of the language. Thus they are no more difficult to come to terms with than an API for a well-written library. In this sense ordinary API's are going to start looking like crippled Scala. The other nice thing about these is that you don't need to lose type safety to use macros. That's really where the innovation is here. I don't yet fully understand how this all works at the type-theoretic level. I'm imagining that expressions (as opposed to the values they represent) are thought of as belonging to higher kinded types. The macros then reject expression which aren't of the right kind. I have to say, I am absolutely flawed by the comments so far about this. I think this is going to have an absolutely pivotal effect on future programming languages. This is how macros should always have been. I don't know of any other language which does this. It's a totally new area in language theory to explore.
- draegtun 15y ago
  This is how macros should always have been. I don't know of any other language which does this They look very similar to Perl6 macros. See recent discussion here: http://news.ycombinator.com/item?id=3124920 http://news.ycombinator.com/item?id=3124920
losethos 15y ago
I am against macros. They are not allowed in LoseThos.

39 points by mike_esspe 15y ago

I'm not that familiar with Scala, but the examples aren't very convincing. Why must `printf` be defined as a macro? Why not a plain old function? What are the benefits to the second example? The other thing that bothers me is the empty 'Use cases' page. Scala macros might be a solution to a problem nobody has. A problem that people _do_ have is the breadth of Scala language, and macros will add to the conceptual dog pile.

mike_esspe 15y ago
You can make printf type safe with macro, so it will complain at compile time if you are passing incorrect type.
bickfordb 15y ago
This lets you embed programs in other languages (DSLs) within your program that are parsed and compiled at compile time instead of run-time. This has the following benefits over run-time DSLs / languages embedded inside of strings: 1. The embedded language will be type checked at compile time. For example a regular expression macro could enforce that your regular expression has balanced parentheses in capture groups. This potentially lets you shift a class of errors from run-time to compile time. 2. The language will be parsed at compile time saving run-time cycles. 3. The host language can be extended and experimented with without adding new syntax. Features like this actually limit the growth of the "dog pile", since many language experiments features can be tested as macros first. Some common DSL examples are regular expressions, parser generators, SQL / ORM generators and LINQ
bad_user 15y ago
An impressive example of what you could do with macros is LINQ - a static and type-safe DSL for querying collections of any kind. People did impressive things with it, like offloading the work to the GPU. Another example would be about adding your own improvements to the time-system, improvements that are actually going to help you keep the code safe. Like the units of measure from F#. However, adding powerful macros to a language that was not designed for macros and that is NOT simple ... I do think it is a bad idea. I'd be happy to see in Scala something like the code quotations from F#. Anything less is useless, anything more will make Scala way too complicated.
dxbydt 15y ago
> Scala macros might be a solution to a problem nobody has Pls read slide 10 of this document - http://scalamacros.org/talks/2011-10-18-WhatsUp.pdf http://scalamacros.org/talks/2011-10-18-WhatsUp.pdf --Proposed Code snippet with macro --- class Queryable[T, Repr](query: Query) { macro def filter(p: T => Boolean): Repr = <[ val b = $this.newBuilder b.query = Filter($this.query, $reify(p)) b.result ]> } val products = db.products products.filter(p => p.startsWith("foo")).toList -- My takeaways : 1. adding LINQ support to Scala via macros 2. macros virtualize XXX, where XXX = vals, vars, defs, types, classes, traits. So macro XXX can be used in place of XXX. The author also says "We can tailor macros to fit your particular use case". He mentions OR-mapping, and a Scala-to-javascript compiler that might end up using these macro invocations, among other things.
scalamacros 15y ago
Thanks for pointing out the weak spots of the content! We've posted several compelling use cases enabled by macros (http://scalamacros.org/usecases/index.html http://scalamacros.org/usecases/index.html) and added some more information to the main page that uncovers the inner beauty of the examples. Btw, we realize the responsibility of adding a competely new feature to Scala, so every new macro-related idea is being weighed against the complexity it brings to the table. So far we've done away with several really interesting prospects in order to keep the overall proposal simple.

HnNoPassMailer 15y ago

I am an average programmer, I know C#, Java and dart. Can anyone explain macro's? Wikipedia/internet is not very helpfull because of ambiguity. Thanks!

sreque 15y ago
We're all average programmers here that know C#, Java, and/or Dart, but here goes! A macro, in its simplest form, is a function of type syntax -> syntax, or, a function that takes syntax as input and produces syntax as output. These functions are invoked at "compile time" as the compiler is processing your source code, and the resulting code is then compiled or interpreted along with the rest of the code. The way you write these functions, what these functions are capable of doing, and even the precise data type of syntax varies between the languages that implement macros. The most powerful macro systems allow you to "extend" your language with new syntactic constructs and features. In this sense you can think of macros as a way for you to team up with the compiler to improve the language. Macros also allow you to generate repetitive code that would be tedious or error-prone to hand-write and maintain on your own. The easiest way to understand good macro systems would be to learn about them from a Lisp-variant like Clojure, Scheme, or Common Lisp, as macros have typically been the domain of these languages. It is still an active area of language design research to try to effectively integrate a good macro system into non-Lisp languages. Scala's macro proposal is supposedly heavily inspired by a language called Nemerle, which runs on the .NET platform. I don't know much about Nemerle, but it appears to be innovating very well in this particular space.
- nickik 15y ago
  The Dylan language (witch had Lisp syntax at first)probebly had one of the early macro-systems in a language with syntax. Its ok to work with but much harder then a normal lisp. One of the designers of Dylan tried to push this in his new language called PLOT (Programming Language for Old Timers). It has pythonic syntax and provids a full powerd macro-system. Unfortently there is no implmentation of PLOT just a discription: http://users.rcn.com/david-moon/PLOT/ http://users.rcn.com/david-moon/PLOT/
Dn_Ab 15y ago

bborud 15y ago

Macros are bad, but in order to understand why they are bad it helps if you are not a language nut. Most language nuts have one shortcoming: they do not understand their consumer. If you want proof you only need to look at the error messages generated by most compilers. They probably make perfect sense to someone who is trying to debug the compiler, but they usually leave the consumer, ie the programmer using the compiler, somewhat confused. Macros are bad because they redefine the language. The point of having a language in the first place is to make it easy to communicate. This is only possible if the semantics are understood. If you do good language design the semantics are clear. If you do sloppy language design people have to look things up all the time. Now, if everyone gets to extend the language in arbitrary ways, you have suddenly unloaded a massive burden onto your programmers. It is no longer enough to say that you know language X. In order to understand the software you have to understand the specific dialects (plural) of language X -- because for a non-trivial project with lots of people extending the language in various ways, you will end up with multiple dialects. When you can no longer bring up a piece of code in your editor and see what it does without having to reference dozens of other files that may modify the semantics of the language (sometimes in non-obvious ways), you are going to be a lot less productive. I think Joshua Bloch was absolutely right when he said that one of the really good decisions of Java was to NOT have macros. He made some of the same points I made above. Now I understand that people think macros can lead to elegant code, but that code only has value if other people can understand it. If you need to re-learn the language, in big or small ways, to understand it, its value is very limited. And again: what excites language nuts and what has real value is not always the same thing.

dman 15y ago
You make some valid points, but they would be better made if you toned down your rhetoric about people who work on programming languages.
- bborud 15y ago
  I'm an engineer at heart. One of the hardest things that I have to do on a daily basis is to ask myself where what I think is cool conflicts with what the user needs. Or indeed if my cool idea will harm the user. As engineers we tend to focus on the cool bit. I too have fallen into that trap and I expect to do it many times in the future. But I try to be aware that this can and will happen. I think asking language nuts (which isn't limited to those who write compilers, but also extends to practitioners with deep knowlede) to be aware of this is particularly important in that people look to them for enlightenment and advice -- and when they do harmful things it has a huge multiplier effect.

wbhart 15y ago

This is a significant breakthrough. Scala is once again showing the way forward. Instead of having ad hoc macros (which everyone knows are bad), Scala macros don't change the syntax of the language. Thus they are no more difficult to come to terms with than an API for a well-written library. In this sense ordinary API's are going to start looking like crippled Scala. The other nice thing about these is that you don't need to lose type safety to use macros. That's really where the innovation is here. I don't yet fully understand how this all works at the type-theoretic level. I'm imagining that expressions (as opposed to the values they represent) are thought of as belonging to higher kinded types. The macros then reject expression which aren't of the right kind. I have to say, I am absolutely flawed by the comments so far about this. I think this is going to have an absolutely pivotal effect on future programming languages. This is how macros should always have been. I don't know of any other language which does this. It's a totally new area in language theory to explore.

draegtun 15y ago
This is how macros should always have been. I don't know of any other language which does this They look very similar to Perl6 macros. See recent discussion here: http://news.ycombinator.com/item?id=3124920 http://news.ycombinator.com/item?id=3124920

losethos 15y ago

I am against macros. They are not allowed in LoseThos.